maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/vector-shift-ashr-256.ll

1933 lines
98 KiB
LLVM
Raw Normal View History

Make utils/update_llc_test_checks.py note that the assertions are autogenerated. Also update existing test cases which appear to be generated by it and weren't modified (other than addition of the header) by rerunning it. llvm-svn: 253917
2015-11-24 05:33:58 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[X86][SSE] shift/rotate tests - remove unnecessary mcpu arguments and regenerate/cleanup llvm-svn: 251232
2015-10-25 20:07:45 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX1
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx2 | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX2
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+xop,+avx | FileCheck %s --check-prefix=ALL --check-prefix=XOP --check-prefix=XOPAVX1
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+xop,+avx2 | FileCheck %s --check-prefix=ALL --check-prefix=XOP --check-prefix=XOPAVX2
[X86][AVX512] Use AVX512 attributes instead of -mcpu in vector shift tests llvm-svn: 312529
2017-09-05 20:23:45 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq | FileCheck %s --check-prefix=ALL --check-prefix=AVX512 --check-prefix=AVX512DQ
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512bw | FileCheck %s --check-prefix=ALL --check-prefix=AVX512 --check-prefix=AVX512BW
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,+avx512vl | FileCheck %s --check-prefix=ALL --check-prefix=AVX512VL --check-prefix=AVX512DQVL
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512bw,+avx512vl | FileCheck %s --check-prefix=ALL --check-prefix=AVX512VL --check-prefix=AVX512BWVL
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; 32-bit runs to make sure we do reasonable things for i64 shifts.
; RUN: llc < %s -mtriple=i686-unknown-unknown -mattr=+avx | FileCheck %s --check-prefix=ALL --check-prefix=X32-AVX --check-prefix=X32-AVX1
; RUN: llc < %s -mtriple=i686-unknown-unknown -mattr=+avx2 | FileCheck %s --check-prefix=ALL --check-prefix=X32-AVX --check-prefix=X32-AVX2
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
;
; Variable Shifts
;
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <4 x i64> @var_shift_v4i64(<4 x i64> %a, <4 x i64> %b) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: var_shift_v4i64:
; AVX1: # BB#0:
[X86][SSE] Vectorize i64 ASHR operations This patch vectorizes the v2i64/v4i64 ASHR shift operations - the last remaining integer vector shifts that are still being transferred to/from the scalar unit to be completed. Differential Revision: http://reviews.llvm.org/D11439 llvm-svn: 243569
2015-07-30 04:31:45 +08:00
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [9223372036854775808,9223372036854775808]
; AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm4
; AVX1-NEXT: vpshufd {{.*#+}} xmm5 = xmm2[2,3,0,1]
; AVX1-NEXT: vpsrlq %xmm5, %xmm3, %xmm6
; AVX1-NEXT: vpblendw {{.*#+}} xmm4 = xmm4[0,1,2,3],xmm6[4,5,6,7]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm6
; AVX1-NEXT: vpsrlq %xmm2, %xmm6, %xmm2
; AVX1-NEXT: vpsrlq %xmm5, %xmm6, %xmm5
; AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm2[0,1,2,3],xmm5[4,5,6,7]
; AVX1-NEXT: vpxor %xmm4, %xmm2, %xmm2
; AVX1-NEXT: vpsubq %xmm4, %xmm2, %xmm2
; AVX1-NEXT: vpsrlq %xmm1, %xmm3, %xmm4
; AVX1-NEXT: vpshufd {{.*#+}} xmm5 = xmm1[2,3,0,1]
; AVX1-NEXT: vpsrlq %xmm5, %xmm3, %xmm3
; AVX1-NEXT: vpblendw {{.*#+}} xmm3 = xmm4[0,1,2,3],xmm3[4,5,6,7]
; AVX1-NEXT: vpsrlq %xmm1, %xmm0, %xmm1
; AVX1-NEXT: vpsrlq %xmm5, %xmm0, %xmm0
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm1[0,1,2,3],xmm0[4,5,6,7]
; AVX1-NEXT: vpxor %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpsubq %xmm3, %xmm0, %xmm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: var_shift_v4i64:
; AVX2: # BB#0:
[X86][AVX] Regenerate shift test to show constant broadcast comment llvm-svn: 308323
2017-07-19 00:07:12 +08:00
; AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808]
[X86][SSE] Vectorize i64 ASHR operations This patch vectorizes the v2i64/v4i64 ASHR shift operations - the last remaining integer vector shifts that are still being transferred to/from the scalar unit to be completed. Differential Revision: http://reviews.llvm.org/D11439 llvm-svn: 243569
2015-07-30 04:31:45 +08:00
; AVX2-NEXT: vpsrlvq %ymm1, %ymm2, %ymm3
; AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; AVX2-NEXT: vpsrlvq %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpsubq %ymm3, %ymm0, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: var_shift_v4i64:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
; XOPAVX1-NEXT: vpsubq %xmm2, %xmm3, %xmm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; XOPAVX1-NEXT: vpshaq %xmm2, %xmm4, %xmm2
; XOPAVX1-NEXT: vpsubq %xmm1, %xmm3, %xmm1
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: var_shift_v4i64:
; XOPAVX2: # BB#0:
[X86][AVX] Regenerate shift test to show constant broadcast comment llvm-svn: 308323
2017-07-19 00:07:12 +08:00
; XOPAVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808]
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
; XOPAVX2-NEXT: vpsrlvq %ymm1, %ymm2, %ymm3
; XOPAVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; XOPAVX2-NEXT: vpsrlvq %ymm1, %ymm0, %ymm0
; XOPAVX2-NEXT: vpsubq %ymm3, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: var_shift_v4i64:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
[X86][AVX512] Add support for variable ASHR v2i64/v4i64 support without VLX Use v8i64 variable ASHR instructions if we don't have VLX. This is a reduced version of D28537 that just adds support for variable shifts - I'll continue with that patch (for just constant/uniform shifts) once I've fixed the type legalization issue in avx512-cvt.ll. Differential Revision: https://reviews.llvm.org/D28604 llvm-svn: 291901
2017-01-13 21:16:19 +08:00
; AVX512-NEXT: # kill: %YMM1<def> %YMM1<kill> %ZMM1<def>
; AVX512-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<def>
; AVX512-NEXT: vpsravq %zmm1, %zmm0, %zmm0
; AVX512-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<kill>
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: var_shift_v4i64:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsravq %ymm1, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: var_shift_v4i64:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; X32-AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [0,2147483648,0,2147483648]
; X32-AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm4
; X32-AVX1-NEXT: vpshufd {{.*#+}} xmm5 = xmm2[2,3,0,1]
; X32-AVX1-NEXT: vpsrlq %xmm5, %xmm3, %xmm6
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm4 = xmm4[0,1,2,3],xmm6[4,5,6,7]
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm6
; X32-AVX1-NEXT: vpsrlq %xmm2, %xmm6, %xmm2
; X32-AVX1-NEXT: vpsrlq %xmm5, %xmm6, %xmm5
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm2[0,1,2,3],xmm5[4,5,6,7]
; X32-AVX1-NEXT: vpxor %xmm4, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsubq %xmm4, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsrlq %xmm1, %xmm3, %xmm4
; X32-AVX1-NEXT: vpshufd {{.*#+}} xmm5 = xmm1[2,3,0,1]
; X32-AVX1-NEXT: vpsrlq %xmm5, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm3 = xmm4[0,1,2,3],xmm3[4,5,6,7]
; X32-AVX1-NEXT: vpsrlq %xmm1, %xmm0, %xmm1
; X32-AVX1-NEXT: vpsrlq %xmm5, %xmm0, %xmm0
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm1[0,1,2,3],xmm0[4,5,6,7]
; X32-AVX1-NEXT: vpxor %xmm3, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsubq %xmm3, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: var_shift_v4i64:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vmovdqa {{.*#+}} ymm2 = [0,2147483648,0,2147483648,0,2147483648,0,2147483648]
; X32-AVX2-NEXT: vpsrlvq %ymm1, %ymm2, %ymm3
; X32-AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsrlvq %ymm1, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsubq %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <4 x i64> %a, %b
ret <4 x i64> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <8 x i32> @var_shift_v8i32(<8 x i32> %a, <8 x i32> %b) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: var_shift_v8i32:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm3
[X86][SSE] Vectorized v4i32 non-uniform shifts. While the v4i32 shl operation is already vectorized using a cvttps2dq/pmulld pattern, the lshr/ashr opeations are still scalarized. This patch adds vectorization support for non-uniform v4i32 shift operations - it splats constant shift amounts to allow them to use the immediate sse shift instructions, or extracts/zero-extends non-constant shift amounts. The individual results are then blended together. Differential Revision: http://reviews.llvm.org/D11063 llvm-svn: 241989
2015-07-12 19:15:19 +08:00
; AVX1-NEXT: vpsrldq {{.*#+}} xmm4 = xmm3[12,13,14,15],zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero
; AVX1-NEXT: vpsrad %xmm4, %xmm2, %xmm4
; AVX1-NEXT: vpsrlq $32, %xmm3, %xmm5
; AVX1-NEXT: vpsrad %xmm5, %xmm2, %xmm5
; AVX1-NEXT: vpblendw {{.*#+}} xmm4 = xmm5[0,1,2,3],xmm4[4,5,6,7]
; AVX1-NEXT: vpxor %xmm5, %xmm5, %xmm5
; AVX1-NEXT: vpunpckhdq {{.*#+}} xmm6 = xmm3[2],xmm5[2],xmm3[3],xmm5[3]
; AVX1-NEXT: vpsrad %xmm6, %xmm2, %xmm6
; AVX1-NEXT: vpmovzxdq {{.*#+}} xmm3 = xmm3[0],zero,xmm3[1],zero
; AVX1-NEXT: vpsrad %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm2[0,1,2,3],xmm6[4,5,6,7]
; AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm2[0,1],xmm4[2,3],xmm2[4,5],xmm4[6,7]
; AVX1-NEXT: vpsrldq {{.*#+}} xmm3 = xmm1[12,13,14,15],zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero
; AVX1-NEXT: vpsrad %xmm3, %xmm0, %xmm3
; AVX1-NEXT: vpsrlq $32, %xmm1, %xmm4
; AVX1-NEXT: vpsrad %xmm4, %xmm0, %xmm4
; AVX1-NEXT: vpblendw {{.*#+}} xmm3 = xmm4[0,1,2,3],xmm3[4,5,6,7]
; AVX1-NEXT: vpunpckhdq {{.*#+}} xmm4 = xmm1[2],xmm5[2],xmm1[3],xmm5[3]
; AVX1-NEXT: vpsrad %xmm4, %xmm0, %xmm4
; AVX1-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
; AVX1-NEXT: vpsrad %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm4[4,5,6,7]
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm3[2,3],xmm0[4,5],xmm3[6,7]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: var_shift_v8i32:
; AVX2: # BB#0:
; AVX2-NEXT: vpsravd %ymm1, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: var_shift_v8i32:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
; XOPAVX1-NEXT: vpsubd %xmm2, %xmm3, %xmm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; XOPAVX1-NEXT: vpshad %xmm2, %xmm4, %xmm2
; XOPAVX1-NEXT: vpsubd %xmm1, %xmm3, %xmm1
; XOPAVX1-NEXT: vpshad %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: var_shift_v8i32:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpsravd %ymm1, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: var_shift_v8i32:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: vpsravd %ymm1, %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: var_shift_v8i32:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsravd %ymm1, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: var_shift_v8i32:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; X32-AVX1-NEXT: vextractf128 $1, %ymm1, %xmm3
; X32-AVX1-NEXT: vpsrldq {{.*#+}} xmm4 = xmm3[12,13,14,15],zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero
; X32-AVX1-NEXT: vpsrad %xmm4, %xmm2, %xmm4
; X32-AVX1-NEXT: vpsrlq $32, %xmm3, %xmm5
; X32-AVX1-NEXT: vpsrad %xmm5, %xmm2, %xmm5
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm4 = xmm5[0,1,2,3],xmm4[4,5,6,7]
; X32-AVX1-NEXT: vpxor %xmm5, %xmm5, %xmm5
; X32-AVX1-NEXT: vpunpckhdq {{.*#+}} xmm6 = xmm3[2],xmm5[2],xmm3[3],xmm5[3]
; X32-AVX1-NEXT: vpsrad %xmm6, %xmm2, %xmm6
; X32-AVX1-NEXT: vpmovzxdq {{.*#+}} xmm3 = xmm3[0],zero,xmm3[1],zero
; X32-AVX1-NEXT: vpsrad %xmm3, %xmm2, %xmm2
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm2[0,1,2,3],xmm6[4,5,6,7]
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm2[0,1],xmm4[2,3],xmm2[4,5],xmm4[6,7]
; X32-AVX1-NEXT: vpsrldq {{.*#+}} xmm3 = xmm1[12,13,14,15],zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero,zero
; X32-AVX1-NEXT: vpsrad %xmm3, %xmm0, %xmm3
; X32-AVX1-NEXT: vpsrlq $32, %xmm1, %xmm4
; X32-AVX1-NEXT: vpsrad %xmm4, %xmm0, %xmm4
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm3 = xmm4[0,1,2,3],xmm3[4,5,6,7]
; X32-AVX1-NEXT: vpunpckhdq {{.*#+}} xmm4 = xmm1[2],xmm5[2],xmm1[3],xmm5[3]
; X32-AVX1-NEXT: vpsrad %xmm4, %xmm0, %xmm4
; X32-AVX1-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
; X32-AVX1-NEXT: vpsrad %xmm1, %xmm0, %xmm0
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm4[4,5,6,7]
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm3[2,3],xmm0[4,5],xmm3[6,7]
; X32-AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: var_shift_v8i32:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpsravd %ymm1, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <8 x i32> %a, %b
ret <8 x i32> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <16 x i16> @var_shift_v16i16(<16 x i16> %a, <16 x i16> %b) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: var_shift_v16i16:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; AVX1-NEXT: vpsllw $12, %xmm2, %xmm3
; AVX1-NEXT: vpsllw $4, %xmm2, %xmm2
; AVX1-NEXT: vpor %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; AVX1-NEXT: vpsraw $8, %xmm4, %xmm5
; AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm2
; AVX1-NEXT: vpsraw $4, %xmm2, %xmm4
; AVX1-NEXT: vpblendvb %xmm3, %xmm4, %xmm2, %xmm2
; AVX1-NEXT: vpsraw $2, %xmm2, %xmm4
; AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpblendvb %xmm3, %xmm4, %xmm2, %xmm2
; AVX1-NEXT: vpsraw $1, %xmm2, %xmm4
; AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpblendvb %xmm3, %xmm4, %xmm2, %xmm2
; AVX1-NEXT: vpsllw $12, %xmm1, %xmm3
; AVX1-NEXT: vpsllw $4, %xmm1, %xmm1
; AVX1-NEXT: vpor %xmm3, %xmm1, %xmm1
; AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm3
; AVX1-NEXT: vpsraw $8, %xmm0, %xmm4
; AVX1-NEXT: vpblendvb %xmm1, %xmm4, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $4, %xmm0, %xmm1
; AVX1-NEXT: vpblendvb %xmm3, %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $2, %xmm0, %xmm1
; AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpblendvb %xmm3, %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $1, %xmm0, %xmm1
; AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpblendvb %xmm3, %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: var_shift_v16i16:
; AVX2: # BB#0:
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vpunpckhwd {{.*#+}} ymm3 = ymm1[4],ymm2[4],ymm1[5],ymm2[5],ymm1[6],ymm2[6],ymm1[7],ymm2[7],ymm1[12],ymm2[12],ymm1[13],ymm2[13],ymm1[14],ymm2[14],ymm1[15],ymm2[15]
[X86][AVX2] Fix v16i16 SHL lowering (PR27730) The AVX2 v16i16 shift lowering works by unpacking to 2 x v8i32, performing the shift and then truncating the result. The unpacking is used to place the values in the upper 16-bits so that we can correctly sign-extend for SRA shifts. Unfortunately we weren't ensuring that the lower 16-bits were zero to ensure that SHL correctly shifts in zero bits. llvm-svn: 271796
2016-06-05 00:45:33 +08:00
; AVX2-NEXT: vpunpckhwd {{.*#+}} ymm4 = ymm2[4],ymm0[4],ymm2[5],ymm0[5],ymm2[6],ymm0[6],ymm2[7],ymm0[7],ymm2[12],ymm0[12],ymm2[13],ymm0[13],ymm2[14],ymm0[14],ymm2[15],ymm0[15]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vpsravd %ymm3, %ymm4, %ymm3
; AVX2-NEXT: vpsrld $16, %ymm3, %ymm3
; AVX2-NEXT: vpunpcklwd {{.*#+}} ymm1 = ymm1[0],ymm2[0],ymm1[1],ymm2[1],ymm1[2],ymm2[2],ymm1[3],ymm2[3],ymm1[8],ymm2[8],ymm1[9],ymm2[9],ymm1[10],ymm2[10],ymm1[11],ymm2[11]
[X86][AVX2] Fix v16i16 SHL lowering (PR27730) The AVX2 v16i16 shift lowering works by unpacking to 2 x v8i32, performing the shift and then truncating the result. The unpacking is used to place the values in the upper 16-bits so that we can correctly sign-extend for SRA shifts. Unfortunately we weren't ensuring that the lower 16-bits were zero to ensure that SHL correctly shifts in zero bits. llvm-svn: 271796
2016-06-05 00:45:33 +08:00
; AVX2-NEXT: vpunpcklwd {{.*#+}} ymm0 = ymm2[0],ymm0[0],ymm2[1],ymm0[1],ymm2[2],ymm0[2],ymm2[3],ymm0[3],ymm2[8],ymm0[8],ymm2[9],ymm0[9],ymm2[10],ymm0[10],ymm2[11],ymm0[11]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vpsravd %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpsrld $16, %ymm0, %ymm0
; AVX2-NEXT: vpackusdw %ymm3, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: var_shift_v16i16:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
; XOPAVX1-NEXT: vpsubw %xmm2, %xmm3, %xmm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; XOPAVX1-NEXT: vpshaw %xmm2, %xmm4, %xmm2
; XOPAVX1-NEXT: vpsubw %xmm1, %xmm3, %xmm1
; XOPAVX1-NEXT: vpshaw %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: var_shift_v16i16:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vextracti128 $1, %ymm1, %xmm2
; XOPAVX2-NEXT: vpxor %xmm3, %xmm3, %xmm3
; XOPAVX2-NEXT: vpsubw %xmm2, %xmm3, %xmm2
; XOPAVX2-NEXT: vextracti128 $1, %ymm0, %xmm4
; XOPAVX2-NEXT: vpshaw %xmm2, %xmm4, %xmm2
; XOPAVX2-NEXT: vpsubw %xmm1, %xmm3, %xmm1
; XOPAVX2-NEXT: vpshaw %xmm1, %xmm0, %xmm0
; XOPAVX2-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
[X86][AVX512DQ] Added AVX512DQ to 128/256 bit vector shift tests llvm-svn: 291444
2017-01-09 22:36:09 +08:00
; AVX512DQ-LABEL: var_shift_v16i16:
; AVX512DQ: # BB#0:
[X86][AVX512DQ] Enable v16i16 vector shifts to use an extend+shift+truncate pattern. Use the existing AVX2 v8i16 vector shift lowering for v16i16 on AVX512 targets (AVX512BW will have already have lowered with vpsravw). Cost model updates to follow. llvm-svn: 291445
2017-01-09 23:15:45 +08:00
; AVX512DQ-NEXT: vpmovzxwd {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero
; AVX512DQ-NEXT: vpmovsxwd %ymm0, %zmm0
; AVX512DQ-NEXT: vpsravd %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpmovdw %zmm0, %ymm0
[X86][AVX512DQ] Added AVX512DQ to 128/256 bit vector shift tests llvm-svn: 291444
2017-01-09 22:36:09 +08:00
; AVX512DQ-NEXT: retq
;
; AVX512BW-LABEL: var_shift_v16i16:
; AVX512BW: # BB#0:
; AVX512BW-NEXT: # kill: %YMM1<def> %YMM1<kill> %ZMM1<def>
; AVX512BW-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<def>
; AVX512BW-NEXT: vpsravw %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<kill>
; AVX512BW-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512DQVL-LABEL: var_shift_v16i16:
; AVX512DQVL: # BB#0:
; AVX512DQVL-NEXT: vpmovzxwd {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero
; AVX512DQVL-NEXT: vpmovsxwd %ymm0, %zmm0
; AVX512DQVL-NEXT: vpsravd %zmm1, %zmm0, %zmm0
; AVX512DQVL-NEXT: vpmovdw %zmm0, %ymm0
; AVX512DQVL-NEXT: retq
;
; AVX512BWVL-LABEL: var_shift_v16i16:
; AVX512BWVL: # BB#0:
; AVX512BWVL-NEXT: vpsravw %ymm1, %ymm0, %ymm0
; AVX512BWVL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: var_shift_v16i16:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; X32-AVX1-NEXT: vpsllw $12, %xmm2, %xmm3
; X32-AVX1-NEXT: vpsllw $4, %xmm2, %xmm2
; X32-AVX1-NEXT: vpor %xmm3, %xmm2, %xmm2
; X32-AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm3
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; X32-AVX1-NEXT: vpsraw $8, %xmm4, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm2
; X32-AVX1-NEXT: vpsraw $4, %xmm2, %xmm4
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm4, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsraw $2, %xmm2, %xmm4
; X32-AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm4, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsraw $1, %xmm2, %xmm4
; X32-AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm4, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsllw $12, %xmm1, %xmm3
; X32-AVX1-NEXT: vpsllw $4, %xmm1, %xmm1
; X32-AVX1-NEXT: vpor %xmm3, %xmm1, %xmm1
; X32-AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm3
; X32-AVX1-NEXT: vpsraw $8, %xmm0, %xmm4
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm4, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $4, %xmm0, %xmm1
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm1, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $2, %xmm0, %xmm1
; X32-AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm1, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $1, %xmm0, %xmm1
; X32-AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm1, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: var_shift_v16i16:
; X32-AVX2: # BB#0:
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; X32-AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
; X32-AVX2-NEXT: vpunpckhwd {{.*#+}} ymm3 = ymm1[4],ymm2[4],ymm1[5],ymm2[5],ymm1[6],ymm2[6],ymm1[7],ymm2[7],ymm1[12],ymm2[12],ymm1[13],ymm2[13],ymm1[14],ymm2[14],ymm1[15],ymm2[15]
; X32-AVX2-NEXT: vpunpckhwd {{.*#+}} ymm4 = ymm2[4],ymm0[4],ymm2[5],ymm0[5],ymm2[6],ymm0[6],ymm2[7],ymm0[7],ymm2[12],ymm0[12],ymm2[13],ymm0[13],ymm2[14],ymm0[14],ymm2[15],ymm0[15]
; X32-AVX2-NEXT: vpsravd %ymm3, %ymm4, %ymm3
; X32-AVX2-NEXT: vpsrld $16, %ymm3, %ymm3
; X32-AVX2-NEXT: vpunpcklwd {{.*#+}} ymm1 = ymm1[0],ymm2[0],ymm1[1],ymm2[1],ymm1[2],ymm2[2],ymm1[3],ymm2[3],ymm1[8],ymm2[8],ymm1[9],ymm2[9],ymm1[10],ymm2[10],ymm1[11],ymm2[11]
; X32-AVX2-NEXT: vpunpcklwd {{.*#+}} ymm0 = ymm2[0],ymm0[0],ymm2[1],ymm0[1],ymm2[2],ymm0[2],ymm2[3],ymm0[3],ymm2[8],ymm0[8],ymm2[9],ymm0[9],ymm2[10],ymm0[10],ymm2[11],ymm0[11]
; X32-AVX2-NEXT: vpsravd %ymm1, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsrld $16, %ymm0, %ymm0
; X32-AVX2-NEXT: vpackusdw %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <16 x i16> %a, %b
ret <16 x i16> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <32 x i8> @var_shift_v32i8(<32 x i8> %a, <32 x i8> %b) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: var_shift_v32i8:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; AVX1-NEXT: vpsllw $5, %xmm2, %xmm2
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm3 = xmm0[8],xmm2[8],xmm0[9],xmm2[9],xmm0[10],xmm2[10],xmm0[11],xmm2[11],xmm0[12],xmm2[12],xmm0[13],xmm2[13],xmm0[14],xmm2[14],xmm0[15],xmm2[15]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm5 = xmm0[8],xmm4[8],xmm0[9],xmm4[9],xmm0[10],xmm4[10],xmm0[11],xmm4[11],xmm0[12],xmm4[12],xmm0[13],xmm4[13],xmm0[14],xmm4[14],xmm0[15],xmm4[15]
; AVX1-NEXT: vpsraw $4, %xmm5, %xmm6
; AVX1-NEXT: vpblendvb %xmm3, %xmm6, %xmm5, %xmm5
; AVX1-NEXT: vpsraw $2, %xmm5, %xmm6
; AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpblendvb %xmm3, %xmm6, %xmm5, %xmm5
; AVX1-NEXT: vpsraw $1, %xmm5, %xmm6
; AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpblendvb %xmm3, %xmm6, %xmm5, %xmm3
; AVX1-NEXT: vpsrlw $8, %xmm3, %xmm3
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm2 = xmm0[0],xmm2[0],xmm0[1],xmm2[1],xmm0[2],xmm2[2],xmm0[3],xmm2[3],xmm0[4],xmm2[4],xmm0[5],xmm2[5],xmm0[6],xmm2[6],xmm0[7],xmm2[7]
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm4 = xmm0[0],xmm4[0],xmm0[1],xmm4[1],xmm0[2],xmm4[2],xmm0[3],xmm4[3],xmm0[4],xmm4[4],xmm0[5],xmm4[5],xmm0[6],xmm4[6],xmm0[7],xmm4[7]
; AVX1-NEXT: vpsraw $4, %xmm4, %xmm5
; AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsraw $2, %xmm4, %xmm5
; AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsraw $1, %xmm4, %xmm5
; AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm2
; AVX1-NEXT: vpsrlw $8, %xmm2, %xmm2
; AVX1-NEXT: vpackuswb %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpsllw $5, %xmm1, %xmm1
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm3 = xmm0[8],xmm1[8],xmm0[9],xmm1[9],xmm0[10],xmm1[10],xmm0[11],xmm1[11],xmm0[12],xmm1[12],xmm0[13],xmm1[13],xmm0[14],xmm1[14],xmm0[15],xmm1[15]
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm4 = xmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
; AVX1-NEXT: vpsraw $4, %xmm4, %xmm5
; AVX1-NEXT: vpblendvb %xmm3, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsraw $2, %xmm4, %xmm5
; AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpblendvb %xmm3, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsraw $1, %xmm4, %xmm5
; AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpblendvb %xmm3, %xmm5, %xmm4, %xmm3
; AVX1-NEXT: vpsrlw $8, %xmm3, %xmm3
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm1 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
; AVX1-NEXT: vpsraw $4, %xmm0, %xmm4
; AVX1-NEXT: vpblendvb %xmm1, %xmm4, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $2, %xmm0, %xmm4
; AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm1
; AVX1-NEXT: vpblendvb %xmm1, %xmm4, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $1, %xmm0, %xmm4
; AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm1
; AVX1-NEXT: vpblendvb %xmm1, %xmm4, %xmm0, %xmm0
; AVX1-NEXT: vpsrlw $8, %xmm0, %xmm0
; AVX1-NEXT: vpackuswb %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: var_shift_v32i8:
; AVX2: # BB#0:
; AVX2-NEXT: vpsllw $5, %ymm1, %ymm1
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX2-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX2-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX2-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX2-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: var_shift_v32i8:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; XOPAVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
; XOPAVX1-NEXT: vpsubb %xmm2, %xmm3, %xmm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; XOPAVX1-NEXT: vpshab %xmm2, %xmm4, %xmm2
; XOPAVX1-NEXT: vpsubb %xmm1, %xmm3, %xmm1
; XOPAVX1-NEXT: vpshab %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: var_shift_v32i8:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vextracti128 $1, %ymm1, %xmm2
; XOPAVX2-NEXT: vpxor %xmm3, %xmm3, %xmm3
; XOPAVX2-NEXT: vpsubb %xmm2, %xmm3, %xmm2
; XOPAVX2-NEXT: vextracti128 $1, %ymm0, %xmm4
; XOPAVX2-NEXT: vpshab %xmm2, %xmm4, %xmm2
; XOPAVX2-NEXT: vpsubb %xmm1, %xmm3, %xmm1
; XOPAVX2-NEXT: vpshab %xmm1, %xmm0, %xmm0
; XOPAVX2-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
[X86][AVX512] Enable v16i8/v32i8 vector shifts to use an extend+shift+truncate pattern. Use the existing AVX2 v8i16 vector shift lowering for v16i8 (extending to v16i32) on AVX512 targets and v32i8 (extending to v32i16) on AVX512BW targets. Cost model updates to follow. llvm-svn: 291451
2017-01-10 01:20:03 +08:00
; AVX512DQ-LABEL: var_shift_v32i8:
; AVX512DQ: # BB#0:
; AVX512DQ-NEXT: vpsllw $5, %ymm1, %ymm1
; AVX512DQ-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX512DQ-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512DQ-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQ-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX512DQ-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQ-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX512DQ-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX512DQ-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512DQ-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX512DQ-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512DQ-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX512DQ-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX512DQ-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512DQ-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: retq
;
; AVX512BW-LABEL: var_shift_v32i8:
; AVX512BW: # BB#0:
; AVX512BW-NEXT: vpmovzxbw {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero,ymm1[16],zero,ymm1[17],zero,ymm1[18],zero,ymm1[19],zero,ymm1[20],zero,ymm1[21],zero,ymm1[22],zero,ymm1[23],zero,ymm1[24],zero,ymm1[25],zero,ymm1[26],zero,ymm1[27],zero,ymm1[28],zero,ymm1[29],zero,ymm1[30],zero,ymm1[31],zero
; AVX512BW-NEXT: vpmovsxbw %ymm0, %zmm0
; AVX512BW-NEXT: vpsravw %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpmovwb %zmm0, %ymm0
; AVX512BW-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512DQVL-LABEL: var_shift_v32i8:
; AVX512DQVL: # BB#0:
; AVX512DQVL-NEXT: vpsllw $5, %ymm1, %ymm1
; AVX512DQVL-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX512DQVL-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512DQVL-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQVL-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQVL-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX512DQVL-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX512DQVL-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512DQVL-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX512DQVL-NEXT: retq
;
; AVX512BWVL-LABEL: var_shift_v32i8:
; AVX512BWVL: # BB#0:
; AVX512BWVL-NEXT: vpmovzxbw {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero,ymm1[16],zero,ymm1[17],zero,ymm1[18],zero,ymm1[19],zero,ymm1[20],zero,ymm1[21],zero,ymm1[22],zero,ymm1[23],zero,ymm1[24],zero,ymm1[25],zero,ymm1[26],zero,ymm1[27],zero,ymm1[28],zero,ymm1[29],zero,ymm1[30],zero,ymm1[31],zero
; AVX512BWVL-NEXT: vpmovsxbw %ymm0, %zmm0
; AVX512BWVL-NEXT: vpsravw %zmm1, %zmm0, %zmm0
; AVX512BWVL-NEXT: vpmovwb %zmm0, %ymm0
; AVX512BWVL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: var_shift_v32i8:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; X32-AVX1-NEXT: vpsllw $5, %xmm2, %xmm2
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm3 = xmm0[8],xmm2[8],xmm0[9],xmm2[9],xmm0[10],xmm2[10],xmm0[11],xmm2[11],xmm0[12],xmm2[12],xmm0[13],xmm2[13],xmm0[14],xmm2[14],xmm0[15],xmm2[15]
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm4
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm5 = xmm0[8],xmm4[8],xmm0[9],xmm4[9],xmm0[10],xmm4[10],xmm0[11],xmm4[11],xmm0[12],xmm4[12],xmm0[13],xmm4[13],xmm0[14],xmm4[14],xmm0[15],xmm4[15]
; X32-AVX1-NEXT: vpsraw $4, %xmm5, %xmm6
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm6, %xmm5, %xmm5
; X32-AVX1-NEXT: vpsraw $2, %xmm5, %xmm6
; X32-AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm6, %xmm5, %xmm5
; X32-AVX1-NEXT: vpsraw $1, %xmm5, %xmm6
; X32-AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm6, %xmm5, %xmm3
; X32-AVX1-NEXT: vpsrlw $8, %xmm3, %xmm3
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm2 = xmm0[0],xmm2[0],xmm0[1],xmm2[1],xmm0[2],xmm2[2],xmm0[3],xmm2[3],xmm0[4],xmm2[4],xmm0[5],xmm2[5],xmm0[6],xmm2[6],xmm0[7],xmm2[7]
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm4 = xmm0[0],xmm4[0],xmm0[1],xmm4[1],xmm0[2],xmm4[2],xmm0[3],xmm4[3],xmm0[4],xmm4[4],xmm0[5],xmm4[5],xmm0[6],xmm4[6],xmm0[7],xmm4[7]
; X32-AVX1-NEXT: vpsraw $4, %xmm4, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm4
; X32-AVX1-NEXT: vpsraw $2, %xmm4, %xmm5
; X32-AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm2
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm4
; X32-AVX1-NEXT: vpsraw $1, %xmm4, %xmm5
; X32-AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm2
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm2
; X32-AVX1-NEXT: vpsrlw $8, %xmm2, %xmm2
; X32-AVX1-NEXT: vpackuswb %xmm3, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsllw $5, %xmm1, %xmm1
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm3 = xmm0[8],xmm1[8],xmm0[9],xmm1[9],xmm0[10],xmm1[10],xmm0[11],xmm1[11],xmm0[12],xmm1[12],xmm0[13],xmm1[13],xmm0[14],xmm1[14],xmm0[15],xmm1[15]
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm4 = xmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
; X32-AVX1-NEXT: vpsraw $4, %xmm4, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm5, %xmm4, %xmm4
; X32-AVX1-NEXT: vpsraw $2, %xmm4, %xmm5
; X32-AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm5, %xmm4, %xmm4
; X32-AVX1-NEXT: vpsraw $1, %xmm4, %xmm5
; X32-AVX1-NEXT: vpaddw %xmm3, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendvb %xmm3, %xmm5, %xmm4, %xmm3
; X32-AVX1-NEXT: vpsrlw $8, %xmm3, %xmm3
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm1 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
; X32-AVX1-NEXT: vpsraw $4, %xmm0, %xmm4
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm4, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $2, %xmm0, %xmm4
; X32-AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm1
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm4, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $1, %xmm0, %xmm4
; X32-AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm1
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm4, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsrlw $8, %xmm0, %xmm0
; X32-AVX1-NEXT: vpackuswb %xmm3, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: var_shift_v32i8:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpsllw $5, %ymm1, %ymm1
; X32-AVX2-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; X32-AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; X32-AVX2-NEXT: vpsraw $4, %ymm3, %ymm4
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; X32-AVX2-NEXT: vpsraw $2, %ymm3, %ymm4
; X32-AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; X32-AVX2-NEXT: vpsraw $1, %ymm3, %ymm4
; X32-AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; X32-AVX2-NEXT: vpsrlw $8, %ymm2, %ymm2
; X32-AVX2-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; X32-AVX2-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; X32-AVX2-NEXT: vpsraw $4, %ymm0, %ymm3
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsraw $2, %ymm0, %ymm3
; X32-AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsraw $1, %ymm0, %ymm3
; X32-AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0
; X32-AVX2-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <32 x i8> %a, %b
ret <32 x i8> %shift
}
;
; Uniform Variable Shifts
;
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <4 x i64> @splatvar_shift_v4i64(<4 x i64> %a, <4 x i64> %b) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: splatvar_shift_v4i64:
; AVX1: # BB#0:
[X86][SSE] Vectorize i64 ASHR operations This patch vectorizes the v2i64/v4i64 ASHR shift operations - the last remaining integer vector shifts that are still being transferred to/from the scalar unit to be completed. Differential Revision: http://reviews.llvm.org/D11439 llvm-svn: 243569
2015-07-30 04:31:45 +08:00
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808]
; AVX1-NEXT: vpsrlq %xmm1, %xmm2, %xmm2
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vpsrlq %xmm1, %xmm3, %xmm3
; AVX1-NEXT: vpxor %xmm2, %xmm3, %xmm3
; AVX1-NEXT: vpsubq %xmm2, %xmm3, %xmm3
; AVX1-NEXT: vpsrlq %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpxor %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpsubq %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm0, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: splatvar_shift_v4i64:
; AVX2: # BB#0:
[X86][AVX] Regenerate shift test to show constant broadcast comment llvm-svn: 308323
2017-07-19 00:07:12 +08:00
; AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808]
[X86][SSE] Vectorize i64 ASHR operations This patch vectorizes the v2i64/v4i64 ASHR shift operations - the last remaining integer vector shifts that are still being transferred to/from the scalar unit to be completed. Differential Revision: http://reviews.llvm.org/D11439 llvm-svn: 243569
2015-07-30 04:31:45 +08:00
; AVX2-NEXT: vpsrlq %xmm1, %ymm2, %ymm2
; AVX2-NEXT: vpsrlq %xmm1, %ymm0, %ymm0
; AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; AVX2-NEXT: vpsubq %ymm2, %ymm0, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: splatvar_shift_v4i64:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vmovddup {{.*#+}} xmm1 = xmm1[0,0]
; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; XOPAVX1-NEXT: vpsubq %xmm1, %xmm2, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: splatvar_shift_v4i64:
; XOPAVX2: # BB#0:
[X86][AVX] Regenerate shift test to show constant broadcast comment llvm-svn: 308323
2017-07-19 00:07:12 +08:00
; XOPAVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808]
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
; XOPAVX2-NEXT: vpsrlq %xmm1, %ymm2, %ymm2
; XOPAVX2-NEXT: vpsrlq %xmm1, %ymm0, %ymm0
; XOPAVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; XOPAVX2-NEXT: vpsubq %ymm2, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: splatvar_shift_v4i64:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
[X86][AVX512] Add support for variable ASHR v2i64/v4i64 support without VLX Use v8i64 variable ASHR instructions if we don't have VLX. This is a reduced version of D28537 that just adds support for variable shifts - I'll continue with that patch (for just constant/uniform shifts) once I've fixed the type legalization issue in avx512-cvt.ll. Differential Revision: https://reviews.llvm.org/D28604 llvm-svn: 291901
2017-01-13 21:16:19 +08:00
; AVX512-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<def>
[X86][AVX512] Add support for ASHR v2i64/v4i64 support without VLX Use v8i64 ASHR instructions if we don't have VLX. Differential Revision: https://reviews.llvm.org/D28537 llvm-svn: 295656
2017-02-20 20:16:38 +08:00
; AVX512-NEXT: vpsraq %xmm1, %zmm0, %zmm0
[X86][AVX512] Add support for variable ASHR v2i64/v4i64 support without VLX Use v8i64 variable ASHR instructions if we don't have VLX. This is a reduced version of D28537 that just adds support for variable shifts - I'll continue with that patch (for just constant/uniform shifts) once I've fixed the type legalization issue in avx512-cvt.ll. Differential Revision: https://reviews.llvm.org/D28604 llvm-svn: 291901
2017-01-13 21:16:19 +08:00
; AVX512-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<kill>
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: splatvar_shift_v4i64:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsraq %xmm1, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: splatvar_shift_v4i64:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [0,2147483648,0,2147483648]
; X32-AVX1-NEXT: vpsrlq %xmm1, %xmm2, %xmm2
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; X32-AVX1-NEXT: vpsrlq %xmm1, %xmm3, %xmm3
; X32-AVX1-NEXT: vpxor %xmm2, %xmm3, %xmm3
; X32-AVX1-NEXT: vpsubq %xmm2, %xmm3, %xmm3
; X32-AVX1-NEXT: vpsrlq %xmm1, %xmm0, %xmm0
; X32-AVX1-NEXT: vpxor %xmm2, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsubq %xmm2, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: splatvar_shift_v4i64:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vmovdqa {{.*#+}} ymm2 = [0,2147483648,0,2147483648,0,2147483648,0,2147483648]
; X32-AVX2-NEXT: vpsrlq %xmm1, %ymm2, %ymm2
; X32-AVX2-NEXT: vpsrlq %xmm1, %ymm0, %ymm0
; X32-AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsubq %ymm2, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%splat = shufflevector <4 x i64> %b, <4 x i64> undef, <4 x i32> zeroinitializer
%shift = ashr <4 x i64> %a, %splat
ret <4 x i64> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <8 x i32> @splatvar_shift_v8i32(<8 x i32> %a, <8 x i32> %b) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: splatvar_shift_v8i32:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; AVX1-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpsrad %xmm1, %xmm2, %xmm2
; AVX1-NEXT: vpsrad %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: splatvar_shift_v8i32:
; AVX2: # BB#0:
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; AVX2-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vpsrad %xmm1, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: splatvar_shift_v8i32:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; XOPAVX1-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
; XOPAVX1-NEXT: vpsrad %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpsrad %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: splatvar_shift_v8i32:
; XOPAVX2: # BB#0:
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; XOPAVX2-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
; XOPAVX2-NEXT: vpsrad %xmm1, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: splatvar_shift_v8i32:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; AVX512-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: vpsrad %xmm1, %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: splatvar_shift_v8i32:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
; AVX512VL-NEXT: vpsrad %xmm1, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: splatvar_shift_v8i32:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; X32-AVX1-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
; X32-AVX1-NEXT: vpsrad %xmm1, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsrad %xmm1, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: splatvar_shift_v8i32:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpmovzxdq {{.*#+}} xmm1 = xmm1[0],zero,xmm1[1],zero
; X32-AVX2-NEXT: vpsrad %xmm1, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%splat = shufflevector <8 x i32> %b, <8 x i32> undef, <8 x i32> zeroinitializer
%shift = ashr <8 x i32> %a, %splat
ret <8 x i32> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <16 x i16> @splatvar_shift_v16i16(<16 x i16> %a, <16 x i16> %b) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: splatvar_shift_v16i16:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; AVX1-NEXT: vpmovzxwq {{.*#+}} xmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpsraw %xmm1, %xmm2, %xmm2
; AVX1-NEXT: vpsraw %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: splatvar_shift_v16i16:
; AVX2: # BB#0:
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; AVX2-NEXT: vpmovzxwq {{.*#+}} xmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vpsraw %xmm1, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: splatvar_shift_v16i16:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; XOPAVX1-NEXT: vpmovzxwq {{.*#+}} xmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
; XOPAVX1-NEXT: vpsraw %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpsraw %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: splatvar_shift_v16i16:
; XOPAVX2: # BB#0:
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; XOPAVX2-NEXT: vpmovzxwq {{.*#+}} xmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
; XOPAVX2-NEXT: vpsraw %xmm1, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: splatvar_shift_v16i16:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
[X86] Optimize vector shifts with variable but uniform shift amounts Summary: For instructions such as PSLLW/PSLLD/PSLLQ a variable shift amount may be passed in an XMM register. The lower 64-bits of the register are evaluated to determine the shift amount. This patch improves the construction of the vector containing the shift amount. Reviewers: craig.topper, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D28353 llvm-svn: 291120
2017-01-05 23:11:43 +08:00
; AVX512-NEXT: vpmovzxwq {{.*#+}} xmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: vpsraw %xmm1, %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: splatvar_shift_v16i16:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpmovzxwq {{.*#+}} xmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero
; AVX512VL-NEXT: vpsraw %xmm1, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: splatvar_shift_v16i16:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; X32-AVX1-NEXT: vpmovzxwq {{.*#+}} xmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero
; X32-AVX1-NEXT: vpsraw %xmm1, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsraw %xmm1, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: splatvar_shift_v16i16:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpmovzxwq {{.*#+}} xmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero
; X32-AVX2-NEXT: vpsraw %xmm1, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%splat = shufflevector <16 x i16> %b, <16 x i16> undef, <16 x i32> zeroinitializer
%shift = ashr <16 x i16> %a, %splat
ret <16 x i16> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <32 x i8> @splatvar_shift_v32i8(<32 x i8> %a, <32 x i8> %b) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: splatvar_shift_v32i8:
; AVX1: # BB#0:
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpshufb %xmm2, %xmm1, %xmm1
; AVX1-NEXT: vpsllw $5, %xmm1, %xmm1
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm2 = xmm0[8],xmm1[8],xmm0[9],xmm1[9],xmm0[10],xmm1[10],xmm0[11],xmm1[11],xmm0[12],xmm1[12],xmm0[13],xmm1[13],xmm0[14],xmm1[14],xmm0[15],xmm1[15]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm4 = xmm0[8],xmm3[8],xmm0[9],xmm3[9],xmm0[10],xmm3[10],xmm0[11],xmm3[11],xmm0[12],xmm3[12],xmm0[13],xmm3[13],xmm0[14],xmm3[14],xmm0[15],xmm3[15]
; AVX1-NEXT: vpsraw $4, %xmm4, %xmm5
; AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsraw $2, %xmm4, %xmm5
; AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm6
; AVX1-NEXT: vpblendvb %xmm6, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsraw $1, %xmm4, %xmm5
; AVX1-NEXT: vpaddw %xmm6, %xmm6, %xmm9
; AVX1-NEXT: vpblendvb %xmm9, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsrlw $8, %xmm4, %xmm8
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm1 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm3 = xmm0[0],xmm3[0],xmm0[1],xmm3[1],xmm0[2],xmm3[2],xmm0[3],xmm3[3],xmm0[4],xmm3[4],xmm0[5],xmm3[5],xmm0[6],xmm3[6],xmm0[7],xmm3[7]
; AVX1-NEXT: vpsraw $4, %xmm3, %xmm5
; AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm3, %xmm3
; AVX1-NEXT: vpsraw $2, %xmm3, %xmm5
; AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm4
; AVX1-NEXT: vpblendvb %xmm4, %xmm5, %xmm3, %xmm3
; AVX1-NEXT: vpsraw $1, %xmm3, %xmm5
; AVX1-NEXT: vpaddw %xmm4, %xmm4, %xmm7
; AVX1-NEXT: vpblendvb %xmm7, %xmm5, %xmm3, %xmm3
; AVX1-NEXT: vpsrlw $8, %xmm3, %xmm3
; AVX1-NEXT: vpackuswb %xmm8, %xmm3, %xmm8
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm5 = xmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
; AVX1-NEXT: vpsraw $4, %xmm5, %xmm3
; AVX1-NEXT: vpblendvb %xmm2, %xmm3, %xmm5, %xmm2
; AVX1-NEXT: vpsraw $2, %xmm2, %xmm3
; AVX1-NEXT: vpblendvb %xmm6, %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpsraw $1, %xmm2, %xmm3
; AVX1-NEXT: vpblendvb %xmm9, %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpsrlw $8, %xmm2, %xmm2
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
; AVX1-NEXT: vpsraw $4, %xmm0, %xmm3
; AVX1-NEXT: vpblendvb %xmm1, %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $2, %xmm0, %xmm1
; AVX1-NEXT: vpblendvb %xmm4, %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $1, %xmm0, %xmm1
; AVX1-NEXT: vpblendvb %xmm7, %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpsrlw $8, %xmm0, %xmm0
; AVX1-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm8, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: splatvar_shift_v32i8:
; AVX2: # BB#0:
; AVX2-NEXT: vpbroadcastb %xmm1, %ymm1
; AVX2-NEXT: vpsllw $5, %ymm1, %ymm1
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX2-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX2-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX2-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX2-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: splatvar_shift_v32i8:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; XOPAVX1-NEXT: vpshufb %xmm2, %xmm1, %xmm1
; XOPAVX1-NEXT: vpsubb %xmm1, %xmm2, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vpshab %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpshab %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: splatvar_shift_v32i8:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpbroadcastb %xmm1, %ymm1
; XOPAVX2-NEXT: vextracti128 $1, %ymm1, %xmm2
; XOPAVX2-NEXT: vpxor %xmm3, %xmm3, %xmm3
; XOPAVX2-NEXT: vpsubb %xmm2, %xmm3, %xmm2
; XOPAVX2-NEXT: vextracti128 $1, %ymm0, %xmm4
; XOPAVX2-NEXT: vpshab %xmm2, %xmm4, %xmm2
; XOPAVX2-NEXT: vpsubb %xmm1, %xmm3, %xmm1
; XOPAVX2-NEXT: vpshab %xmm1, %xmm0, %xmm0
; XOPAVX2-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
[X86][AVX512] Enable v16i8/v32i8 vector shifts to use an extend+shift+truncate pattern. Use the existing AVX2 v8i16 vector shift lowering for v16i8 (extending to v16i32) on AVX512 targets and v32i8 (extending to v32i16) on AVX512BW targets. Cost model updates to follow. llvm-svn: 291451
2017-01-10 01:20:03 +08:00
; AVX512DQ-LABEL: splatvar_shift_v32i8:
; AVX512DQ: # BB#0:
; AVX512DQ-NEXT: vpbroadcastb %xmm1, %ymm1
; AVX512DQ-NEXT: vpsllw $5, %ymm1, %ymm1
; AVX512DQ-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX512DQ-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512DQ-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQ-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX512DQ-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQ-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX512DQ-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX512DQ-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512DQ-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX512DQ-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512DQ-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX512DQ-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX512DQ-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512DQ-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: retq
;
; AVX512BW-LABEL: splatvar_shift_v32i8:
; AVX512BW: # BB#0:
; AVX512BW-NEXT: vpbroadcastb %xmm1, %ymm1
; AVX512BW-NEXT: vpmovsxbw %ymm0, %zmm0
; AVX512BW-NEXT: vpmovzxbw {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero,ymm1[16],zero,ymm1[17],zero,ymm1[18],zero,ymm1[19],zero,ymm1[20],zero,ymm1[21],zero,ymm1[22],zero,ymm1[23],zero,ymm1[24],zero,ymm1[25],zero,ymm1[26],zero,ymm1[27],zero,ymm1[28],zero,ymm1[29],zero,ymm1[30],zero,ymm1[31],zero
; AVX512BW-NEXT: vpsravw %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpmovwb %zmm0, %ymm0
; AVX512BW-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512DQVL-LABEL: splatvar_shift_v32i8:
; AVX512DQVL: # BB#0:
; AVX512DQVL-NEXT: vpbroadcastb %xmm1, %ymm1
; AVX512DQVL-NEXT: vpsllw $5, %ymm1, %ymm1
; AVX512DQVL-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX512DQVL-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512DQVL-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQVL-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQVL-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX512DQVL-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX512DQVL-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512DQVL-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX512DQVL-NEXT: retq
;
; AVX512BWVL-LABEL: splatvar_shift_v32i8:
; AVX512BWVL: # BB#0:
; AVX512BWVL-NEXT: vpbroadcastb %xmm1, %ymm1
; AVX512BWVL-NEXT: vpmovsxbw %ymm0, %zmm0
; AVX512BWVL-NEXT: vpmovzxbw {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero,ymm1[16],zero,ymm1[17],zero,ymm1[18],zero,ymm1[19],zero,ymm1[20],zero,ymm1[21],zero,ymm1[22],zero,ymm1[23],zero,ymm1[24],zero,ymm1[25],zero,ymm1[26],zero,ymm1[27],zero,ymm1[28],zero,ymm1[29],zero,ymm1[30],zero,ymm1[31],zero
; AVX512BWVL-NEXT: vpsravw %zmm1, %zmm0, %zmm0
; AVX512BWVL-NEXT: vpmovwb %zmm0, %ymm0
; AVX512BWVL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: splatvar_shift_v32i8:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; X32-AVX1-NEXT: vpshufb %xmm2, %xmm1, %xmm1
; X32-AVX1-NEXT: vpsllw $5, %xmm1, %xmm1
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm2 = xmm0[8],xmm1[8],xmm0[9],xmm1[9],xmm0[10],xmm1[10],xmm0[11],xmm1[11],xmm0[12],xmm1[12],xmm0[13],xmm1[13],xmm0[14],xmm1[14],xmm0[15],xmm1[15]
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm4 = xmm0[8],xmm3[8],xmm0[9],xmm3[9],xmm0[10],xmm3[10],xmm0[11],xmm3[11],xmm0[12],xmm3[12],xmm0[13],xmm3[13],xmm0[14],xmm3[14],xmm0[15],xmm3[15]
; X32-AVX1-NEXT: vpsraw $4, %xmm4, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm4
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm5 = xmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
; X32-AVX1-NEXT: vpsraw $4, %xmm5, %xmm6
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm5, %xmm5
; X32-AVX1-NEXT: vpsraw $2, %xmm4, %xmm6
; X32-AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm2
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm4, %xmm4
; X32-AVX1-NEXT: vpsraw $2, %xmm5, %xmm6
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm5, %xmm5
; X32-AVX1-NEXT: vpsraw $1, %xmm4, %xmm6
; X32-AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm2
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm4, %xmm4
; X32-AVX1-NEXT: vpsraw $1, %xmm5, %xmm6
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm5, %xmm2
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm1 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm3 = xmm0[0],xmm3[0],xmm0[1],xmm3[1],xmm0[2],xmm3[2],xmm0[3],xmm3[3],xmm0[4],xmm3[4],xmm0[5],xmm3[5],xmm0[6],xmm3[6],xmm0[7],xmm3[7]
; X32-AVX1-NEXT: vpsraw $4, %xmm3, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm3, %xmm3
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
; X32-AVX1-NEXT: vpsraw $4, %xmm0, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $2, %xmm3, %xmm5
; X32-AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm1
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm3, %xmm3
; X32-AVX1-NEXT: vpsraw $2, %xmm0, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $1, %xmm3, %xmm5
; X32-AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm1
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm3, %xmm3
; X32-AVX1-NEXT: vpsraw $1, %xmm0, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsrlw $8, %xmm4, %xmm1
; X32-AVX1-NEXT: vpsrlw $8, %xmm3, %xmm3
; X32-AVX1-NEXT: vpackuswb %xmm1, %xmm3, %xmm1
; X32-AVX1-NEXT: vpsrlw $8, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsrlw $8, %xmm0, %xmm0
; X32-AVX1-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: splatvar_shift_v32i8:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpbroadcastb %xmm1, %ymm1
; X32-AVX2-NEXT: vpsllw $5, %ymm1, %ymm1
; X32-AVX2-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; X32-AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; X32-AVX2-NEXT: vpsraw $4, %ymm3, %ymm4
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; X32-AVX2-NEXT: vpsraw $2, %ymm3, %ymm4
; X32-AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; X32-AVX2-NEXT: vpsraw $1, %ymm3, %ymm4
; X32-AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; X32-AVX2-NEXT: vpsrlw $8, %ymm2, %ymm2
; X32-AVX2-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; X32-AVX2-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; X32-AVX2-NEXT: vpsraw $4, %ymm0, %ymm3
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsraw $2, %ymm0, %ymm3
; X32-AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsraw $1, %ymm0, %ymm3
; X32-AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0
; X32-AVX2-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%splat = shufflevector <32 x i8> %b, <32 x i8> undef, <32 x i32> zeroinitializer
%shift = ashr <32 x i8> %a, %splat
ret <32 x i8> %shift
}
;
; Constant Shifts
;
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <4 x i64> @constant_shift_v4i64(<4 x i64> %a) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: constant_shift_v4i64:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
[X86][SSE] Vectorize i64 ASHR operations This patch vectorizes the v2i64/v4i64 ASHR shift operations - the last remaining integer vector shifts that are still being transferred to/from the scalar unit to be completed. Differential Revision: http://reviews.llvm.org/D11439 llvm-svn: 243569
2015-07-30 04:31:45 +08:00
; AVX1-NEXT: vpsrlq $62, %xmm1, %xmm2
; AVX1-NEXT: vpsrlq $31, %xmm1, %xmm1
; AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0,1,2,3],xmm2[4,5,6,7]
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [4294967296,2]
; AVX1-NEXT: vpxor %xmm2, %xmm1, %xmm1
; AVX1-NEXT: vpsubq %xmm2, %xmm1, %xmm1
; AVX1-NEXT: vpsrlq $7, %xmm0, %xmm2
; AVX1-NEXT: vpsrlq $1, %xmm0, %xmm0
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm2[4,5,6,7]
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [4611686018427387904,72057594037927936]
; AVX1-NEXT: vpxor %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpsubq %xmm2, %xmm0, %xmm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: constant_shift_v4i64:
; AVX2: # BB#0:
[X86][SSE] Vectorize i64 ASHR operations This patch vectorizes the v2i64/v4i64 ASHR shift operations - the last remaining integer vector shifts that are still being transferred to/from the scalar unit to be completed. Differential Revision: http://reviews.llvm.org/D11439 llvm-svn: 243569
2015-07-30 04:31:45 +08:00
; AVX2-NEXT: vpsrlvq {{.*}}(%rip), %ymm0, %ymm0
; AVX2-NEXT: vmovdqa {{.*#+}} ymm1 = [4611686018427387904,72057594037927936,4294967296,2]
; AVX2-NEXT: vpxor %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpsubq %ymm1, %ymm0, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: constant_shift_v4i64:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; XOPAVX1-NEXT: vpsubq {{.*}}(%rip), %xmm1, %xmm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; XOPAVX1-NEXT: vpshaq %xmm2, %xmm3, %xmm2
; XOPAVX1-NEXT: vpsubq {{.*}}(%rip), %xmm1, %xmm1
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: constant_shift_v4i64:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpsrlvq {{.*}}(%rip), %ymm0, %ymm0
; XOPAVX2-NEXT: vmovdqa {{.*#+}} ymm1 = [4611686018427387904,72057594037927936,4294967296,2]
; XOPAVX2-NEXT: vpxor %ymm1, %ymm0, %ymm0
; XOPAVX2-NEXT: vpsubq %ymm1, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: constant_shift_v4i64:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
[X86][AVX512] Add support for variable ASHR v2i64/v4i64 support without VLX Use v8i64 variable ASHR instructions if we don't have VLX. This is a reduced version of D28537 that just adds support for variable shifts - I'll continue with that patch (for just constant/uniform shifts) once I've fixed the type legalization issue in avx512-cvt.ll. Differential Revision: https://reviews.llvm.org/D28604 llvm-svn: 291901
2017-01-13 21:16:19 +08:00
; AVX512-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<def>
; AVX512-NEXT: vmovdqa {{.*#+}} ymm1 = [1,7,31,62]
; AVX512-NEXT: vpsravq %zmm1, %zmm0, %zmm0
; AVX512-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<kill>
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: constant_shift_v4i64:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsravq {{.*}}(%rip), %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: constant_shift_v4i64:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vmovdqa {{.*#+}} ymm1 = [1,0,7,0,31,0,62,0]
; X32-AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
; X32-AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [0,2147483648,0,2147483648]
; X32-AVX1-NEXT: vpsrlq %xmm2, %xmm3, %xmm4
; X32-AVX1-NEXT: vpshufd {{.*#+}} xmm5 = xmm2[2,3,0,1]
; X32-AVX1-NEXT: vpsrlq %xmm5, %xmm3, %xmm6
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm4 = xmm4[0,1,2,3],xmm6[4,5,6,7]
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm6
; X32-AVX1-NEXT: vpsrlq %xmm2, %xmm6, %xmm2
; X32-AVX1-NEXT: vpsrlq %xmm5, %xmm6, %xmm5
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm2[0,1,2,3],xmm5[4,5,6,7]
; X32-AVX1-NEXT: vpxor %xmm4, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsubq %xmm4, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsrlq %xmm1, %xmm3, %xmm4
; X32-AVX1-NEXT: vpshufd {{.*#+}} xmm5 = xmm1[2,3,0,1]
; X32-AVX1-NEXT: vpsrlq %xmm5, %xmm3, %xmm3
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm3 = xmm4[0,1,2,3],xmm3[4,5,6,7]
; X32-AVX1-NEXT: vpsrlq %xmm1, %xmm0, %xmm1
; X32-AVX1-NEXT: vpsrlq %xmm5, %xmm0, %xmm0
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm1[0,1,2,3],xmm0[4,5,6,7]
; X32-AVX1-NEXT: vpxor %xmm3, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsubq %xmm3, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: constant_shift_v4i64:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vmovdqa {{.*#+}} ymm1 = [1,0,7,0,31,0,62,0]
; X32-AVX2-NEXT: vmovdqa {{.*#+}} ymm2 = [0,2147483648,0,2147483648,0,2147483648,0,2147483648]
; X32-AVX2-NEXT: vpsrlvq %ymm1, %ymm2, %ymm3
; X32-AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsrlvq %ymm1, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsubq %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <4 x i64> %a, <i64 1, i64 7, i64 31, i64 62>
ret <4 x i64> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <8 x i32> @constant_shift_v8i32(<8 x i32> %a) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: constant_shift_v8i32:
; AVX1: # BB#0:
[X86][SSE] Vectorized v4i32 non-uniform shifts. While the v4i32 shl operation is already vectorized using a cvttps2dq/pmulld pattern, the lshr/ashr opeations are still scalarized. This patch adds vectorization support for non-uniform v4i32 shift operations - it splats constant shift amounts to allow them to use the immediate sse shift instructions, or extracts/zero-extends non-constant shift amounts. The individual results are then blended together. Differential Revision: http://reviews.llvm.org/D11063 llvm-svn: 241989
2015-07-12 19:15:19 +08:00
; AVX1-NEXT: vpsrad $7, %xmm0, %xmm1
; AVX1-NEXT: vpsrad $5, %xmm0, %xmm2
; AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm2[0,1,2,3],xmm1[4,5,6,7]
; AVX1-NEXT: vpsrad $6, %xmm0, %xmm2
; AVX1-NEXT: vpsrad $4, %xmm0, %xmm3
; AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm3[0,1,2,3],xmm2[4,5,6,7]
; AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm2[0,1],xmm1[2,3],xmm2[4,5],xmm1[6,7]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX1-NEXT: vpsrad $7, %xmm0, %xmm2
; AVX1-NEXT: vpsrad $9, %xmm0, %xmm3
; AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm3[0,1,2,3],xmm2[4,5,6,7]
; AVX1-NEXT: vpsrad $8, %xmm0, %xmm0
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: constant_shift_v8i32:
; AVX2: # BB#0:
; AVX2-NEXT: vpsravd {{.*}}(%rip), %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: constant_shift_v8i32:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpshad {{.*}}(%rip), %xmm0, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; XOPAVX1-NEXT: vpshad {{.*}}(%rip), %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: constant_shift_v8i32:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpsravd {{.*}}(%rip), %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: constant_shift_v8i32:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: vpsravd {{.*}}(%rip), %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: constant_shift_v8i32:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsravd {{.*}}(%rip), %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: constant_shift_v8i32:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vpsrad $7, %xmm0, %xmm1
; X32-AVX1-NEXT: vpsrad $5, %xmm0, %xmm2
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm2[0,1,2,3],xmm1[4,5,6,7]
; X32-AVX1-NEXT: vpsrad $6, %xmm0, %xmm2
; X32-AVX1-NEXT: vpsrad $4, %xmm0, %xmm3
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm3[0,1,2,3],xmm2[4,5,6,7]
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm2[0,1],xmm1[2,3],xmm2[4,5],xmm1[6,7]
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; X32-AVX1-NEXT: vpsrad $7, %xmm0, %xmm2
; X32-AVX1-NEXT: vpsrad $9, %xmm0, %xmm3
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm2 = xmm3[0,1,2,3],xmm2[4,5,6,7]
; X32-AVX1-NEXT: vpsrad $8, %xmm0, %xmm0
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; X32-AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: constant_shift_v8i32:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpsravd {{\.LCPI.*}}, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <8 x i32> %a, <i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 8, i32 7>
ret <8 x i32> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <16 x i16> @constant_shift_v16i16(<16 x i16> %a) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: constant_shift_v16i16:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
[X86][SSE41] Avoid variable blend for constant v8i16 shifts The SSE41 v8i16 shift lowering using (v)pblendvb is great for non-constant shift amounts, but if it is constant then we can efficiently reduce the VSELECT to shuffles with the pre-SSE41 lowering. llvm-svn: 263383
2016-03-14 02:35:59 +08:00
; AVX1-NEXT: vpsraw $8, %xmm1, %xmm1
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpsraw $4, %xmm1, %xmm2
[X86][SSE41] Avoid variable blend for constant v8i16 shifts The SSE41 v8i16 shift lowering using (v)pblendvb is great for non-constant shift amounts, but if it is constant then we can efficiently reduce the VSELECT to shuffles with the pre-SSE41 lowering. llvm-svn: 263383
2016-03-14 02:35:59 +08:00
; AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0,1,2,3],xmm2[4,5,6,7]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpsraw $2, %xmm1, %xmm2
[X86][SSE41] Avoid variable blend for constant v8i16 shifts The SSE41 v8i16 shift lowering using (v)pblendvb is great for non-constant shift amounts, but if it is constant then we can efficiently reduce the VSELECT to shuffles with the pre-SSE41 lowering. llvm-svn: 263383
2016-03-14 02:35:59 +08:00
; AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0,1],xmm2[2,3],xmm1[4,5],xmm2[6,7]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpsraw $1, %xmm1, %xmm2
[X86][SSE41] Avoid variable blend for constant v8i16 shifts The SSE41 v8i16 shift lowering using (v)pblendvb is great for non-constant shift amounts, but if it is constant then we can efficiently reduce the VSELECT to shuffles with the pre-SSE41 lowering. llvm-svn: 263383
2016-03-14 02:35:59 +08:00
; AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0],xmm2[1],xmm1[2],xmm2[3],xmm1[4],xmm2[5],xmm1[6],xmm2[7]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpsraw $4, %xmm0, %xmm2
[X86][SSE41] Avoid variable blend for constant v8i16 shifts The SSE41 v8i16 shift lowering using (v)pblendvb is great for non-constant shift amounts, but if it is constant then we can efficiently reduce the VSELECT to shuffles with the pre-SSE41 lowering. llvm-svn: 263383
2016-03-14 02:35:59 +08:00
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm2[4,5,6,7]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpsraw $2, %xmm0, %xmm2
[X86][SSE41] Avoid variable blend for constant v8i16 shifts The SSE41 v8i16 shift lowering using (v)pblendvb is great for non-constant shift amounts, but if it is constant then we can efficiently reduce the VSELECT to shuffles with the pre-SSE41 lowering. llvm-svn: 263383
2016-03-14 02:35:59 +08:00
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpsraw $1, %xmm0, %xmm2
[X86][SSE41] Avoid variable blend for constant v8i16 shifts The SSE41 v8i16 shift lowering using (v)pblendvb is great for non-constant shift amounts, but if it is constant then we can efficiently reduce the VSELECT to shuffles with the pre-SSE41 lowering. llvm-svn: 263383
2016-03-14 02:35:59 +08:00
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0],xmm2[1],xmm0[2],xmm2[3],xmm0[4],xmm2[5],xmm0[6],xmm2[7]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: constant_shift_v16i16:
; AVX2: # BB#0:
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vmovdqa {{.*#+}} ymm2 = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
; AVX2-NEXT: vpunpckhwd {{.*#+}} ymm3 = ymm2[4],ymm1[4],ymm2[5],ymm1[5],ymm2[6],ymm1[6],ymm2[7],ymm1[7],ymm2[12],ymm1[12],ymm2[13],ymm1[13],ymm2[14],ymm1[14],ymm2[15],ymm1[15]
[X86][AVX2] Fix v16i16 SHL lowering (PR27730) The AVX2 v16i16 shift lowering works by unpacking to 2 x v8i32, performing the shift and then truncating the result. The unpacking is used to place the values in the upper 16-bits so that we can correctly sign-extend for SRA shifts. Unfortunately we weren't ensuring that the lower 16-bits were zero to ensure that SHL correctly shifts in zero bits. llvm-svn: 271796
2016-06-05 00:45:33 +08:00
; AVX2-NEXT: vpunpckhwd {{.*#+}} ymm4 = ymm1[4],ymm0[4],ymm1[5],ymm0[5],ymm1[6],ymm0[6],ymm1[7],ymm0[7],ymm1[12],ymm0[12],ymm1[13],ymm0[13],ymm1[14],ymm0[14],ymm1[15],ymm0[15]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vpsravd %ymm3, %ymm4, %ymm3
; AVX2-NEXT: vpsrld $16, %ymm3, %ymm3
[X86][AVX2] Fix v16i16 SHL lowering (PR27730) The AVX2 v16i16 shift lowering works by unpacking to 2 x v8i32, performing the shift and then truncating the result. The unpacking is used to place the values in the upper 16-bits so that we can correctly sign-extend for SRA shifts. Unfortunately we weren't ensuring that the lower 16-bits were zero to ensure that SHL correctly shifts in zero bits. llvm-svn: 271796
2016-06-05 00:45:33 +08:00
; AVX2-NEXT: vpunpcklwd {{.*#+}} ymm2 = ymm2[0],ymm1[0],ymm2[1],ymm1[1],ymm2[2],ymm1[2],ymm2[3],ymm1[3],ymm2[8],ymm1[8],ymm2[9],ymm1[9],ymm2[10],ymm1[10],ymm2[11],ymm1[11]
; AVX2-NEXT: vpunpcklwd {{.*#+}} ymm0 = ymm1[0],ymm0[0],ymm1[1],ymm0[1],ymm1[2],ymm0[2],ymm1[3],ymm0[3],ymm1[8],ymm0[8],ymm1[9],ymm0[9],ymm1[10],ymm0[10],ymm1[11],ymm0[11]
; AVX2-NEXT: vpsravd %ymm2, %ymm0, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vpsrld $16, %ymm0, %ymm0
; AVX2-NEXT: vpackusdw %ymm3, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: constant_shift_v16i16:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; XOPAVX1-NEXT: vpsubw {{.*}}(%rip), %xmm1, %xmm2
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; XOPAVX1-NEXT: vpshaw %xmm2, %xmm3, %xmm2
; XOPAVX1-NEXT: vpsubw {{.*}}(%rip), %xmm1, %xmm1
; XOPAVX1-NEXT: vpshaw %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: constant_shift_v16i16:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
; XOPAVX2-NEXT: vpsubw {{.*}}(%rip), %xmm1, %xmm2
; XOPAVX2-NEXT: vextracti128 $1, %ymm0, %xmm3
; XOPAVX2-NEXT: vpshaw %xmm2, %xmm3, %xmm2
; XOPAVX2-NEXT: vpsubw {{.*}}(%rip), %xmm1, %xmm1
; XOPAVX2-NEXT: vpshaw %xmm1, %xmm0, %xmm0
; XOPAVX2-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
[X86][AVX512DQ] Added AVX512DQ to 128/256 bit vector shift tests llvm-svn: 291444
2017-01-09 22:36:09 +08:00
; AVX512DQ-LABEL: constant_shift_v16i16:
; AVX512DQ: # BB#0:
[X86][AVX512DQ] Enable v16i16 vector shifts to use an extend+shift+truncate pattern. Use the existing AVX2 v8i16 vector shift lowering for v16i16 on AVX512 targets (AVX512BW will have already have lowered with vpsravw). Cost model updates to follow. llvm-svn: 291445
2017-01-09 23:15:45 +08:00
; AVX512DQ-NEXT: vpmovsxwd %ymm0, %zmm0
; AVX512DQ-NEXT: vpsravd {{.*}}(%rip), %zmm0, %zmm0
; AVX512DQ-NEXT: vpmovdw %zmm0, %ymm0
[X86][AVX512DQ] Added AVX512DQ to 128/256 bit vector shift tests llvm-svn: 291444
2017-01-09 22:36:09 +08:00
; AVX512DQ-NEXT: retq
;
; AVX512BW-LABEL: constant_shift_v16i16:
; AVX512BW: # BB#0:
; AVX512BW-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<def>
; AVX512BW-NEXT: vmovdqa {{.*#+}} ymm1 = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
; AVX512BW-NEXT: vpsravw %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<kill>
; AVX512BW-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512DQVL-LABEL: constant_shift_v16i16:
; AVX512DQVL: # BB#0:
; AVX512DQVL-NEXT: vpmovsxwd %ymm0, %zmm0
; AVX512DQVL-NEXT: vpsravd {{.*}}(%rip), %zmm0, %zmm0
; AVX512DQVL-NEXT: vpmovdw %zmm0, %ymm0
; AVX512DQVL-NEXT: retq
;
; AVX512BWVL-LABEL: constant_shift_v16i16:
; AVX512BWVL: # BB#0:
; AVX512BWVL-NEXT: vpsravw {{.*}}(%rip), %ymm0, %ymm0
; AVX512BWVL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: constant_shift_v16i16:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
; X32-AVX1-NEXT: vpsraw $8, %xmm1, %xmm1
; X32-AVX1-NEXT: vpsraw $4, %xmm1, %xmm2
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0,1,2,3],xmm2[4,5,6,7]
; X32-AVX1-NEXT: vpsraw $2, %xmm1, %xmm2
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0,1],xmm2[2,3],xmm1[4,5],xmm2[6,7]
; X32-AVX1-NEXT: vpsraw $1, %xmm1, %xmm2
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0],xmm2[1],xmm1[2],xmm2[3],xmm1[4],xmm2[5],xmm1[6],xmm2[7]
; X32-AVX1-NEXT: vpsraw $4, %xmm0, %xmm2
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm2[4,5,6,7]
; X32-AVX1-NEXT: vpsraw $2, %xmm0, %xmm2
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; X32-AVX1-NEXT: vpsraw $1, %xmm0, %xmm2
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0],xmm2[1],xmm0[2],xmm2[3],xmm0[4],xmm2[5],xmm0[6],xmm2[7]
; X32-AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: constant_shift_v16i16:
; X32-AVX2: # BB#0:
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; X32-AVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
; X32-AVX2-NEXT: vmovdqa {{.*#+}} ymm2 = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
; X32-AVX2-NEXT: vpunpckhwd {{.*#+}} ymm3 = ymm2[4],ymm1[4],ymm2[5],ymm1[5],ymm2[6],ymm1[6],ymm2[7],ymm1[7],ymm2[12],ymm1[12],ymm2[13],ymm1[13],ymm2[14],ymm1[14],ymm2[15],ymm1[15]
; X32-AVX2-NEXT: vpunpckhwd {{.*#+}} ymm4 = ymm1[4],ymm0[4],ymm1[5],ymm0[5],ymm1[6],ymm0[6],ymm1[7],ymm0[7],ymm1[12],ymm0[12],ymm1[13],ymm0[13],ymm1[14],ymm0[14],ymm1[15],ymm0[15]
; X32-AVX2-NEXT: vpsravd %ymm3, %ymm4, %ymm3
; X32-AVX2-NEXT: vpsrld $16, %ymm3, %ymm3
; X32-AVX2-NEXT: vpunpcklwd {{.*#+}} ymm2 = ymm2[0],ymm1[0],ymm2[1],ymm1[1],ymm2[2],ymm1[2],ymm2[3],ymm1[3],ymm2[8],ymm1[8],ymm2[9],ymm1[9],ymm2[10],ymm1[10],ymm2[11],ymm1[11]
; X32-AVX2-NEXT: vpunpcklwd {{.*#+}} ymm0 = ymm1[0],ymm0[0],ymm1[1],ymm0[1],ymm1[2],ymm0[2],ymm1[3],ymm0[3],ymm1[8],ymm0[8],ymm1[9],ymm0[9],ymm1[10],ymm0[10],ymm1[11],ymm0[11]
; X32-AVX2-NEXT: vpsravd %ymm2, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsrld $16, %ymm0, %ymm0
; X32-AVX2-NEXT: vpackusdw %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <16 x i16> %a, <i16 0, i16 1, i16 2, i16 3, i16 4, i16 5, i16 6, i16 7, i16 8, i16 9, i16 10, i16 11, i16 12, i16 13, i16 14, i16 15>
ret <16 x i16> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <32 x i8> @constant_shift_v32i8(<32 x i8> %a) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: constant_shift_v32i8:
; AVX1: # BB#0:
[X86][SSE] Add support for constant folding vector logical shift by immediates llvm-svn: 292915
2017-01-24 19:21:57 +08:00
; AVX1-NEXT: vmovdqa {{.*#+}} xmm1 = [8192,24640,41088,57536,49376,32928,16480,32]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm2 = xmm0[8],xmm1[8],xmm0[9],xmm1[9],xmm0[10],xmm1[10],xmm0[11],xmm1[11],xmm0[12],xmm1[12],xmm0[13],xmm1[13],xmm0[14],xmm1[14],xmm0[15],xmm1[15]
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm4 = xmm0[8],xmm3[8],xmm0[9],xmm3[9],xmm0[10],xmm3[10],xmm0[11],xmm3[11],xmm0[12],xmm3[12],xmm0[13],xmm3[13],xmm0[14],xmm3[14],xmm0[15],xmm3[15]
; AVX1-NEXT: vpsraw $4, %xmm4, %xmm5
; AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsraw $2, %xmm4, %xmm5
; AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm6
; AVX1-NEXT: vpblendvb %xmm6, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsraw $1, %xmm4, %xmm5
; AVX1-NEXT: vpaddw %xmm6, %xmm6, %xmm9
; AVX1-NEXT: vpblendvb %xmm9, %xmm5, %xmm4, %xmm4
; AVX1-NEXT: vpsrlw $8, %xmm4, %xmm8
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm1 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm3 = xmm0[0],xmm3[0],xmm0[1],xmm3[1],xmm0[2],xmm3[2],xmm0[3],xmm3[3],xmm0[4],xmm3[4],xmm0[5],xmm3[5],xmm0[6],xmm3[6],xmm0[7],xmm3[7]
; AVX1-NEXT: vpsraw $4, %xmm3, %xmm5
; AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm3, %xmm3
; AVX1-NEXT: vpsraw $2, %xmm3, %xmm5
; AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm4
; AVX1-NEXT: vpblendvb %xmm4, %xmm5, %xmm3, %xmm3
; AVX1-NEXT: vpsraw $1, %xmm3, %xmm5
; AVX1-NEXT: vpaddw %xmm4, %xmm4, %xmm7
; AVX1-NEXT: vpblendvb %xmm7, %xmm5, %xmm3, %xmm3
; AVX1-NEXT: vpsrlw $8, %xmm3, %xmm3
; AVX1-NEXT: vpackuswb %xmm8, %xmm3, %xmm8
; AVX1-NEXT: vpunpckhbw {{.*#+}} xmm5 = xmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
; AVX1-NEXT: vpsraw $4, %xmm5, %xmm3
; AVX1-NEXT: vpblendvb %xmm2, %xmm3, %xmm5, %xmm2
; AVX1-NEXT: vpsraw $2, %xmm2, %xmm3
; AVX1-NEXT: vpblendvb %xmm6, %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpsraw $1, %xmm2, %xmm3
; AVX1-NEXT: vpblendvb %xmm9, %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpsrlw $8, %xmm2, %xmm2
; AVX1-NEXT: vpunpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
; AVX1-NEXT: vpsraw $4, %xmm0, %xmm3
; AVX1-NEXT: vpblendvb %xmm1, %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $2, %xmm0, %xmm1
; AVX1-NEXT: vpblendvb %xmm4, %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpsraw $1, %xmm0, %xmm1
; AVX1-NEXT: vpblendvb %xmm7, %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpsrlw $8, %xmm0, %xmm0
; AVX1-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm8, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: constant_shift_v32i8:
; AVX2: # BB#0:
[X86][SSE] Add support for constant folding vector logical shift by immediates llvm-svn: 292915
2017-01-24 19:21:57 +08:00
; AVX2-NEXT: vmovdqa {{.*#+}} ymm1 = [8192,24640,41088,57536,49376,32928,16480,32,8192,24640,41088,57536,49376,32928,16480,32]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX2-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX2-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX2-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX2-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX2-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX2-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX2-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: constant_shift_v32i8:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; XOPAVX1-NEXT: vpsubb {{.*}}(%rip), %xmm1, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vpshab %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpshab %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: constant_shift_v32i8:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
; XOPAVX2-NEXT: vpsubb {{.*}}(%rip), %xmm1, %xmm1
; XOPAVX2-NEXT: vextracti128 $1, %ymm0, %xmm2
; XOPAVX2-NEXT: vpshab %xmm1, %xmm2, %xmm2
; XOPAVX2-NEXT: vpshab %xmm1, %xmm0, %xmm0
; XOPAVX2-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
[X86][AVX512] Enable v16i8/v32i8 vector shifts to use an extend+shift+truncate pattern. Use the existing AVX2 v8i16 vector shift lowering for v16i8 (extending to v16i32) on AVX512 targets and v32i8 (extending to v32i16) on AVX512BW targets. Cost model updates to follow. llvm-svn: 291451
2017-01-10 01:20:03 +08:00
; AVX512DQ-LABEL: constant_shift_v32i8:
; AVX512DQ: # BB#0:
[X86][SSE] Add support for constant folding vector logical shift by immediates llvm-svn: 292915
2017-01-24 19:21:57 +08:00
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm1 = [8192,24640,41088,57536,49376,32928,16480,32,8192,24640,41088,57536,49376,32928,16480,32]
[X86][AVX512] Enable v16i8/v32i8 vector shifts to use an extend+shift+truncate pattern. Use the existing AVX2 v8i16 vector shift lowering for v16i8 (extending to v16i32) on AVX512 targets and v32i8 (extending to v32i16) on AVX512BW targets. Cost model updates to follow. llvm-svn: 291451
2017-01-10 01:20:03 +08:00
; AVX512DQ-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX512DQ-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512DQ-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQ-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX512DQ-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQ-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX512DQ-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQ-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX512DQ-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512DQ-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX512DQ-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512DQ-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX512DQ-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX512DQ-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQ-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512DQ-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: retq
;
; AVX512BW-LABEL: constant_shift_v32i8:
; AVX512BW: # BB#0:
; AVX512BW-NEXT: vpmovsxbw %ymm0, %zmm0
; AVX512BW-NEXT: vpsravw {{.*}}(%rip), %zmm0, %zmm0
; AVX512BW-NEXT: vpmovwb %zmm0, %ymm0
; AVX512BW-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512DQVL-LABEL: constant_shift_v32i8:
; AVX512DQVL: # BB#0:
[X86][SSE] Add support for constant folding vector logical shift by immediates llvm-svn: 292915
2017-01-24 19:21:57 +08:00
; AVX512DQVL-NEXT: vmovdqa {{.*#+}} ymm1 = [8192,24640,41088,57536,49376,32928,16480,32,8192,24640,41088,57536,49376,32928,16480,32]
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
; AVX512DQVL-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; AVX512DQVL-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; AVX512DQVL-NEXT: vpsraw $4, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQVL-NEXT: vpsraw $2, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; AVX512DQVL-NEXT: vpsraw $1, %ymm3, %ymm4
; AVX512DQVL-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; AVX512DQVL-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512DQVL-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; AVX512DQVL-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; AVX512DQVL-NEXT: vpsraw $4, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsraw $2, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsraw $1, %ymm0, %ymm3
; AVX512DQVL-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; AVX512DQVL-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512DQVL-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX512DQVL-NEXT: retq
;
; AVX512BWVL-LABEL: constant_shift_v32i8:
; AVX512BWVL: # BB#0:
; AVX512BWVL-NEXT: vpmovsxbw %ymm0, %zmm0
; AVX512BWVL-NEXT: vpsravw {{.*}}(%rip), %zmm0, %zmm0
; AVX512BWVL-NEXT: vpmovwb %zmm0, %ymm0
; AVX512BWVL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: constant_shift_v32i8:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vmovdqa {{.*#+}} xmm1 = [8192,24640,41088,57536,49376,32928,16480,32]
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm2 = xmm0[8],xmm1[8],xmm0[9],xmm1[9],xmm0[10],xmm1[10],xmm0[11],xmm1[11],xmm0[12],xmm1[12],xmm0[13],xmm1[13],xmm0[14],xmm1[14],xmm0[15],xmm1[15]
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm4 = xmm0[8],xmm3[8],xmm0[9],xmm3[9],xmm0[10],xmm3[10],xmm0[11],xmm3[11],xmm0[12],xmm3[12],xmm0[13],xmm3[13],xmm0[14],xmm3[14],xmm0[15],xmm3[15]
; X32-AVX1-NEXT: vpsraw $4, %xmm4, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm5, %xmm4, %xmm4
; X32-AVX1-NEXT: vpunpckhbw {{.*#+}} xmm5 = xmm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
; X32-AVX1-NEXT: vpsraw $4, %xmm5, %xmm6
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm5, %xmm5
; X32-AVX1-NEXT: vpsraw $2, %xmm4, %xmm6
; X32-AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm2
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm4, %xmm4
; X32-AVX1-NEXT: vpsraw $2, %xmm5, %xmm6
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm5, %xmm5
; X32-AVX1-NEXT: vpsraw $1, %xmm4, %xmm6
; X32-AVX1-NEXT: vpaddw %xmm2, %xmm2, %xmm2
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm4, %xmm4
; X32-AVX1-NEXT: vpsraw $1, %xmm5, %xmm6
; X32-AVX1-NEXT: vpblendvb %xmm2, %xmm6, %xmm5, %xmm2
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm1 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm3 = xmm0[0],xmm3[0],xmm0[1],xmm3[1],xmm0[2],xmm3[2],xmm0[3],xmm3[3],xmm0[4],xmm3[4],xmm0[5],xmm3[5],xmm0[6],xmm3[6],xmm0[7],xmm3[7]
; X32-AVX1-NEXT: vpsraw $4, %xmm3, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm3, %xmm3
; X32-AVX1-NEXT: vpunpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
; X32-AVX1-NEXT: vpsraw $4, %xmm0, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $2, %xmm3, %xmm5
; X32-AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm1
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm3, %xmm3
; X32-AVX1-NEXT: vpsraw $2, %xmm0, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsraw $1, %xmm3, %xmm5
; X32-AVX1-NEXT: vpaddw %xmm1, %xmm1, %xmm1
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm3, %xmm3
; X32-AVX1-NEXT: vpsraw $1, %xmm0, %xmm5
; X32-AVX1-NEXT: vpblendvb %xmm1, %xmm5, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsrlw $8, %xmm4, %xmm1
; X32-AVX1-NEXT: vpsrlw $8, %xmm3, %xmm3
; X32-AVX1-NEXT: vpackuswb %xmm1, %xmm3, %xmm1
; X32-AVX1-NEXT: vpsrlw $8, %xmm2, %xmm2
; X32-AVX1-NEXT: vpsrlw $8, %xmm0, %xmm0
; X32-AVX1-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: constant_shift_v32i8:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vmovdqa {{.*#+}} ymm1 = [8192,24640,41088,57536,49376,32928,16480,32,8192,24640,41088,57536,49376,32928,16480,32]
; X32-AVX2-NEXT: vpunpckhbw {{.*#+}} ymm2 = ymm0[8],ymm1[8],ymm0[9],ymm1[9],ymm0[10],ymm1[10],ymm0[11],ymm1[11],ymm0[12],ymm1[12],ymm0[13],ymm1[13],ymm0[14],ymm1[14],ymm0[15],ymm1[15],ymm0[24],ymm1[24],ymm0[25],ymm1[25],ymm0[26],ymm1[26],ymm0[27],ymm1[27],ymm0[28],ymm1[28],ymm0[29],ymm1[29],ymm0[30],ymm1[30],ymm0[31],ymm1[31]
; X32-AVX2-NEXT: vpunpckhbw {{.*#+}} ymm3 = ymm0[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15,24,24,25,25,26,26,27,27,28,28,29,29,30,30,31,31]
; X32-AVX2-NEXT: vpsraw $4, %ymm3, %ymm4
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; X32-AVX2-NEXT: vpsraw $2, %ymm3, %ymm4
; X32-AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm3
; X32-AVX2-NEXT: vpsraw $1, %ymm3, %ymm4
; X32-AVX2-NEXT: vpaddw %ymm2, %ymm2, %ymm2
; X32-AVX2-NEXT: vpblendvb %ymm2, %ymm4, %ymm3, %ymm2
; X32-AVX2-NEXT: vpsrlw $8, %ymm2, %ymm2
; X32-AVX2-NEXT: vpunpcklbw {{.*#+}} ymm1 = ymm0[0],ymm1[0],ymm0[1],ymm1[1],ymm0[2],ymm1[2],ymm0[3],ymm1[3],ymm0[4],ymm1[4],ymm0[5],ymm1[5],ymm0[6],ymm1[6],ymm0[7],ymm1[7],ymm0[16],ymm1[16],ymm0[17],ymm1[17],ymm0[18],ymm1[18],ymm0[19],ymm1[19],ymm0[20],ymm1[20],ymm0[21],ymm1[21],ymm0[22],ymm1[22],ymm0[23],ymm1[23]
; X32-AVX2-NEXT: vpunpcklbw {{.*#+}} ymm0 = ymm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7,16,16,17,17,18,18,19,19,20,20,21,21,22,22,23,23]
; X32-AVX2-NEXT: vpsraw $4, %ymm0, %ymm3
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsraw $2, %ymm0, %ymm3
; X32-AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsraw $1, %ymm0, %ymm3
; X32-AVX2-NEXT: vpaddw %ymm1, %ymm1, %ymm1
; X32-AVX2-NEXT: vpblendvb %ymm1, %ymm3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsrlw $8, %ymm0, %ymm0
; X32-AVX2-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <32 x i8> %a, <i8 0, i8 1, i8 2, i8 3, i8 4, i8 5, i8 6, i8 7, i8 7, i8 6, i8 5, i8 4, i8 3, i8 2, i8 1, i8 0, i8 0, i8 1, i8 2, i8 3, i8 4, i8 5, i8 6, i8 7, i8 7, i8 6, i8 5, i8 4, i8 3, i8 2, i8 1, i8 0>
ret <32 x i8> %shift
}
;
; Uniform Constant Shifts
;
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <4 x i64> @splatconstant_shift_v4i64(<4 x i64> %a) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: splatconstant_shift_v4i64:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
[X86][SSE] Vector shift test cleanup. NFC. llvm-svn: 241730
2015-07-09 05:11:17 +08:00
; AVX1-NEXT: vpsrad $7, %xmm1, %xmm2
; AVX1-NEXT: vpsrlq $7, %xmm1, %xmm1
; AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0,1],xmm2[2,3],xmm1[4,5],xmm2[6,7]
; AVX1-NEXT: vpsrad $7, %xmm0, %xmm2
; AVX1-NEXT: vpsrlq $7, %xmm0, %xmm0
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: splatconstant_shift_v4i64:
; AVX2: # BB#0:
[X86][SSE] Vector shift test cleanup. NFC. llvm-svn: 241730
2015-07-09 05:11:17 +08:00
; AVX2-NEXT: vpsrad $7, %ymm0, %ymm1
; AVX2-NEXT: vpsrlq $7, %ymm0, %ymm0
[X86][SSE] Vectorized i64 uniform constant SRA shifts This patch adds vectorization support for uniform constant i64 arithmetic shift right operators. Differential Revision: http://reviews.llvm.org/D9645 llvm-svn: 241514
2015-07-07 06:35:19 +08:00
; AVX2-NEXT: vpblendd {{.*#+}} ymm0 = ymm0[0],ymm1[1],ymm0[2],ymm1[3],ymm0[4],ymm1[5],ymm0[6],ymm1[7]
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: splatconstant_shift_v4i64:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; XOPAVX1-NEXT: vpsubq {{.*}}(%rip), %xmm1, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpshaq %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: splatconstant_shift_v4i64:
; XOPAVX2: # BB#0:
[X86][XOP] Use default AVX2 lowering for v4i64 ashr by splat constants XOP shifts only support 128-bit vectors, so we were ending up with less optimal codegen requiring constants llvm-svn: 308430
2017-07-19 18:29:31 +08:00
; XOPAVX2-NEXT: vpsrad $7, %ymm0, %ymm1
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
; XOPAVX2-NEXT: vpsrlq $7, %ymm0, %ymm0
[X86][XOP] Use default AVX2 lowering for v4i64 ashr by splat constants XOP shifts only support 128-bit vectors, so we were ending up with less optimal codegen requiring constants llvm-svn: 308430
2017-07-19 18:29:31 +08:00
; XOPAVX2-NEXT: vpblendd {{.*#+}} ymm0 = ymm0[0],ymm1[1],ymm0[2],ymm1[3],ymm0[4],ymm1[5],ymm0[6],ymm1[7]
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: splatconstant_shift_v4i64:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
[X86][AVX512] Add support for ASHR v2i64/v4i64 support without VLX Use v8i64 ASHR instructions if we don't have VLX. Differential Revision: https://reviews.llvm.org/D28537 llvm-svn: 295656
2017-02-20 20:16:38 +08:00
; AVX512-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<def>
; AVX512-NEXT: vpsraq $7, %zmm0, %zmm0
; AVX512-NEXT: # kill: %YMM0<def> %YMM0<kill> %ZMM0<kill>
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: splatconstant_shift_v4i64:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsraq $7, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: splatconstant_shift_v4i64:
; X32-AVX1: # BB#0:
[X86][AVX] Allow 32-bit targets to peek through subvectors to extract constant splats for vXi64 shifts. llvm-svn: 303009
2017-05-14 19:46:26 +08:00
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
; X32-AVX1-NEXT: vpsrad $7, %xmm1, %xmm2
; X32-AVX1-NEXT: vpsrlq $7, %xmm1, %xmm1
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm1 = xmm1[0,1],xmm2[2,3],xmm1[4,5],xmm2[6,7]
; X32-AVX1-NEXT: vpsrad $7, %xmm0, %xmm2
; X32-AVX1-NEXT: vpsrlq $7, %xmm0, %xmm0
; X32-AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; X32-AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: splatconstant_shift_v4i64:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpsrad $7, %ymm0, %ymm1
; X32-AVX2-NEXT: vpsrlq $7, %ymm0, %ymm0
; X32-AVX2-NEXT: vpblendd {{.*#+}} ymm0 = ymm0[0],ymm1[1],ymm0[2],ymm1[3],ymm0[4],ymm1[5],ymm0[6],ymm1[7]
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <4 x i64> %a, <i64 7, i64 7, i64 7, i64 7>
ret <4 x i64> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <8 x i32> @splatconstant_shift_v8i32(<8 x i32> %a) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: splatconstant_shift_v8i32:
; AVX1: # BB#0:
; AVX1-NEXT: vpsrad $5, %xmm0, %xmm1
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX1-NEXT: vpsrad $5, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: splatconstant_shift_v8i32:
; AVX2: # BB#0:
; AVX2-NEXT: vpsrad $5, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: splatconstant_shift_v8i32:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpsrad $5, %xmm0, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; XOPAVX1-NEXT: vpsrad $5, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: splatconstant_shift_v8i32:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpsrad $5, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: splatconstant_shift_v8i32:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: vpsrad $5, %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: splatconstant_shift_v8i32:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsrad $5, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: splatconstant_shift_v8i32:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vpsrad $5, %xmm0, %xmm1
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; X32-AVX1-NEXT: vpsrad $5, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: splatconstant_shift_v8i32:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpsrad $5, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <8 x i32> %a, <i32 5, i32 5, i32 5, i32 5, i32 5, i32 5, i32 5, i32 5>
ret <8 x i32> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <16 x i16> @splatconstant_shift_v16i16(<16 x i16> %a) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: splatconstant_shift_v16i16:
; AVX1: # BB#0:
; AVX1-NEXT: vpsraw $3, %xmm0, %xmm1
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX1-NEXT: vpsraw $3, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: splatconstant_shift_v16i16:
; AVX2: # BB#0:
; AVX2-NEXT: vpsraw $3, %ymm0, %ymm0
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: splatconstant_shift_v16i16:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpsraw $3, %xmm0, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; XOPAVX1-NEXT: vpsraw $3, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: splatconstant_shift_v16i16:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpsraw $3, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: splatconstant_shift_v16i16:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: vpsraw $3, %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: splatconstant_shift_v16i16:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsraw $3, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: splatconstant_shift_v16i16:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vpsraw $3, %xmm0, %xmm1
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm0
; X32-AVX1-NEXT: vpsraw $3, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: splatconstant_shift_v16i16:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpsraw $3, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <16 x i16> %a, <i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3, i16 3>
ret <16 x i16> %shift
}
[X86][SSE] Added nounwind attribute to vector shift tests. Stop i686 codegen from generating cfi directives. llvm-svn: 242443
2015-07-17 05:14:26 +08:00
define <32 x i8> @splatconstant_shift_v32i8(<32 x i8> %a) nounwind {
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX1-LABEL: splatconstant_shift_v32i8:
; AVX1: # BB#0:
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX1-NEXT: vpsrlw $3, %xmm1, %xmm1
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31]
; AVX1-NEXT: vpand %xmm2, %xmm1, %xmm1
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; AVX1-NEXT: vpxor %xmm3, %xmm1, %xmm1
; AVX1-NEXT: vpsubb %xmm3, %xmm1, %xmm1
; AVX1-NEXT: vpsrlw $3, %xmm0, %xmm0
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpxor %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpsubb %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: splatconstant_shift_v32i8:
; AVX2: # BB#0:
[X86][SSE] Vector shift test cleanup. NFC. llvm-svn: 241730
2015-07-09 05:11:17 +08:00
; AVX2-NEXT: vpsrlw $3, %ymm0, %ymm0
; AVX2-NEXT: vpand {{.*}}(%rip), %ymm0, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: vmovdqa {{.*#+}} ymm1 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
[X86][SSE] Vector shift test cleanup. NFC. llvm-svn: 241730
2015-07-09 05:11:17 +08:00
; AVX2-NEXT: vpxor %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpsubb %ymm1, %ymm0, %ymm0
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
; AVX2-NEXT: retq
[X86][XOP] Added support for the lowering of 128-bit vector shifts to XOP shift instructions The XOP shifts just have logical/arithmetic versions and the left/right shifts are controlled by whether the value is positive/negative. Because of this I've added new X86ISD nodes instead of trying to force them to use the existing shift nodes. Additionally Excavator cores (bdver4) support XOP and AVX2 - meaning that it should use the AVX2 shifts when it can and fall back to XOP in other cases. Differential Revision: http://reviews.llvm.org/D8690 llvm-svn: 248878
2015-09-30 16:17:50 +08:00
;
; XOPAVX1-LABEL: splatconstant_shift_v32i8:
; XOPAVX1: # BB#0:
; XOPAVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; XOPAVX1-NEXT: vpsubb {{.*}}(%rip), %xmm1, %xmm1
; XOPAVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOPAVX1-NEXT: vpshab %xmm1, %xmm2, %xmm2
; XOPAVX1-NEXT: vpshab %xmm1, %xmm0, %xmm0
; XOPAVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; XOPAVX1-NEXT: retq
;
; XOPAVX2-LABEL: splatconstant_shift_v32i8:
; XOPAVX2: # BB#0:
; XOPAVX2-NEXT: vpsrlw $3, %ymm0, %ymm0
; XOPAVX2-NEXT: vpand {{.*}}(%rip), %ymm0, %ymm0
; XOPAVX2-NEXT: vmovdqa {{.*#+}} ymm1 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; XOPAVX2-NEXT: vpxor %ymm1, %ymm0, %ymm0
; XOPAVX2-NEXT: vpsubb %ymm1, %ymm0, %ymm0
; XOPAVX2-NEXT: retq
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
;
; AVX512-LABEL: splatconstant_shift_v32i8:
[X86][SSE] Standardized triples in vector shift tests Made no sense for them to be different and caused useless diffs in assembly remarks. llvm-svn: 291274
2017-01-07 03:56:57 +08:00
; AVX512: # BB#0:
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
; AVX512-NEXT: vpsrlw $3, %ymm0, %ymm0
; AVX512-NEXT: vpand {{.*}}(%rip), %ymm0, %ymm0
; AVX512-NEXT: vmovdqa {{.*#+}} ymm1 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; AVX512-NEXT: vpxor %ymm1, %ymm0, %ymm0
; AVX512-NEXT: vpsubb %ymm1, %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX512VL] Added AVX512VL to 128/256 bit vector shift tests llvm-svn: 291488
2017-01-10 06:13:51 +08:00
;
; AVX512VL-LABEL: splatconstant_shift_v32i8:
; AVX512VL: # BB#0:
; AVX512VL-NEXT: vpsrlw $3, %ymm0, %ymm0
; AVX512VL-NEXT: vpand {{.*}}(%rip), %ymm0, %ymm0
; AVX512VL-NEXT: vmovdqa {{.*#+}} ymm1 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; AVX512VL-NEXT: vpxor %ymm1, %ymm0, %ymm0
; AVX512VL-NEXT: vpsubb %ymm1, %ymm0, %ymm0
; AVX512VL-NEXT: retq
[X86][AVX] Add additional 32-bit target vector shift tests Shows issue with 32-bits not being able to peek through subvectors to extract constant splats llvm-svn: 303008
2017-05-14 19:13:03 +08:00
;
; X32-AVX1-LABEL: splatconstant_shift_v32i8:
; X32-AVX1: # BB#0:
; X32-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
; X32-AVX1-NEXT: vpsrlw $3, %xmm1, %xmm1
; X32-AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31]
; X32-AVX1-NEXT: vpand %xmm2, %xmm1, %xmm1
; X32-AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; X32-AVX1-NEXT: vpxor %xmm3, %xmm1, %xmm1
; X32-AVX1-NEXT: vpsubb %xmm3, %xmm1, %xmm1
; X32-AVX1-NEXT: vpsrlw $3, %xmm0, %xmm0
; X32-AVX1-NEXT: vpand %xmm2, %xmm0, %xmm0
; X32-AVX1-NEXT: vpxor %xmm3, %xmm0, %xmm0
; X32-AVX1-NEXT: vpsubb %xmm3, %xmm0, %xmm0
; X32-AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; X32-AVX1-NEXT: retl
;
; X32-AVX2-LABEL: splatconstant_shift_v32i8:
; X32-AVX2: # BB#0:
; X32-AVX2-NEXT: vpsrlw $3, %ymm0, %ymm0
; X32-AVX2-NEXT: vpand {{\.LCPI.*}}, %ymm0, %ymm0
; X32-AVX2-NEXT: vmovdqa {{.*#+}} ymm1 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; X32-AVX2-NEXT: vpxor %ymm1, %ymm0, %ymm0
; X32-AVX2-NEXT: vpsubb %ymm1, %ymm0, %ymm0
; X32-AVX2-NEXT: retl
[X86][AVX] Added full set of 256-bit vector shift tests. llvm-svn: 240542
2015-06-24 21:52:25 +08:00
%shift = ashr <32 x i8> %a, <i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3, i8 3>
ret <32 x i8> %shift
}