maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/var-permute-256.ll

1165 lines
53 KiB
LLVM
Raw Normal View History

X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+xop | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,XOP
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,AVX1
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx2 | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,INT256,AVX2
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512f | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,INT256,AVX512,AVX512F
[X86][AVX512DQ] Add DQ var permute 256 tests as requested on D42487 llvm-svn: 323970
2018-02-01 22:44:50 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,INT256,AVX512,AVX512DQ
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512bw | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,INT256,AVX512,AVX512BW
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512vbmi | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,INT256,AVX512,VBMI
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512f,+avx512vl | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,INT256,AVX512VL
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,+avx512vl | FileCheck %s --check-prefixes=AVX,AVXNOVLBW,INT256,AVX512VL,AVX512VLDQ
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512bw,+avx512vl | FileCheck %s --check-prefixes=AVX,INT256,AVX512VL,AVX512VLBW
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512bw,+avx512vl,+avx512vbmi | FileCheck %s --check-prefixes=AVX,INT256,AVX512VL,VLVBMI
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
define <4 x i64> @var_shuffle_v4i64(<4 x i64> %v, <4 x i64> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v4i64:
; XOP: # %bb.0:
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpaddq %xmm1, %xmm1, %xmm3
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: vextractf128 $1, %ymm1, %xmm1
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: vpaddq %xmm1, %xmm1, %xmm1
; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; XOP-NEXT: vpermil2pd $0, %ymm1, %ymm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-LABEL: var_shuffle_v4i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: # %bb.0:
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpaddq %xmm1, %xmm1, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vpaddq %xmm1, %xmm1, %xmm1
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm4
; AVX1-NEXT: vpermilpd %ymm4, %ymm2, %ymm2
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpermilpd %ymm4, %ymm0, %ymm0
; AVX1-NEXT: vpcmpgtq {{.*}}(%rip), %xmm3, %xmm3
; AVX1-NEXT: vpcmpgtq {{\.LCPI.*}}+{{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; AVX1-NEXT: vblendvpd %ymm1, %ymm2, %ymm0, %ymm0
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-NEXT: retq
;
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX2-LABEL: var_shuffle_v4i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: # %bb.0:
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX2-NEXT: vpaddq %ymm1, %ymm1, %ymm1
[X86][SSE] LowerBUILD_VECTORAsVariablePermute - add support for scaling index vectors This allows us to use PSHUFB for v8i16/v4i32 and VPERMD/PERMPS for v4i64/v4f64 variable shuffles. Differential Revision: https://reviews.llvm.org/D42487 llvm-svn: 323987
2018-02-02 02:10:30 +08:00
; AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [2,2,2,2]
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX2-NEXT: vpcmpgtq %ymm2, %ymm1, %ymm2
; AVX2-NEXT: vpermpd {{.*#+}} ymm3 = ymm0[2,3,2,3]
; AVX2-NEXT: vpermilpd %ymm1, %ymm3, %ymm3
; AVX2-NEXT: vpermpd {{.*#+}} ymm0 = ymm0[0,1,0,1]
; AVX2-NEXT: vpermilpd %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vblendvpd %ymm2, %ymm3, %ymm0, %ymm0
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX2-NEXT: retq
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512-LABEL: var_shuffle_v4i64:
; AVX512: # %bb.0:
; AVX512-NEXT: # kill: def $ymm1 killed $ymm1 def $zmm1
; AVX512-NEXT: # kill: def $ymm0 killed $ymm0 def $zmm0
; AVX512-NEXT: vpermpd %zmm0, %zmm1, %zmm0
; AVX512-NEXT: # kill: def $ymm0 killed $ymm0 killed $zmm0
; AVX512-NEXT: retq
[X86][AVX512DQ] Add DQ var permute 256 tests as requested on D42487 llvm-svn: 323970
2018-02-01 22:44:50 +08:00
;
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX512VL-LABEL: var_shuffle_v4i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512VL: # %bb.0:
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX512VL-NEXT: vpermpd %ymm0, %ymm1, %ymm0
; AVX512VL-NEXT: retq
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
%index0 = extractelement <4 x i64> %indices, i32 0
%index1 = extractelement <4 x i64> %indices, i32 1
%index2 = extractelement <4 x i64> %indices, i32 2
%index3 = extractelement <4 x i64> %indices, i32 3
%v0 = extractelement <4 x i64> %v, i64 %index0
%v1 = extractelement <4 x i64> %v, i64 %index1
%v2 = extractelement <4 x i64> %v, i64 %index2
%v3 = extractelement <4 x i64> %v, i64 %index3
%ret0 = insertelement <4 x i64> undef, i64 %v0, i32 0
%ret1 = insertelement <4 x i64> %ret0, i64 %v1, i32 1
%ret2 = insertelement <4 x i64> %ret1, i64 %v2, i32 2
%ret3 = insertelement <4 x i64> %ret2, i64 %v3, i32 3
ret <4 x i64> %ret3
}
define <8 x i32> @var_shuffle_v8i32(<8 x i32> %v, <8 x i32> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v8i32:
; XOP: # %bb.0:
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpermil2ps $0, %ymm1, %ymm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-LABEL: var_shuffle_v8i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: # %bb.0:
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpermilps %ymm1, %ymm2, %ymm2
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; AVX1-NEXT: vpermilps %ymm1, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpcmpgtd {{.*}}(%rip), %xmm1, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
; AVX1-NEXT: vpcmpgtd {{\.LCPI.*}}+{{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; AVX1-NEXT: vblendvps %ymm1, %ymm2, %ymm0, %ymm0
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-NEXT: retq
;
; INT256-LABEL: var_shuffle_v8i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; INT256: # %bb.0:
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; INT256-NEXT: vpermps %ymm0, %ymm1, %ymm0
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; INT256-NEXT: retq
%index0 = extractelement <8 x i32> %indices, i32 0
%index1 = extractelement <8 x i32> %indices, i32 1
%index2 = extractelement <8 x i32> %indices, i32 2
%index3 = extractelement <8 x i32> %indices, i32 3
%index4 = extractelement <8 x i32> %indices, i32 4
%index5 = extractelement <8 x i32> %indices, i32 5
%index6 = extractelement <8 x i32> %indices, i32 6
%index7 = extractelement <8 x i32> %indices, i32 7
%v0 = extractelement <8 x i32> %v, i32 %index0
%v1 = extractelement <8 x i32> %v, i32 %index1
%v2 = extractelement <8 x i32> %v, i32 %index2
%v3 = extractelement <8 x i32> %v, i32 %index3
%v4 = extractelement <8 x i32> %v, i32 %index4
%v5 = extractelement <8 x i32> %v, i32 %index5
%v6 = extractelement <8 x i32> %v, i32 %index6
%v7 = extractelement <8 x i32> %v, i32 %index7
%ret0 = insertelement <8 x i32> undef, i32 %v0, i32 0
%ret1 = insertelement <8 x i32> %ret0, i32 %v1, i32 1
%ret2 = insertelement <8 x i32> %ret1, i32 %v2, i32 2
%ret3 = insertelement <8 x i32> %ret2, i32 %v3, i32 3
%ret4 = insertelement <8 x i32> %ret3, i32 %v4, i32 4
%ret5 = insertelement <8 x i32> %ret4, i32 %v5, i32 5
%ret6 = insertelement <8 x i32> %ret5, i32 %v6, i32 6
%ret7 = insertelement <8 x i32> %ret6, i32 %v7, i32 7
ret <8 x i32> %ret7
}
define <16 x i16> @var_shuffle_v16i16(<16 x i16> %v, <16 x i16> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v16i16:
; XOP: # %bb.0:
[X86][XOP] createVariablePermute - use VPPERM for v16i16 variable permutes llvm-svn: 327218
2018-03-11 02:33:29 +08:00
; XOP-NEXT: vmovdqa {{.*#+}} xmm2 = [256,256,256,256,256,256,256,256]
; XOP-NEXT: vmovdqa {{.*#+}} xmm3 = [514,514,514,514,514,514,514,514]
; XOP-NEXT: vpmacsww %xmm2, %xmm3, %xmm1, %xmm4
; XOP-NEXT: vextractf128 $1, %ymm1, %xmm1
; XOP-NEXT: vpmacsww %xmm2, %xmm3, %xmm1, %xmm1
; XOP-NEXT: vextractf128 $1, %ymm0, %xmm2
; XOP-NEXT: vpperm %xmm1, %xmm2, %xmm0, %xmm1
; XOP-NEXT: vpperm %xmm4, %xmm2, %xmm0, %xmm0
; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-LABEL: var_shuffle_v16i16:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: # %bb.0:
[X86][AVX] createVariablePermute - scale v16i16 variable permutes to use v32i8 codegen XOP was already doing this, and now AVX performs v32i8 variable permutes as well. llvm-svn: 327245
2018-03-12 01:23:54 +08:00
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [514,514,514,514,514,514,514,514]
; AVX1-NEXT: vpmullw %xmm2, %xmm1, %xmm3
; AVX1-NEXT: vmovdqa {{.*#+}} xmm4 = [256,256,256,256,256,256,256,256]
; AVX1-NEXT: vpaddw %xmm4, %xmm3, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
; AVX1-NEXT: vpmullw %xmm2, %xmm1, %xmm1
; AVX1-NEXT: vpaddw %xmm4, %xmm1, %xmm1
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX1-NEXT: vpcmpgtb %xmm2, %xmm1, %xmm4
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm5
; AVX1-NEXT: vpshufb %xmm1, %xmm5, %xmm6
; AVX1-NEXT: vpshufb %xmm1, %xmm0, %xmm1
; AVX1-NEXT: vpblendvb %xmm4, %xmm6, %xmm1, %xmm1
; AVX1-NEXT: vpcmpgtb %xmm2, %xmm3, %xmm2
; AVX1-NEXT: vpshufb %xmm3, %xmm5, %xmm4
; AVX1-NEXT: vpshufb %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpblendvb %xmm2, %xmm4, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-NEXT: retq
;
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX2-LABEL: var_shuffle_v16i16:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: # %bb.0:
[X86][AVX] createVariablePermute - scale v16i16 variable permutes to use v32i8 codegen XOP was already doing this, and now AVX performs v32i8 variable permutes as well. llvm-svn: 327245
2018-03-12 01:23:54 +08:00
; AVX2-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX2-NEXT: vpaddw {{.*}}(%rip), %ymm1, %ymm1
; AVX2-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm2
; AVX2-NEXT: vpshufb %ymm1, %ymm2, %ymm2
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm3
; AVX2-NEXT: vpblendd {{.*#+}} ymm0 = ymm3[0,1,2,3],ymm0[4,5,6,7]
; AVX2-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm0, %ymm2, %ymm0
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX2-NEXT: retq
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512-LABEL: var_shuffle_v16i16:
; AVX512: # %bb.0:
; AVX512-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX512-NEXT: vpaddw {{.*}}(%rip), %ymm1, %ymm1
; AVX512-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm2
; AVX512-NEXT: vpshufb %ymm1, %ymm2, %ymm2
; AVX512-NEXT: vextracti128 $1, %ymm0, %xmm3
; AVX512-NEXT: vpblendd {{.*#+}} ymm0 = ymm3[0,1,2,3],ymm0[4,5,6,7]
; AVX512-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX512-NEXT: vpblendvb %ymm1, %ymm0, %ymm2, %ymm0
; AVX512-NEXT: retq
[X86][AVX512DQ] Add DQ var permute 256 tests as requested on D42487 llvm-svn: 323970
2018-02-01 22:44:50 +08:00
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512VLDQ-LABEL: var_shuffle_v16i16:
; AVX512VLDQ: # %bb.0:
; AVX512VLDQ-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX512VLDQ-NEXT: vpaddw {{.*}}(%rip), %ymm1, %ymm1
; AVX512VLDQ-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm2
; AVX512VLDQ-NEXT: vpshufb %ymm1, %ymm2, %ymm2
; AVX512VLDQ-NEXT: vextracti128 $1, %ymm0, %xmm3
; AVX512VLDQ-NEXT: vpblendd {{.*#+}} ymm0 = ymm3[0,1,2,3],ymm0[4,5,6,7]
; AVX512VLDQ-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512VLDQ-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX512VLDQ-NEXT: vpblendvb %ymm1, %ymm0, %ymm2, %ymm0
; AVX512VLDQ-NEXT: retq
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
;
; AVX512VLBW-LABEL: var_shuffle_v16i16:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512VLBW: # %bb.0:
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX512VLBW-NEXT: vpermw %ymm0, %ymm1, %ymm0
; AVX512VLBW-NEXT: retq
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
;
; VLVBMI-LABEL: var_shuffle_v16i16:
; VLVBMI: # %bb.0:
; VLVBMI-NEXT: vpermw %ymm0, %ymm1, %ymm0
; VLVBMI-NEXT: retq
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
%index0 = extractelement <16 x i16> %indices, i32 0
%index1 = extractelement <16 x i16> %indices, i32 1
%index2 = extractelement <16 x i16> %indices, i32 2
%index3 = extractelement <16 x i16> %indices, i32 3
%index4 = extractelement <16 x i16> %indices, i32 4
%index5 = extractelement <16 x i16> %indices, i32 5
%index6 = extractelement <16 x i16> %indices, i32 6
%index7 = extractelement <16 x i16> %indices, i32 7
%index8 = extractelement <16 x i16> %indices, i32 8
%index9 = extractelement <16 x i16> %indices, i32 9
%index10 = extractelement <16 x i16> %indices, i32 10
%index11 = extractelement <16 x i16> %indices, i32 11
%index12 = extractelement <16 x i16> %indices, i32 12
%index13 = extractelement <16 x i16> %indices, i32 13
%index14 = extractelement <16 x i16> %indices, i32 14
%index15 = extractelement <16 x i16> %indices, i32 15
%v0 = extractelement <16 x i16> %v, i16 %index0
%v1 = extractelement <16 x i16> %v, i16 %index1
%v2 = extractelement <16 x i16> %v, i16 %index2
%v3 = extractelement <16 x i16> %v, i16 %index3
%v4 = extractelement <16 x i16> %v, i16 %index4
%v5 = extractelement <16 x i16> %v, i16 %index5
%v6 = extractelement <16 x i16> %v, i16 %index6
%v7 = extractelement <16 x i16> %v, i16 %index7
%v8 = extractelement <16 x i16> %v, i16 %index8
%v9 = extractelement <16 x i16> %v, i16 %index9
%v10 = extractelement <16 x i16> %v, i16 %index10
%v11 = extractelement <16 x i16> %v, i16 %index11
%v12 = extractelement <16 x i16> %v, i16 %index12
%v13 = extractelement <16 x i16> %v, i16 %index13
%v14 = extractelement <16 x i16> %v, i16 %index14
%v15 = extractelement <16 x i16> %v, i16 %index15
%ret0 = insertelement <16 x i16> undef, i16 %v0, i32 0
%ret1 = insertelement <16 x i16> %ret0, i16 %v1, i32 1
%ret2 = insertelement <16 x i16> %ret1, i16 %v2, i32 2
%ret3 = insertelement <16 x i16> %ret2, i16 %v3, i32 3
%ret4 = insertelement <16 x i16> %ret3, i16 %v4, i32 4
%ret5 = insertelement <16 x i16> %ret4, i16 %v5, i32 5
%ret6 = insertelement <16 x i16> %ret5, i16 %v6, i32 6
%ret7 = insertelement <16 x i16> %ret6, i16 %v7, i32 7
%ret8 = insertelement <16 x i16> %ret7, i16 %v8, i32 8
%ret9 = insertelement <16 x i16> %ret8, i16 %v9, i32 9
%ret10 = insertelement <16 x i16> %ret9, i16 %v10, i32 10
%ret11 = insertelement <16 x i16> %ret10, i16 %v11, i32 11
%ret12 = insertelement <16 x i16> %ret11, i16 %v12, i32 12
%ret13 = insertelement <16 x i16> %ret12, i16 %v13, i32 13
%ret14 = insertelement <16 x i16> %ret13, i16 %v14, i32 14
%ret15 = insertelement <16 x i16> %ret14, i16 %v15, i32 15
ret <16 x i16> %ret15
}
define <32 x i8> @var_shuffle_v32i8(<32 x i8> %v, <32 x i8> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v32i8:
; XOP: # %bb.0:
; XOP-NEXT: vextractf128 $1, %ymm1, %xmm2
[X86][XOP] createVariablePermute - use VPPERM for v32i8 variable permutes llvm-svn: 327213
2018-03-11 00:51:45 +08:00
; XOP-NEXT: vextractf128 $1, %ymm0, %xmm3
; XOP-NEXT: vpperm %xmm2, %xmm3, %xmm0, %xmm2
; XOP-NEXT: vpperm %xmm1, %xmm3, %xmm0, %xmm0
; XOP-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-LABEL: var_shuffle_v32i8:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: # %bb.0:
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
[X86][AVX] createVariablePermute - use PSHUFB+PCMPGT+SELECT for v32i8 variable permutes Same as the VPERMILPS/VPERMILPD approach for v8f32/v4f64 cases, rely on PSHUFB using bits[3:0] for indexing - we can ignore the sign bit (zero element) as those index vector values are considered undefined. The select between the lo/hi permute results based on the index size. llvm-svn: 327242
2018-03-12 00:28:11 +08:00
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX1-NEXT: vpcmpgtb %xmm3, %xmm2, %xmm4
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm5
; AVX1-NEXT: vpshufb %xmm2, %xmm5, %xmm6
; AVX1-NEXT: vpshufb %xmm2, %xmm0, %xmm2
; AVX1-NEXT: vpblendvb %xmm4, %xmm6, %xmm2, %xmm2
; AVX1-NEXT: vpcmpgtb %xmm3, %xmm1, %xmm3
; AVX1-NEXT: vpshufb %xmm1, %xmm5, %xmm4
; AVX1-NEXT: vpshufb %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpblendvb %xmm3, %xmm4, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
; AVX1-NEXT: retq
;
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX2-LABEL: var_shuffle_v32i8:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: # %bb.0:
[X86][AVX] createVariablePermute - use PSHUFB+PCMPGT+SELECT for v32i8 variable permutes Same as the VPERMILPS/VPERMILPD approach for v8f32/v4f64 cases, rely on PSHUFB using bits[3:0] for indexing - we can ignore the sign bit (zero element) as those index vector values are considered undefined. The select between the lo/hi permute results based on the index size. llvm-svn: 327242
2018-03-12 00:28:11 +08:00
; AVX2-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm2
; AVX2-NEXT: vpshufb %ymm1, %ymm2, %ymm2
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm3
; AVX2-NEXT: vpblendd {{.*#+}} ymm0 = ymm3[0,1,2,3],ymm0[4,5,6,7]
; AVX2-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm0, %ymm2, %ymm0
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX2-NEXT: retq
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512-LABEL: var_shuffle_v32i8:
; AVX512: # %bb.0:
; AVX512-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm2
; AVX512-NEXT: vpshufb %ymm1, %ymm2, %ymm2
; AVX512-NEXT: vextracti128 $1, %ymm0, %xmm3
; AVX512-NEXT: vpblendd {{.*#+}} ymm0 = ymm3[0,1,2,3],ymm0[4,5,6,7]
; AVX512-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX512-NEXT: vpblendvb %ymm1, %ymm0, %ymm2, %ymm0
; AVX512-NEXT: retq
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512VLDQ-LABEL: var_shuffle_v32i8:
; AVX512VLDQ: # %bb.0:
; AVX512VLDQ-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm2
; AVX512VLDQ-NEXT: vpshufb %ymm1, %ymm2, %ymm2
; AVX512VLDQ-NEXT: vextracti128 $1, %ymm0, %xmm3
; AVX512VLDQ-NEXT: vpblendd {{.*#+}} ymm0 = ymm3[0,1,2,3],ymm0[4,5,6,7]
; AVX512VLDQ-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512VLDQ-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX512VLDQ-NEXT: vpblendvb %ymm1, %ymm0, %ymm2, %ymm0
; AVX512VLDQ-NEXT: retq
[X86][AVX512DQ] Add DQ var permute 256 tests as requested on D42487 llvm-svn: 323970
2018-02-01 22:44:50 +08:00
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512VLBW-LABEL: var_shuffle_v32i8:
; AVX512VLBW: # %bb.0:
; AVX512VLBW-NEXT: vextracti128 $1, %ymm0, %xmm2
; AVX512VLBW-NEXT: vpblendd {{.*#+}} ymm2 = ymm2[0,1,2,3],ymm0[4,5,6,7]
; AVX512VLBW-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm0
; AVX512VLBW-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512VLBW-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %k1
; AVX512VLBW-NEXT: vpshufb %ymm1, %ymm2, %ymm0 {%k1}
; AVX512VLBW-NEXT: retq
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; VLVBMI-LABEL: var_shuffle_v32i8:
; VLVBMI: # %bb.0:
; VLVBMI-NEXT: vpermb %ymm0, %ymm1, %ymm0
; VLVBMI-NEXT: retq
X86 Tests: Add tests for vector permutes with variable indices. NFC. Basic tests which are the equivalent of single-source shufflevector with variable mask. llvm-svn: 316216
2017-10-20 23:32:14 +08:00
%index0 = extractelement <32 x i8> %indices, i32 0
%index1 = extractelement <32 x i8> %indices, i32 1
%index2 = extractelement <32 x i8> %indices, i32 2
%index3 = extractelement <32 x i8> %indices, i32 3
%index4 = extractelement <32 x i8> %indices, i32 4
%index5 = extractelement <32 x i8> %indices, i32 5
%index6 = extractelement <32 x i8> %indices, i32 6
%index7 = extractelement <32 x i8> %indices, i32 7
%index8 = extractelement <32 x i8> %indices, i32 8
%index9 = extractelement <32 x i8> %indices, i32 9
%index10 = extractelement <32 x i8> %indices, i32 10
%index11 = extractelement <32 x i8> %indices, i32 11
%index12 = extractelement <32 x i8> %indices, i32 12
%index13 = extractelement <32 x i8> %indices, i32 13
%index14 = extractelement <32 x i8> %indices, i32 14
%index15 = extractelement <32 x i8> %indices, i32 15
%index16 = extractelement <32 x i8> %indices, i32 16
%index17 = extractelement <32 x i8> %indices, i32 17
%index18 = extractelement <32 x i8> %indices, i32 18
%index19 = extractelement <32 x i8> %indices, i32 19
%index20 = extractelement <32 x i8> %indices, i32 20
%index21 = extractelement <32 x i8> %indices, i32 21
%index22 = extractelement <32 x i8> %indices, i32 22
%index23 = extractelement <32 x i8> %indices, i32 23
%index24 = extractelement <32 x i8> %indices, i32 24
%index25 = extractelement <32 x i8> %indices, i32 25
%index26 = extractelement <32 x i8> %indices, i32 26
%index27 = extractelement <32 x i8> %indices, i32 27
%index28 = extractelement <32 x i8> %indices, i32 28
%index29 = extractelement <32 x i8> %indices, i32 29
%index30 = extractelement <32 x i8> %indices, i32 30
%index31 = extractelement <32 x i8> %indices, i32 31
%v0 = extractelement <32 x i8> %v, i8 %index0
%v1 = extractelement <32 x i8> %v, i8 %index1
%v2 = extractelement <32 x i8> %v, i8 %index2
%v3 = extractelement <32 x i8> %v, i8 %index3
%v4 = extractelement <32 x i8> %v, i8 %index4
%v5 = extractelement <32 x i8> %v, i8 %index5
%v6 = extractelement <32 x i8> %v, i8 %index6
%v7 = extractelement <32 x i8> %v, i8 %index7
%v8 = extractelement <32 x i8> %v, i8 %index8
%v9 = extractelement <32 x i8> %v, i8 %index9
%v10 = extractelement <32 x i8> %v, i8 %index10
%v11 = extractelement <32 x i8> %v, i8 %index11
%v12 = extractelement <32 x i8> %v, i8 %index12
%v13 = extractelement <32 x i8> %v, i8 %index13
%v14 = extractelement <32 x i8> %v, i8 %index14
%v15 = extractelement <32 x i8> %v, i8 %index15
%v16 = extractelement <32 x i8> %v, i8 %index16
%v17 = extractelement <32 x i8> %v, i8 %index17
%v18 = extractelement <32 x i8> %v, i8 %index18
%v19 = extractelement <32 x i8> %v, i8 %index19
%v20 = extractelement <32 x i8> %v, i8 %index20
%v21 = extractelement <32 x i8> %v, i8 %index21
%v22 = extractelement <32 x i8> %v, i8 %index22
%v23 = extractelement <32 x i8> %v, i8 %index23
%v24 = extractelement <32 x i8> %v, i8 %index24
%v25 = extractelement <32 x i8> %v, i8 %index25
%v26 = extractelement <32 x i8> %v, i8 %index26
%v27 = extractelement <32 x i8> %v, i8 %index27
%v28 = extractelement <32 x i8> %v, i8 %index28
%v29 = extractelement <32 x i8> %v, i8 %index29
%v30 = extractelement <32 x i8> %v, i8 %index30
%v31 = extractelement <32 x i8> %v, i8 %index31
%ret0 = insertelement <32 x i8> undef, i8 %v0, i32 0
%ret1 = insertelement <32 x i8> %ret0, i8 %v1, i32 1
%ret2 = insertelement <32 x i8> %ret1, i8 %v2, i32 2
%ret3 = insertelement <32 x i8> %ret2, i8 %v3, i32 3
%ret4 = insertelement <32 x i8> %ret3, i8 %v4, i32 4
%ret5 = insertelement <32 x i8> %ret4, i8 %v5, i32 5
%ret6 = insertelement <32 x i8> %ret5, i8 %v6, i32 6
%ret7 = insertelement <32 x i8> %ret6, i8 %v7, i32 7
%ret8 = insertelement <32 x i8> %ret7, i8 %v8, i32 8
%ret9 = insertelement <32 x i8> %ret8, i8 %v9, i32 9
%ret10 = insertelement <32 x i8> %ret9, i8 %v10, i32 10
%ret11 = insertelement <32 x i8> %ret10, i8 %v11, i32 11
%ret12 = insertelement <32 x i8> %ret11, i8 %v12, i32 12
%ret13 = insertelement <32 x i8> %ret12, i8 %v13, i32 13
%ret14 = insertelement <32 x i8> %ret13, i8 %v14, i32 14
%ret15 = insertelement <32 x i8> %ret14, i8 %v15, i32 15
%ret16 = insertelement <32 x i8> %ret15, i8 %v16, i32 16
%ret17 = insertelement <32 x i8> %ret16, i8 %v17, i32 17
%ret18 = insertelement <32 x i8> %ret17, i8 %v18, i32 18
%ret19 = insertelement <32 x i8> %ret18, i8 %v19, i32 19
%ret20 = insertelement <32 x i8> %ret19, i8 %v20, i32 20
%ret21 = insertelement <32 x i8> %ret20, i8 %v21, i32 21
%ret22 = insertelement <32 x i8> %ret21, i8 %v22, i32 22
%ret23 = insertelement <32 x i8> %ret22, i8 %v23, i32 23
%ret24 = insertelement <32 x i8> %ret23, i8 %v24, i32 24
%ret25 = insertelement <32 x i8> %ret24, i8 %v25, i32 25
%ret26 = insertelement <32 x i8> %ret25, i8 %v26, i32 26
%ret27 = insertelement <32 x i8> %ret26, i8 %v27, i32 27
%ret28 = insertelement <32 x i8> %ret27, i8 %v28, i32 28
%ret29 = insertelement <32 x i8> %ret28, i8 %v29, i32 29
%ret30 = insertelement <32 x i8> %ret29, i8 %v30, i32 30
%ret31 = insertelement <32 x i8> %ret30, i8 %v31, i32 31
ret <32 x i8> %ret31
}
X86 Tests: Update the variable-index permute tests with FP types. NFC. These cases will be addressed in a future update to D39126. llvm-svn: 316946
2017-10-31 03:29:15 +08:00
define <4 x double> @var_shuffle_v4f64(<4 x double> %v, <4 x i64> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v4f64:
; XOP: # %bb.0:
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpaddq %xmm1, %xmm1, %xmm3
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: vextractf128 $1, %ymm1, %xmm1
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: vpaddq %xmm1, %xmm1, %xmm1
; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; XOP-NEXT: vpermil2pd $0, %ymm1, %ymm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
X86 Tests: Update the variable-index permute tests with FP types. NFC. These cases will be addressed in a future update to D39126. llvm-svn: 316946
2017-10-31 03:29:15 +08:00
; AVX1-LABEL: var_shuffle_v4f64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: # %bb.0:
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpaddq %xmm1, %xmm1, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vpaddq %xmm1, %xmm1, %xmm1
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm4
; AVX1-NEXT: vpermilpd %ymm4, %ymm2, %ymm2
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpermilpd %ymm4, %ymm0, %ymm0
; AVX1-NEXT: vpcmpgtq {{.*}}(%rip), %xmm3, %xmm3
; AVX1-NEXT: vpcmpgtq {{\.LCPI.*}}+{{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; AVX1-NEXT: vblendvpd %ymm1, %ymm2, %ymm0, %ymm0
X86 Tests: Update the variable-index permute tests with FP types. NFC. These cases will be addressed in a future update to D39126. llvm-svn: 316946
2017-10-31 03:29:15 +08:00
; AVX1-NEXT: retq
;
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX2-LABEL: var_shuffle_v4f64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: # %bb.0:
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX2-NEXT: vpaddq %ymm1, %ymm1, %ymm1
[X86][SSE] LowerBUILD_VECTORAsVariablePermute - add support for scaling index vectors This allows us to use PSHUFB for v8i16/v4i32 and VPERMD/PERMPS for v4i64/v4f64 variable shuffles. Differential Revision: https://reviews.llvm.org/D42487 llvm-svn: 323987
2018-02-02 02:10:30 +08:00
; AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [2,2,2,2]
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX2-NEXT: vpcmpgtq %ymm2, %ymm1, %ymm2
; AVX2-NEXT: vpermpd {{.*#+}} ymm3 = ymm0[2,3,2,3]
; AVX2-NEXT: vpermilpd %ymm1, %ymm3, %ymm3
; AVX2-NEXT: vpermpd {{.*#+}} ymm0 = ymm0[0,1,0,1]
; AVX2-NEXT: vpermilpd %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vblendvpd %ymm2, %ymm3, %ymm0, %ymm0
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX2-NEXT: retq
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512-LABEL: var_shuffle_v4f64:
; AVX512: # %bb.0:
; AVX512-NEXT: # kill: def $ymm1 killed $ymm1 def $zmm1
; AVX512-NEXT: # kill: def $ymm0 killed $ymm0 def $zmm0
; AVX512-NEXT: vpermpd %zmm0, %zmm1, %zmm0
; AVX512-NEXT: # kill: def $ymm0 killed $ymm0 killed $zmm0
; AVX512-NEXT: retq
[X86][AVX512DQ] Add DQ var permute 256 tests as requested on D42487 llvm-svn: 323970
2018-02-01 22:44:50 +08:00
;
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX512VL-LABEL: var_shuffle_v4f64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512VL: # %bb.0:
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; AVX512VL-NEXT: vpermpd %ymm0, %ymm1, %ymm0
; AVX512VL-NEXT: retq
X86 Tests: Update the variable-index permute tests with FP types. NFC. These cases will be addressed in a future update to D39126. llvm-svn: 316946
2017-10-31 03:29:15 +08:00
%index0 = extractelement <4 x i64> %indices, i32 0
%index1 = extractelement <4 x i64> %indices, i32 1
%index2 = extractelement <4 x i64> %indices, i32 2
%index3 = extractelement <4 x i64> %indices, i32 3
%v0 = extractelement <4 x double> %v, i64 %index0
%v1 = extractelement <4 x double> %v, i64 %index1
%v2 = extractelement <4 x double> %v, i64 %index2
%v3 = extractelement <4 x double> %v, i64 %index3
%ret0 = insertelement <4 x double> undef, double %v0, i32 0
%ret1 = insertelement <4 x double> %ret0, double %v1, i32 1
%ret2 = insertelement <4 x double> %ret1, double %v2, i32 2
%ret3 = insertelement <4 x double> %ret2, double %v3, i32 3
ret <4 x double> %ret3
}
define <8 x float> @var_shuffle_v8f32(<8 x float> %v, <8 x i32> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v8f32:
; XOP: # %bb.0:
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpermil2ps $0, %ymm1, %ymm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
X86 Tests: Update the variable-index permute tests with FP types. NFC. These cases will be addressed in a future update to D39126. llvm-svn: 316946
2017-10-31 03:29:15 +08:00
; AVX1-LABEL: var_shuffle_v8f32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: # %bb.0:
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpermilps %ymm1, %ymm2, %ymm2
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; AVX1-NEXT: vpermilps %ymm1, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpcmpgtd {{.*}}(%rip), %xmm1, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
; AVX1-NEXT: vpcmpgtd {{\.LCPI.*}}+{{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; AVX1-NEXT: vblendvps %ymm1, %ymm2, %ymm0, %ymm0
X86 Tests: Update the variable-index permute tests with FP types. NFC. These cases will be addressed in a future update to D39126. llvm-svn: 316946
2017-10-31 03:29:15 +08:00
; AVX1-NEXT: retq
;
; INT256-LABEL: var_shuffle_v8f32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; INT256: # %bb.0:
X86 ISel: Basic support for variable-index vector permutations Summary: Try to lower a BUILD_VECTOR composed of extract-extract chains that can be reasoned to be a permutation of a vector by indices in a non-constant vector. We saw this pattern created by ISPC, which resolts to creating it due to the requirement that shufflevector's mask operand be a *constant* vector. I didn't check this but we could possibly use this pattern for lowering the X86 permute C-instrinsics instead of llvm.x86 instrinsics. This change can be followed by more improvements: 1. Handle vectors with undef elements. 2. Utilize pshufb and zero-mask-blending to support more effiecient construction of vectors with constant-0 elements. 3. Use smaller-element vectors of same width, and "interpolate" the indices, when no native operation available. Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: chandlerc, DavidKreitzer Differential Revision: https://reviews.llvm.org/D39126 llvm-svn: 317463
2017-11-06 16:25:46 +08:00
; INT256-NEXT: vpermps %ymm0, %ymm1, %ymm0
X86 Tests: Update the variable-index permute tests with FP types. NFC. These cases will be addressed in a future update to D39126. llvm-svn: 316946
2017-10-31 03:29:15 +08:00
; INT256-NEXT: retq
%index0 = extractelement <8 x i32> %indices, i32 0
%index1 = extractelement <8 x i32> %indices, i32 1
%index2 = extractelement <8 x i32> %indices, i32 2
%index3 = extractelement <8 x i32> %indices, i32 3
%index4 = extractelement <8 x i32> %indices, i32 4
%index5 = extractelement <8 x i32> %indices, i32 5
%index6 = extractelement <8 x i32> %indices, i32 6
%index7 = extractelement <8 x i32> %indices, i32 7
%v0 = extractelement <8 x float> %v, i32 %index0
%v1 = extractelement <8 x float> %v, i32 %index1
%v2 = extractelement <8 x float> %v, i32 %index2
%v3 = extractelement <8 x float> %v, i32 %index3
%v4 = extractelement <8 x float> %v, i32 %index4
%v5 = extractelement <8 x float> %v, i32 %index5
%v6 = extractelement <8 x float> %v, i32 %index6
%v7 = extractelement <8 x float> %v, i32 %index7
%ret0 = insertelement <8 x float> undef, float %v0, i32 0
%ret1 = insertelement <8 x float> %ret0, float %v1, i32 1
%ret2 = insertelement <8 x float> %ret1, float %v2, i32 2
%ret3 = insertelement <8 x float> %ret2, float %v3, i32 3
%ret4 = insertelement <8 x float> %ret3, float %v4, i32 4
%ret5 = insertelement <8 x float> %ret4, float %v5, i32 5
%ret6 = insertelement <8 x float> %ret5, float %v6, i32 6
%ret7 = insertelement <8 x float> %ret6, float %v7, i32 7
ret <8 x float> %ret7
}
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
;
; PR35820 - Unequal source/destination vector sizes
;
define <4 x i64> @var_shuffle_v4i64_from_v2i64(<2 x i64> %v, <4 x i64> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v4i64_from_v2i64:
; XOP: # %bb.0:
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpaddq %xmm1, %xmm1, %xmm3
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: vextractf128 $1, %ymm1, %xmm1
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: vpaddq %xmm1, %xmm1, %xmm1
; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; XOP-NEXT: vpermil2pd $0, %ymm1, %ymm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-LABEL: var_shuffle_v4i64_from_v2i64:
; AVX1: # %bb.0:
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpaddq %xmm1, %xmm1, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vpaddq %xmm1, %xmm1, %xmm1
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm4
; AVX1-NEXT: vpermilpd %ymm4, %ymm2, %ymm2
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpermilpd %ymm4, %ymm0, %ymm0
; AVX1-NEXT: vpcmpgtq {{.*}}(%rip), %xmm3, %xmm3
; AVX1-NEXT: vpcmpgtq {{\.LCPI.*}}+{{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; AVX1-NEXT: vblendvpd %ymm1, %ymm2, %ymm0, %ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: var_shuffle_v4i64_from_v2i64:
; AVX2: # %bb.0:
[X86][SSE] LowerBUILD_VECTORAsVariablePermute - add support for scaling index vectors This allows us to use PSHUFB for v8i16/v4i32 and VPERMD/PERMPS for v4i64/v4f64 variable shuffles. Differential Revision: https://reviews.llvm.org/D42487 llvm-svn: 323987
2018-02-02 02:10:30 +08:00
; AVX2-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX2-NEXT: vpaddq %ymm1, %ymm1, %ymm1
[X86][SSE] LowerBUILD_VECTORAsVariablePermute - add support for scaling index vectors This allows us to use PSHUFB for v8i16/v4i32 and VPERMD/PERMPS for v4i64/v4f64 variable shuffles. Differential Revision: https://reviews.llvm.org/D42487 llvm-svn: 323987
2018-02-02 02:10:30 +08:00
; AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [2,2,2,2]
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX2-NEXT: vpcmpgtq %ymm2, %ymm1, %ymm2
; AVX2-NEXT: vpermpd {{.*#+}} ymm3 = ymm0[2,3,2,3]
; AVX2-NEXT: vpermilpd %ymm1, %ymm3, %ymm3
; AVX2-NEXT: vpermpd {{.*#+}} ymm0 = ymm0[0,1,0,1]
; AVX2-NEXT: vpermilpd %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vblendvpd %ymm2, %ymm3, %ymm0, %ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX2-NEXT: retq
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512-LABEL: var_shuffle_v4i64_from_v2i64:
; AVX512: # %bb.0:
; AVX512-NEXT: # kill: def $ymm1 killed $ymm1 def $zmm1
; AVX512-NEXT: # kill: def $xmm0 killed $xmm0 def $zmm0
; AVX512-NEXT: vpermpd %zmm0, %zmm1, %zmm0
; AVX512-NEXT: # kill: def $ymm0 killed $ymm0 killed $zmm0
; AVX512-NEXT: retq
[X86][AVX512DQ] Add DQ var permute 256 tests as requested on D42487 llvm-svn: 323970
2018-02-01 22:44:50 +08:00
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX512VL-LABEL: var_shuffle_v4i64_from_v2i64:
; AVX512VL: # %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512VL-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX512VL-NEXT: vpermpd %ymm0, %ymm1, %ymm0
; AVX512VL-NEXT: retq
%index0 = extractelement <4 x i64> %indices, i32 0
%index1 = extractelement <4 x i64> %indices, i32 1
%index2 = extractelement <4 x i64> %indices, i32 2
%index3 = extractelement <4 x i64> %indices, i32 3
%v0 = extractelement <2 x i64> %v, i64 %index0
%v1 = extractelement <2 x i64> %v, i64 %index1
%v2 = extractelement <2 x i64> %v, i64 %index2
%v3 = extractelement <2 x i64> %v, i64 %index3
%ret0 = insertelement <4 x i64> undef, i64 %v0, i32 0
%ret1 = insertelement <4 x i64> %ret0, i64 %v1, i32 1
%ret2 = insertelement <4 x i64> %ret1, i64 %v2, i32 2
%ret3 = insertelement <4 x i64> %ret2, i64 %v3, i32 3
ret <4 x i64> %ret3
}
define <8 x i32> @var_shuffle_v8i32_from_v4i32(<4 x i32> %v, <8 x i32> %indices) unnamed_addr nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v8i32_from_v4i32:
; XOP: # %bb.0: # %entry
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpermil2ps $0, %ymm1, %ymm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-LABEL: var_shuffle_v8i32_from_v4i32:
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
; AVX1: # %bb.0: # %entry
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpermilps %ymm1, %ymm2, %ymm2
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; AVX1-NEXT: vpermilps %ymm1, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpcmpgtd {{.*}}(%rip), %xmm1, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
; AVX1-NEXT: vpcmpgtd {{\.LCPI.*}}+{{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; AVX1-NEXT: vblendvps %ymm1, %ymm2, %ymm0, %ymm0
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
; AVX1-NEXT: retq
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; INT256-LABEL: var_shuffle_v8i32_from_v4i32:
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
; INT256: # %bb.0: # %entry
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; INT256-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
; INT256-NEXT: vpermps %ymm0, %ymm1, %ymm0
; INT256-NEXT: retq
entry:
%tmp1 = extractelement <8 x i32> %indices, i32 0
%vecext2.8 = extractelement <4 x i32> %v, i32 %tmp1
%tmp2 = extractelement <8 x i32> %indices, i32 1
%vecext2.9 = extractelement <4 x i32> %v, i32 %tmp2
%tmp3 = extractelement <8 x i32> %indices, i32 2
%vecext2.10 = extractelement <4 x i32> %v, i32 %tmp3
%tmp4 = extractelement <8 x i32> %indices, i32 3
%vecext2.11 = extractelement <4 x i32> %v, i32 %tmp4
%tmp5 = extractelement <8 x i32> %indices, i32 4
%vecext2.12 = extractelement <4 x i32> %v, i32 %tmp5
%tmp6 = extractelement <8 x i32> %indices, i32 5
%vecext2.13 = extractelement <4 x i32> %v, i32 %tmp6
%tmp7 = extractelement <8 x i32> %indices, i32 6
%vecext2.14 = extractelement <4 x i32> %v, i32 %tmp7
%tmp8 = extractelement <8 x i32> %indices, i32 7
%vecext2.15 = extractelement <4 x i32> %v, i32 %tmp8
%tmp9 = insertelement <8 x i32> undef, i32 %vecext2.8, i32 0
%tmp10 = insertelement <8 x i32> %tmp9, i32 %vecext2.9, i32 1
%tmp11 = insertelement <8 x i32> %tmp10, i32 %vecext2.10, i32 2
%tmp12 = insertelement <8 x i32> %tmp11, i32 %vecext2.11, i32 3
%tmp13 = insertelement <8 x i32> %tmp12, i32 %vecext2.12, i32 4
%tmp14 = insertelement <8 x i32> %tmp13, i32 %vecext2.13, i32 5
%tmp15 = insertelement <8 x i32> %tmp14, i32 %vecext2.14, i32 6
%tmp16 = insertelement <8 x i32> %tmp15, i32 %vecext2.15, i32 7
ret <8 x i32> %tmp16
}
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
define <16 x i16> @var_shuffle_v16i16_from_v8i16(<8 x i16> %v, <16 x i16> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v16i16_from_v8i16:
; XOP: # %bb.0:
[X86][XOP] createVariablePermute - use VPPERM for v16i16 variable permutes llvm-svn: 327218
2018-03-11 02:33:29 +08:00
; XOP-NEXT: vmovdqa {{.*#+}} xmm2 = [256,256,256,256,256,256,256,256]
; XOP-NEXT: vmovdqa {{.*#+}} xmm3 = [514,514,514,514,514,514,514,514]
; XOP-NEXT: vpmacsww %xmm2, %xmm3, %xmm1, %xmm4
; XOP-NEXT: vextractf128 $1, %ymm1, %xmm1
; XOP-NEXT: vpmacsww %xmm2, %xmm3, %xmm1, %xmm1
; XOP-NEXT: vpperm %xmm1, %xmm0, %xmm0, %xmm1
; XOP-NEXT: vpperm %xmm4, %xmm0, %xmm0, %xmm0
; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-LABEL: var_shuffle_v16i16_from_v8i16:
; AVX1: # %bb.0:
[X86][AVX] createVariablePermute - scale v16i16 variable permutes to use v32i8 codegen XOP was already doing this, and now AVX performs v32i8 variable permutes as well. llvm-svn: 327245
2018-03-12 01:23:54 +08:00
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [514,514,514,514,514,514,514,514]
; AVX1-NEXT: vpmullw %xmm2, %xmm1, %xmm3
; AVX1-NEXT: vmovdqa {{.*#+}} xmm4 = [256,256,256,256,256,256,256,256]
; AVX1-NEXT: vpaddw %xmm4, %xmm3, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
; AVX1-NEXT: vpmullw %xmm2, %xmm1, %xmm1
; AVX1-NEXT: vpaddw %xmm4, %xmm1, %xmm1
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX1-NEXT: vpcmpgtb %xmm2, %xmm1, %xmm4
; AVX1-NEXT: vpshufb %xmm1, %xmm0, %xmm5
; AVX1-NEXT: vpshufb %xmm1, %xmm0, %xmm1
; AVX1-NEXT: vpblendvb %xmm4, %xmm5, %xmm1, %xmm1
; AVX1-NEXT: vpcmpgtb %xmm2, %xmm3, %xmm2
; AVX1-NEXT: vpshufb %xmm3, %xmm0, %xmm4
; AVX1-NEXT: vpshufb %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpblendvb %xmm2, %xmm4, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: var_shuffle_v16i16_from_v8i16:
; AVX2: # %bb.0:
[X86][AVX] createVariablePermute - scale v16i16 variable permutes to use v32i8 codegen XOP was already doing this, and now AVX performs v32i8 variable permutes as well. llvm-svn: 327245
2018-03-12 01:23:54 +08:00
; AVX2-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; AVX2-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX2-NEXT: vpaddw {{.*}}(%rip), %ymm1, %ymm1
; AVX2-NEXT: vpshufb %ymm1, %ymm0, %ymm2
; AVX2-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm0
; AVX2-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm2, %ymm0, %ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX2-NEXT: retq
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512-LABEL: var_shuffle_v16i16_from_v8i16:
; AVX512: # %bb.0:
; AVX512-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; AVX512-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX512-NEXT: vpaddw {{.*}}(%rip), %ymm1, %ymm1
; AVX512-NEXT: vpshufb %ymm1, %ymm0, %ymm2
; AVX512-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm0
; AVX512-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX512-NEXT: vpblendvb %ymm1, %ymm2, %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512VLDQ-LABEL: var_shuffle_v16i16_from_v8i16:
; AVX512VLDQ: # %bb.0:
; AVX512VLDQ-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; AVX512VLDQ-NEXT: vpmullw {{.*}}(%rip), %ymm1, %ymm1
; AVX512VLDQ-NEXT: vpaddw {{.*}}(%rip), %ymm1, %ymm1
; AVX512VLDQ-NEXT: vpshufb %ymm1, %ymm0, %ymm2
; AVX512VLDQ-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm0
; AVX512VLDQ-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512VLDQ-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX512VLDQ-NEXT: vpblendvb %ymm1, %ymm2, %ymm0, %ymm0
; AVX512VLDQ-NEXT: retq
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
;
; AVX512VLBW-LABEL: var_shuffle_v16i16_from_v8i16:
; AVX512VLBW: # %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512VLBW-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX512VLBW-NEXT: vpermw %ymm0, %ymm1, %ymm0
; AVX512VLBW-NEXT: retq
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
;
; VLVBMI-LABEL: var_shuffle_v16i16_from_v8i16:
; VLVBMI: # %bb.0:
; VLVBMI-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; VLVBMI-NEXT: vpermw %ymm0, %ymm1, %ymm0
; VLVBMI-NEXT: retq
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
%index0 = extractelement <16 x i16> %indices, i32 0
%index1 = extractelement <16 x i16> %indices, i32 1
%index2 = extractelement <16 x i16> %indices, i32 2
%index3 = extractelement <16 x i16> %indices, i32 3
%index4 = extractelement <16 x i16> %indices, i32 4
%index5 = extractelement <16 x i16> %indices, i32 5
%index6 = extractelement <16 x i16> %indices, i32 6
%index7 = extractelement <16 x i16> %indices, i32 7
%index8 = extractelement <16 x i16> %indices, i32 8
%index9 = extractelement <16 x i16> %indices, i32 9
%index10 = extractelement <16 x i16> %indices, i32 10
%index11 = extractelement <16 x i16> %indices, i32 11
%index12 = extractelement <16 x i16> %indices, i32 12
%index13 = extractelement <16 x i16> %indices, i32 13
%index14 = extractelement <16 x i16> %indices, i32 14
%index15 = extractelement <16 x i16> %indices, i32 15
%v0 = extractelement <8 x i16> %v, i16 %index0
%v1 = extractelement <8 x i16> %v, i16 %index1
%v2 = extractelement <8 x i16> %v, i16 %index2
%v3 = extractelement <8 x i16> %v, i16 %index3
%v4 = extractelement <8 x i16> %v, i16 %index4
%v5 = extractelement <8 x i16> %v, i16 %index5
%v6 = extractelement <8 x i16> %v, i16 %index6
%v7 = extractelement <8 x i16> %v, i16 %index7
%v8 = extractelement <8 x i16> %v, i16 %index8
%v9 = extractelement <8 x i16> %v, i16 %index9
%v10 = extractelement <8 x i16> %v, i16 %index10
%v11 = extractelement <8 x i16> %v, i16 %index11
%v12 = extractelement <8 x i16> %v, i16 %index12
%v13 = extractelement <8 x i16> %v, i16 %index13
%v14 = extractelement <8 x i16> %v, i16 %index14
%v15 = extractelement <8 x i16> %v, i16 %index15
%ret0 = insertelement <16 x i16> undef, i16 %v0, i32 0
%ret1 = insertelement <16 x i16> %ret0, i16 %v1, i32 1
%ret2 = insertelement <16 x i16> %ret1, i16 %v2, i32 2
%ret3 = insertelement <16 x i16> %ret2, i16 %v3, i32 3
%ret4 = insertelement <16 x i16> %ret3, i16 %v4, i32 4
%ret5 = insertelement <16 x i16> %ret4, i16 %v5, i32 5
%ret6 = insertelement <16 x i16> %ret5, i16 %v6, i32 6
%ret7 = insertelement <16 x i16> %ret6, i16 %v7, i32 7
%ret8 = insertelement <16 x i16> %ret7, i16 %v8, i32 8
%ret9 = insertelement <16 x i16> %ret8, i16 %v9, i32 9
%ret10 = insertelement <16 x i16> %ret9, i16 %v10, i32 10
%ret11 = insertelement <16 x i16> %ret10, i16 %v11, i32 11
%ret12 = insertelement <16 x i16> %ret11, i16 %v12, i32 12
%ret13 = insertelement <16 x i16> %ret12, i16 %v13, i32 13
%ret14 = insertelement <16 x i16> %ret13, i16 %v14, i32 14
%ret15 = insertelement <16 x i16> %ret14, i16 %v15, i32 15
ret <16 x i16> %ret15
}
define <32 x i8> @var_shuffle_v32i8_from_v16i8(<16 x i8> %v, <32 x i8> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v32i8_from_v16i8:
; XOP: # %bb.0:
; XOP-NEXT: vextractf128 $1, %ymm1, %xmm2
[X86][XOP] createVariablePermute - use VPPERM for v32i8 variable permutes llvm-svn: 327213
2018-03-11 00:51:45 +08:00
; XOP-NEXT: vpperm %xmm2, %xmm0, %xmm0, %xmm2
; XOP-NEXT: vpperm %xmm1, %xmm0, %xmm0, %xmm0
; XOP-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-LABEL: var_shuffle_v32i8_from_v16i8:
; AVX1: # %bb.0:
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm2
[X86][AVX] createVariablePermute - use PSHUFB+PCMPGT+SELECT for v32i8 variable permutes Same as the VPERMILPS/VPERMILPD approach for v8f32/v4f64 cases, rely on PSHUFB using bits[3:0] for indexing - we can ignore the sign bit (zero element) as those index vector values are considered undefined. The select between the lo/hi permute results based on the index size. llvm-svn: 327242
2018-03-12 00:28:11 +08:00
; AVX1-NEXT: vmovdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX1-NEXT: vpcmpgtb %xmm3, %xmm2, %xmm4
; AVX1-NEXT: vpshufb %xmm2, %xmm0, %xmm5
; AVX1-NEXT: vpshufb %xmm2, %xmm0, %xmm2
; AVX1-NEXT: vpblendvb %xmm4, %xmm5, %xmm2, %xmm2
; AVX1-NEXT: vpcmpgtb %xmm3, %xmm1, %xmm3
; AVX1-NEXT: vpshufb %xmm1, %xmm0, %xmm4
; AVX1-NEXT: vpshufb %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpblendvb %xmm3, %xmm4, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: var_shuffle_v32i8_from_v16i8:
; AVX2: # %bb.0:
[X86][AVX] createVariablePermute - use PSHUFB+PCMPGT+SELECT for v32i8 variable permutes Same as the VPERMILPS/VPERMILPD approach for v8f32/v4f64 cases, rely on PSHUFB using bits[3:0] for indexing - we can ignore the sign bit (zero element) as those index vector values are considered undefined. The select between the lo/hi permute results based on the index size. llvm-svn: 327242
2018-03-12 00:28:11 +08:00
; AVX2-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; AVX2-NEXT: vpshufb %ymm1, %ymm0, %ymm2
; AVX2-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm0
; AVX2-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX2-NEXT: vpblendvb %ymm1, %ymm2, %ymm0, %ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX2-NEXT: retq
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512-LABEL: var_shuffle_v32i8_from_v16i8:
; AVX512: # %bb.0:
; AVX512-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; AVX512-NEXT: vpshufb %ymm1, %ymm0, %ymm2
; AVX512-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm0
; AVX512-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX512-NEXT: vpblendvb %ymm1, %ymm2, %ymm0, %ymm0
; AVX512-NEXT: retq
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512VLDQ-LABEL: var_shuffle_v32i8_from_v16i8:
; AVX512VLDQ: # %bb.0:
; AVX512VLDQ-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; AVX512VLDQ-NEXT: vpshufb %ymm1, %ymm0, %ymm2
; AVX512VLDQ-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm0
; AVX512VLDQ-NEXT: vpshufb %ymm1, %ymm0, %ymm0
; AVX512VLDQ-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %ymm1
; AVX512VLDQ-NEXT: vpblendvb %ymm1, %ymm2, %ymm0, %ymm0
; AVX512VLDQ-NEXT: retq
[X86][AVX512DQ] Add DQ var permute 256 tests as requested on D42487 llvm-svn: 323970
2018-02-01 22:44:50 +08:00
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512VLBW-LABEL: var_shuffle_v32i8_from_v16i8:
; AVX512VLBW: # %bb.0:
; AVX512VLBW-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; AVX512VLBW-NEXT: vinserti128 $1, %xmm0, %ymm0, %ymm2
; AVX512VLBW-NEXT: vpshufb %ymm1, %ymm2, %ymm2
; AVX512VLBW-NEXT: vpcmpgtb {{.*}}(%rip), %ymm1, %k1
; AVX512VLBW-NEXT: vpshufb %ymm1, %ymm0, %ymm2 {%k1}
; AVX512VLBW-NEXT: vmovdqa %ymm2, %ymm0
; AVX512VLBW-NEXT: retq
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; VLVBMI-LABEL: var_shuffle_v32i8_from_v16i8:
; VLVBMI: # %bb.0:
; VLVBMI-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; VLVBMI-NEXT: vpermb %ymm0, %ymm1, %ymm0
; VLVBMI-NEXT: retq
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
%index0 = extractelement <32 x i8> %indices, i32 0
%index1 = extractelement <32 x i8> %indices, i32 1
%index2 = extractelement <32 x i8> %indices, i32 2
%index3 = extractelement <32 x i8> %indices, i32 3
%index4 = extractelement <32 x i8> %indices, i32 4
%index5 = extractelement <32 x i8> %indices, i32 5
%index6 = extractelement <32 x i8> %indices, i32 6
%index7 = extractelement <32 x i8> %indices, i32 7
%index8 = extractelement <32 x i8> %indices, i32 8
%index9 = extractelement <32 x i8> %indices, i32 9
%index10 = extractelement <32 x i8> %indices, i32 10
%index11 = extractelement <32 x i8> %indices, i32 11
%index12 = extractelement <32 x i8> %indices, i32 12
%index13 = extractelement <32 x i8> %indices, i32 13
%index14 = extractelement <32 x i8> %indices, i32 14
%index15 = extractelement <32 x i8> %indices, i32 15
%index16 = extractelement <32 x i8> %indices, i32 16
%index17 = extractelement <32 x i8> %indices, i32 17
%index18 = extractelement <32 x i8> %indices, i32 18
%index19 = extractelement <32 x i8> %indices, i32 19
%index20 = extractelement <32 x i8> %indices, i32 20
%index21 = extractelement <32 x i8> %indices, i32 21
%index22 = extractelement <32 x i8> %indices, i32 22
%index23 = extractelement <32 x i8> %indices, i32 23
%index24 = extractelement <32 x i8> %indices, i32 24
%index25 = extractelement <32 x i8> %indices, i32 25
%index26 = extractelement <32 x i8> %indices, i32 26
%index27 = extractelement <32 x i8> %indices, i32 27
%index28 = extractelement <32 x i8> %indices, i32 28
%index29 = extractelement <32 x i8> %indices, i32 29
%index30 = extractelement <32 x i8> %indices, i32 30
%index31 = extractelement <32 x i8> %indices, i32 31
%v0 = extractelement <16 x i8> %v, i8 %index0
%v1 = extractelement <16 x i8> %v, i8 %index1
%v2 = extractelement <16 x i8> %v, i8 %index2
%v3 = extractelement <16 x i8> %v, i8 %index3
%v4 = extractelement <16 x i8> %v, i8 %index4
%v5 = extractelement <16 x i8> %v, i8 %index5
%v6 = extractelement <16 x i8> %v, i8 %index6
%v7 = extractelement <16 x i8> %v, i8 %index7
%v8 = extractelement <16 x i8> %v, i8 %index8
%v9 = extractelement <16 x i8> %v, i8 %index9
%v10 = extractelement <16 x i8> %v, i8 %index10
%v11 = extractelement <16 x i8> %v, i8 %index11
%v12 = extractelement <16 x i8> %v, i8 %index12
%v13 = extractelement <16 x i8> %v, i8 %index13
%v14 = extractelement <16 x i8> %v, i8 %index14
%v15 = extractelement <16 x i8> %v, i8 %index15
%v16 = extractelement <16 x i8> %v, i8 %index16
%v17 = extractelement <16 x i8> %v, i8 %index17
%v18 = extractelement <16 x i8> %v, i8 %index18
%v19 = extractelement <16 x i8> %v, i8 %index19
%v20 = extractelement <16 x i8> %v, i8 %index20
%v21 = extractelement <16 x i8> %v, i8 %index21
%v22 = extractelement <16 x i8> %v, i8 %index22
%v23 = extractelement <16 x i8> %v, i8 %index23
%v24 = extractelement <16 x i8> %v, i8 %index24
%v25 = extractelement <16 x i8> %v, i8 %index25
%v26 = extractelement <16 x i8> %v, i8 %index26
%v27 = extractelement <16 x i8> %v, i8 %index27
%v28 = extractelement <16 x i8> %v, i8 %index28
%v29 = extractelement <16 x i8> %v, i8 %index29
%v30 = extractelement <16 x i8> %v, i8 %index30
%v31 = extractelement <16 x i8> %v, i8 %index31
%ret0 = insertelement <32 x i8> undef, i8 %v0, i32 0
%ret1 = insertelement <32 x i8> %ret0, i8 %v1, i32 1
%ret2 = insertelement <32 x i8> %ret1, i8 %v2, i32 2
%ret3 = insertelement <32 x i8> %ret2, i8 %v3, i32 3
%ret4 = insertelement <32 x i8> %ret3, i8 %v4, i32 4
%ret5 = insertelement <32 x i8> %ret4, i8 %v5, i32 5
%ret6 = insertelement <32 x i8> %ret5, i8 %v6, i32 6
%ret7 = insertelement <32 x i8> %ret6, i8 %v7, i32 7
%ret8 = insertelement <32 x i8> %ret7, i8 %v8, i32 8
%ret9 = insertelement <32 x i8> %ret8, i8 %v9, i32 9
%ret10 = insertelement <32 x i8> %ret9, i8 %v10, i32 10
%ret11 = insertelement <32 x i8> %ret10, i8 %v11, i32 11
%ret12 = insertelement <32 x i8> %ret11, i8 %v12, i32 12
%ret13 = insertelement <32 x i8> %ret12, i8 %v13, i32 13
%ret14 = insertelement <32 x i8> %ret13, i8 %v14, i32 14
%ret15 = insertelement <32 x i8> %ret14, i8 %v15, i32 15
%ret16 = insertelement <32 x i8> %ret15, i8 %v16, i32 16
%ret17 = insertelement <32 x i8> %ret16, i8 %v17, i32 17
%ret18 = insertelement <32 x i8> %ret17, i8 %v18, i32 18
%ret19 = insertelement <32 x i8> %ret18, i8 %v19, i32 19
%ret20 = insertelement <32 x i8> %ret19, i8 %v20, i32 20
%ret21 = insertelement <32 x i8> %ret20, i8 %v21, i32 21
%ret22 = insertelement <32 x i8> %ret21, i8 %v22, i32 22
%ret23 = insertelement <32 x i8> %ret22, i8 %v23, i32 23
%ret24 = insertelement <32 x i8> %ret23, i8 %v24, i32 24
%ret25 = insertelement <32 x i8> %ret24, i8 %v25, i32 25
%ret26 = insertelement <32 x i8> %ret25, i8 %v26, i32 26
%ret27 = insertelement <32 x i8> %ret26, i8 %v27, i32 27
%ret28 = insertelement <32 x i8> %ret27, i8 %v28, i32 28
%ret29 = insertelement <32 x i8> %ret28, i8 %v29, i32 29
%ret30 = insertelement <32 x i8> %ret29, i8 %v30, i32 30
%ret31 = insertelement <32 x i8> %ret30, i8 %v31, i32 31
ret <32 x i8> %ret31
}
define <4 x double> @var_shuffle_v4f64_from_v2f64(<2 x double> %v, <4 x i64> %indices) nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v4f64_from_v2f64:
; XOP: # %bb.0:
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpaddq %xmm1, %xmm1, %xmm3
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: vextractf128 $1, %ymm1, %xmm1
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: vpaddq %xmm1, %xmm1, %xmm1
; XOP-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; XOP-NEXT: vpermil2pd $0, %ymm1, %ymm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-LABEL: var_shuffle_v4f64_from_v2f64:
; AVX1: # %bb.0:
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpaddq %xmm1, %xmm1, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vpaddq %xmm1, %xmm1, %xmm1
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm4
; AVX1-NEXT: vpermilpd %ymm4, %ymm2, %ymm2
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpermilpd %ymm4, %ymm0, %ymm0
; AVX1-NEXT: vpcmpgtq {{.*}}(%rip), %xmm3, %xmm3
; AVX1-NEXT: vpcmpgtq {{\.LCPI.*}}+{{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; AVX1-NEXT: vblendvpd %ymm1, %ymm2, %ymm0, %ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: var_shuffle_v4f64_from_v2f64:
; AVX2: # %bb.0:
[X86][SSE] LowerBUILD_VECTORAsVariablePermute - add support for scaling index vectors This allows us to use PSHUFB for v8i16/v4i32 and VPERMD/PERMPS for v4i64/v4f64 variable shuffles. Differential Revision: https://reviews.llvm.org/D42487 llvm-svn: 323987
2018-02-02 02:10:30 +08:00
; AVX2-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX2-NEXT: vpaddq %ymm1, %ymm1, %ymm1
[X86][SSE] LowerBUILD_VECTORAsVariablePermute - add support for scaling index vectors This allows us to use PSHUFB for v8i16/v4i32 and VPERMD/PERMPS for v4i64/v4f64 variable shuffles. Differential Revision: https://reviews.llvm.org/D42487 llvm-svn: 323987
2018-02-02 02:10:30 +08:00
; AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [2,2,2,2]
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX2-NEXT: vpcmpgtq %ymm2, %ymm1, %ymm2
; AVX2-NEXT: vpermpd {{.*#+}} ymm3 = ymm0[2,3,2,3]
; AVX2-NEXT: vpermilpd %ymm1, %ymm3, %ymm3
; AVX2-NEXT: vpermpd {{.*#+}} ymm0 = ymm0[0,1,0,1]
; AVX2-NEXT: vpermilpd %ymm1, %ymm0, %ymm0
; AVX2-NEXT: vblendvpd %ymm2, %ymm3, %ymm0, %ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX2-NEXT: retq
;
[X86][AVX512] Added more non-VLX test cases Cleaned up check prefixes so that they actually share a bit more llvm-svn: 327246
2018-03-12 02:28:37 +08:00
; AVX512-LABEL: var_shuffle_v4f64_from_v2f64:
; AVX512: # %bb.0:
; AVX512-NEXT: # kill: def $ymm1 killed $ymm1 def $zmm1
; AVX512-NEXT: # kill: def $xmm0 killed $xmm0 def $zmm0
; AVX512-NEXT: vpermpd %zmm0, %zmm1, %zmm0
; AVX512-NEXT: # kill: def $ymm0 killed $ymm0 killed $zmm0
; AVX512-NEXT: retq
[X86][AVX512DQ] Add DQ var permute 256 tests as requested on D42487 llvm-svn: 323970
2018-02-01 22:44:50 +08:00
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX512VL-LABEL: var_shuffle_v4f64_from_v2f64:
; AVX512VL: # %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512VL-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX512VL-NEXT: vpermpd %ymm0, %ymm1, %ymm0
; AVX512VL-NEXT: retq
%index0 = extractelement <4 x i64> %indices, i32 0
%index1 = extractelement <4 x i64> %indices, i32 1
%index2 = extractelement <4 x i64> %indices, i32 2
%index3 = extractelement <4 x i64> %indices, i32 3
%v0 = extractelement <2 x double> %v, i64 %index0
%v1 = extractelement <2 x double> %v, i64 %index1
%v2 = extractelement <2 x double> %v, i64 %index2
%v3 = extractelement <2 x double> %v, i64 %index3
%ret0 = insertelement <4 x double> undef, double %v0, i32 0
%ret1 = insertelement <4 x double> %ret0, double %v1, i32 1
%ret2 = insertelement <4 x double> %ret1, double %v2, i32 2
%ret3 = insertelement <4 x double> %ret2, double %v3, i32 3
ret <4 x double> %ret3
}
define <8 x float> @var_shuffle_v8f32_from_v4f32(<4 x float> %v, <8 x i32> %indices) unnamed_addr nounwind {
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-LABEL: var_shuffle_v8f32_from_v4f32:
; XOP: # %bb.0: # %entry
[X86][XOP] createVariablePermute - use VPERMIL2 for v8i32/v4i64 variable permutes llvm-svn: 327222
2018-03-11 03:49:59 +08:00
; XOP-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpermil2ps $0, %ymm1, %ymm2, %ymm0, %ymm0
[X86][XOP] Add XOP to variable permute tests llvm-svn: 324015
2018-02-02 05:57:37 +08:00
; XOP-NEXT: retq
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; AVX1-LABEL: var_shuffle_v8f32_from_v4f32:
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
; AVX1: # %bb.0: # %entry
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpermilps %ymm1, %ymm2, %ymm2
[X86][AVX] createVariablePermute - use 2xVPERMIL+PCMPGT+SELECT for v8i32/v8f32 and v4i64/v4f64 variable permutes As VPERMILPS/VPERMILPD only selects elements based on the bits[1:0]/bit[1] then we can permute both the (repeated) lo/hi 128-bit vectors in each case and then select between these results based on whether the index was for for lo/hi. For v4i64/v4f64 this avoids some rather nasty v4i64 multiples on the AVX2 implementation, which seems to be worse than the extra port5 pressure from the additional shuffles/blends. llvm-svn: 327239
2018-03-11 19:52:26 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; AVX1-NEXT: vpermilps %ymm1, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpcmpgtd {{.*}}(%rip), %xmm1, %xmm3
; AVX1-NEXT: vextractf128 $1, %ymm1, %xmm1
; AVX1-NEXT: vpcmpgtd {{\.LCPI.*}}+{{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm3, %ymm1
; AVX1-NEXT: vblendvps %ymm1, %ymm2, %ymm0, %ymm0
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
; AVX1-NEXT: retq
;
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
; INT256-LABEL: var_shuffle_v8f32_from_v4f32:
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
; INT256: # %bb.0: # %entry
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; INT256-NEXT: # kill: def $xmm0 killed $xmm0 def $ymm0
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
; INT256-NEXT: vpermps %ymm0, %ymm1, %ymm0
; INT256-NEXT: retq
entry:
%tmp1 = extractelement <8 x i32> %indices, i32 0
%vecext2.8 = extractelement <4 x float> %v, i32 %tmp1
%tmp2 = extractelement <8 x i32> %indices, i32 1
%vecext2.9 = extractelement <4 x float> %v, i32 %tmp2
%tmp3 = extractelement <8 x i32> %indices, i32 2
%vecext2.10 = extractelement <4 x float> %v, i32 %tmp3
%tmp4 = extractelement <8 x i32> %indices, i32 3
%vecext2.11 = extractelement <4 x float> %v, i32 %tmp4
%tmp5 = extractelement <8 x i32> %indices, i32 4
%vecext2.12 = extractelement <4 x float> %v, i32 %tmp5
%tmp6 = extractelement <8 x i32> %indices, i32 5
%vecext2.13 = extractelement <4 x float> %v, i32 %tmp6
%tmp7 = extractelement <8 x i32> %indices, i32 6
%vecext2.14 = extractelement <4 x float> %v, i32 %tmp7
%tmp8 = extractelement <8 x i32> %indices, i32 7
%vecext2.15 = extractelement <4 x float> %v, i32 %tmp8
%tmp9 = insertelement <8 x float> undef, float %vecext2.8, i32 0
%tmp10 = insertelement <8 x float> %tmp9, float %vecext2.9, i32 1
%tmp11 = insertelement <8 x float> %tmp10, float %vecext2.10, i32 2
%tmp12 = insertelement <8 x float> %tmp11, float %vecext2.11, i32 3
%tmp13 = insertelement <8 x float> %tmp12, float %vecext2.12, i32 4
%tmp14 = insertelement <8 x float> %tmp13, float %vecext2.13, i32 5
%tmp15 = insertelement <8 x float> %tmp14, float %vecext2.14, i32 6
%tmp16 = insertelement <8 x float> %tmp15, float %vecext2.15, i32 7
ret <8 x float> %tmp16
}
[X86][AVX] Add more variable permute tests for source vectors smaller than destination llvm-svn: 322948
2018-01-19 22:55:22 +08:00
define <4 x i32> @var_shuffle_v4i32_from_v8i32(<8 x i32> %v, <4 x i32> %indices) unnamed_addr nounwind {
[X86][AVX] createVariablePermute - widen permutes for cases where the source vector is wider than the destination type llvm-svn: 327244
2018-03-12 01:00:46 +08:00
; XOP-LABEL: var_shuffle_v4i32_from_v8i32:
; XOP: # %bb.0: # %entry
; XOP-NEXT: # kill: def $xmm1 killed $xmm1 def $ymm1
; XOP-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; XOP-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; XOP-NEXT: vpermil2ps $0, %ymm1, %ymm2, %ymm0, %ymm0
; XOP-NEXT: # kill: def $xmm0 killed $xmm0 killed $ymm0
; XOP-NEXT: vzeroupper
; XOP-NEXT: retq
;
; AVX1-LABEL: var_shuffle_v4i32_from_v8i32:
; AVX1: # %bb.0: # %entry
; AVX1-NEXT: # kill: def $xmm1 killed $xmm1 def $ymm1
; AVX1-NEXT: vperm2f128 {{.*#+}} ymm2 = ymm0[2,3,2,3]
; AVX1-NEXT: vpermilps %ymm1, %ymm2, %ymm2
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm0, %ymm0
; AVX1-NEXT: vpermilps %ymm1, %ymm0, %ymm0
[x86] allow vector load narrowing with multi-use values This is a long-awaited follow-up suggested in D33578. Since then, we've picked up even more opportunities for vector narrowing from changes like D53784, so there are a lot of test diffs. Apart from 2-3 strange cases, these are all wins. I've structured this to be no-functional-change-intended for any target except for x86 because I couldn't tell if AArch64, ARM, and AMDGPU would improve or not. All of those targets have existing regression tests (4, 4, 10 files respectively) that would be affected. Also, Hexagon overrides the shouldReduceLoadWidth() hook, but doesn't show any regression test diffs. The trade-off is deciding if an extra vector load is better than a single wide load + extract_subvector. For x86, this is almost always better (on paper at least) because we often can fold loads into subsequent ops and not increase the official instruction count. There's also some unknown -- but potentially large -- benefit from using narrower vector ops if wide ops are implemented with multiple uops and/or frequency throttling is avoided. Differential Revision: https://reviews.llvm.org/D54073 llvm-svn: 346595
2018-11-11 04:05:31 +08:00
; AVX1-NEXT: vpcmpgtd {{.*}}(%rip), %xmm1, %xmm1
; AVX1-NEXT: vpcmpgtd {{\.LCPI.*}}+{{.*}}(%rip), %xmm0, %xmm3
; AVX1-NEXT: vinsertf128 $1, %xmm3, %ymm1, %ymm1
[X86][AVX] createVariablePermute - widen permutes for cases where the source vector is wider than the destination type llvm-svn: 327244
2018-03-12 01:00:46 +08:00
; AVX1-NEXT: vblendvps %ymm1, %ymm2, %ymm0, %ymm0
; AVX1-NEXT: # kill: def $xmm0 killed $xmm0 killed $ymm0
; AVX1-NEXT: vzeroupper
; AVX1-NEXT: retq
;
; INT256-LABEL: var_shuffle_v4i32_from_v8i32:
; INT256: # %bb.0: # %entry
; INT256-NEXT: # kill: def $xmm1 killed $xmm1 def $ymm1
; INT256-NEXT: vpermps %ymm0, %ymm1, %ymm0
; INT256-NEXT: # kill: def $xmm0 killed $xmm0 killed $ymm0
; INT256-NEXT: vzeroupper
; INT256-NEXT: retq
X86: Fix LowerBUILD_VECTORAsVariablePermute for case Src is smaller than Indices Summary: As RKSimon suggested in pr35820, in the case that Src is smaller in bit-size than Indices, need to widen Src to avoid type mismatch. Fixes pr35820 Reviewers: RKSimon, craig.topper Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41865 llvm-svn: 322272
2018-01-11 20:26:52 +08:00
entry:
%tmp1 = extractelement <4 x i32> %indices, i32 0
%vecext2.8 = extractelement <8 x i32> %v, i32 %tmp1
%tmp2 = extractelement <4 x i32> %indices, i32 1
%vecext2.9 = extractelement <8 x i32> %v, i32 %tmp2
%tmp3 = extractelement <4 x i32> %indices, i32 2
%vecext2.10 = extractelement <8 x i32> %v, i32 %tmp3
%tmp4 = extractelement <4 x i32> %indices, i32 3
%vecext2.11 = extractelement <8 x i32> %v, i32 %tmp4
%tmp9 = insertelement <4 x i32> undef, i32 %vecext2.8, i32 0
%tmp10 = insertelement <4 x i32> %tmp9, i32 %vecext2.9, i32 1
%tmp11 = insertelement <4 x i32> %tmp10, i32 %vecext2.10, i32 2
%tmp12 = insertelement <4 x i32> %tmp11, i32 %vecext2.11, i32 3
ret <4 x i32> %tmp12
}