maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/vector-lzcnt-512.ll

679 lines
40 KiB
LLVM
Raw Normal View History

[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[X86][AVX512] Use AVX512 attributes instead of -mcpu llvm-svn: 312528
2017-09-05 20:05:14 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,+avx512cd,-avx512bw | FileCheck %s --check-prefix=ALL --check-prefix=AVX512CD
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,+avx512cd,+avx512bw | FileCheck %s --check-prefix=ALL --check-prefix=AVX512CDBW
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,-avx512cd,+avx512bw | FileCheck %s --check-prefix=ALL --check-prefix=AVX512BW
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512dq,-avx512cd,-avx512bw | FileCheck %s --check-prefix=ALL --check-prefix=AVX512DQ
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
define <8 x i64> @testv8i64(<8 x i64> %in) nounwind {
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CD-LABEL: testv8i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CD: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CD-NEXT: vplzcntq %zmm0, %zmm0
; AVX512CD-NEXT: retq
;
; AVX512CDBW-LABEL: testv8i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CDBW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-NEXT: vplzcntq %zmm0, %zmm0
; AVX512CDBW-NEXT: retq
;
; AVX512BW-LABEL: testv8i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512BW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrlq $1, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $2, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $4, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $8, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $16, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $32, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512BW-NEXT: vpandnq %zmm1, %zmm0, %zmm2
[AVX-512] Remove patterns that select vmovdqu8/16 for unmasked loads. Prefer vmovdqa64/vmovdqu64 instead. These were taking priority over the aligned load instructions since there is no vmovda8/16. I don't think there is really a difference between aligned and unaligned on newer cpus so I don't think it matters which instructions we use. But with this change we reduce the size of the isel table a little and we allow the aligned information to pass through to the evex->vec pass and produce the same output has avx/avx2 in some cases. I also generally dislike patterns rooted in a bitcast which these were. Differential Revision: https://reviews.llvm.org/D35977 llvm-svn: 309589
2017-08-01 01:35:44 +08:00
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpshufb %zmm2, %zmm3, %zmm2
[X86] Use vpternlog to implement vector not under AVX512. Previously we had to materialize all 1s in a register using vpternlog or pcmpeq and then xor with that. By using vpternlog directly we can do it in one operation. This is implemented using isel patterns, but we should maybe consider creating a generalized vpternlog combiner. llvm-svn: 323572
2018-01-27 06:17:40 +08:00
; AVX512BW-NEXT: vpternlogq $15, %zmm0, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrlw $4, %zmm0, %zmm0
; AVX512BW-NEXT: vpandq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpshufb %zmm0, %zmm3, %zmm0
; AVX512BW-NEXT: vpaddb %zmm2, %zmm0, %zmm0
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35965 llvm-svn: 309926
2017-08-03 16:50:18 +08:00
; AVX512BW-NEXT: vpxor %xmm1, %xmm1, %xmm1
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsadbw %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: retq
;
; AVX512DQ-LABEL: testv8i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512DQ: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrlq $1, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $2, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $4, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $8, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $16, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $32, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
[X86] Use vpternlog to implement vector not under AVX512. Previously we had to materialize all 1s in a register using vpternlog or pcmpeq and then xor with that. By using vpternlog directly we can do it in one operation. This is implemented using isel patterns, but we should maybe consider creating a generalized vpternlog combiner. llvm-svn: 323572
2018-01-27 06:17:40 +08:00
; AVX512DQ-NEXT: vpternlogq $15, %zmm0, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512DQ-NEXT: vpand %ymm2, %ymm1, %ymm3
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512DQ-NEXT: vpshufb %ymm3, %ymm4, %ymm3
; AVX512DQ-NEXT: vpsrlw $4, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm2, %ymm1, %ymm1
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm4, %ymm1
; AVX512DQ-NEXT: vpaddb %ymm3, %ymm1, %ymm1
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX512DQ-NEXT: vpxor %xmm3, %xmm3, %xmm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm2, %ymm0, %ymm5
; AVX512DQ-NEXT: vpshufb %ymm5, %ymm4, %ymm5
; AVX512DQ-NEXT: vpsrlw $4, %ymm0, %ymm0
; AVX512DQ-NEXT: vpand %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm4, %ymm0
; AVX512DQ-NEXT: vpaddb %ymm5, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512DQ-NEXT: retq
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
%out = call <8 x i64> @llvm.ctlz.v8i64(<8 x i64> %in, i1 0)
ret <8 x i64> %out
}
define <8 x i64> @testv8i64u(<8 x i64> %in) nounwind {
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CD-LABEL: testv8i64u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CD: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CD-NEXT: vplzcntq %zmm0, %zmm0
; AVX512CD-NEXT: retq
;
; AVX512CDBW-LABEL: testv8i64u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CDBW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-NEXT: vplzcntq %zmm0, %zmm0
; AVX512CDBW-NEXT: retq
;
; AVX512BW-LABEL: testv8i64u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512BW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrlq $1, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $2, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $4, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $8, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $16, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpsrlq $32, %zmm0, %zmm1
; AVX512BW-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512BW-NEXT: vpandnq %zmm1, %zmm0, %zmm2
[AVX-512] Remove patterns that select vmovdqu8/16 for unmasked loads. Prefer vmovdqa64/vmovdqu64 instead. These were taking priority over the aligned load instructions since there is no vmovda8/16. I don't think there is really a difference between aligned and unaligned on newer cpus so I don't think it matters which instructions we use. But with this change we reduce the size of the isel table a little and we allow the aligned information to pass through to the evex->vec pass and produce the same output has avx/avx2 in some cases. I also generally dislike patterns rooted in a bitcast which these were. Differential Revision: https://reviews.llvm.org/D35977 llvm-svn: 309589
2017-08-01 01:35:44 +08:00
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpshufb %zmm2, %zmm3, %zmm2
[X86] Use vpternlog to implement vector not under AVX512. Previously we had to materialize all 1s in a register using vpternlog or pcmpeq and then xor with that. By using vpternlog directly we can do it in one operation. This is implemented using isel patterns, but we should maybe consider creating a generalized vpternlog combiner. llvm-svn: 323572
2018-01-27 06:17:40 +08:00
; AVX512BW-NEXT: vpternlogq $15, %zmm0, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrlw $4, %zmm0, %zmm0
; AVX512BW-NEXT: vpandq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpshufb %zmm0, %zmm3, %zmm0
; AVX512BW-NEXT: vpaddb %zmm2, %zmm0, %zmm0
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35965 llvm-svn: 309926
2017-08-03 16:50:18 +08:00
; AVX512BW-NEXT: vpxor %xmm1, %xmm1, %xmm1
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsadbw %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: retq
;
; AVX512DQ-LABEL: testv8i64u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512DQ: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrlq $1, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $2, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $4, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $8, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $16, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
; AVX512DQ-NEXT: vpsrlq $32, %zmm0, %zmm1
; AVX512DQ-NEXT: vporq %zmm1, %zmm0, %zmm0
[X86] Use vpternlog to implement vector not under AVX512. Previously we had to materialize all 1s in a register using vpternlog or pcmpeq and then xor with that. By using vpternlog directly we can do it in one operation. This is implemented using isel patterns, but we should maybe consider creating a generalized vpternlog combiner. llvm-svn: 323572
2018-01-27 06:17:40 +08:00
; AVX512DQ-NEXT: vpternlogq $15, %zmm0, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512DQ-NEXT: vpand %ymm2, %ymm1, %ymm3
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512DQ-NEXT: vpshufb %ymm3, %ymm4, %ymm3
; AVX512DQ-NEXT: vpsrlw $4, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm2, %ymm1, %ymm1
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm4, %ymm1
; AVX512DQ-NEXT: vpaddb %ymm3, %ymm1, %ymm1
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX512DQ-NEXT: vpxor %xmm3, %xmm3, %xmm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm2, %ymm0, %ymm5
; AVX512DQ-NEXT: vpshufb %ymm5, %ymm4, %ymm5
; AVX512DQ-NEXT: vpsrlw $4, %ymm0, %ymm0
; AVX512DQ-NEXT: vpand %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm4, %ymm0
; AVX512DQ-NEXT: vpaddb %ymm5, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512DQ-NEXT: retq
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
%out = call <8 x i64> @llvm.ctlz.v8i64(<8 x i64> %in, i1 -1)
ret <8 x i64> %out
}
define <16 x i32> @testv16i32(<16 x i32> %in) nounwind {
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CD-LABEL: testv16i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CD: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CD-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CD-NEXT: retq
;
; AVX512CDBW-LABEL: testv16i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CDBW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CDBW-NEXT: retq
;
; AVX512BW-LABEL: testv16i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512BW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $1, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $2, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $4, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $8, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $16, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512BW-NEXT: vpandnq %zmm1, %zmm0, %zmm2
[AVX-512] Remove patterns that select vmovdqu8/16 for unmasked loads. Prefer vmovdqa64/vmovdqu64 instead. These were taking priority over the aligned load instructions since there is no vmovda8/16. I don't think there is really a difference between aligned and unaligned on newer cpus so I don't think it matters which instructions we use. But with this change we reduce the size of the isel table a little and we allow the aligned information to pass through to the evex->vec pass and produce the same output has avx/avx2 in some cases. I also generally dislike patterns rooted in a bitcast which these were. Differential Revision: https://reviews.llvm.org/D35977 llvm-svn: 309589
2017-08-01 01:35:44 +08:00
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpshufb %zmm2, %zmm3, %zmm2
[X86] Use vpternlog to implement vector not under AVX512. Previously we had to materialize all 1s in a register using vpternlog or pcmpeq and then xor with that. By using vpternlog directly we can do it in one operation. This is implemented using isel patterns, but we should maybe consider creating a generalized vpternlog combiner. llvm-svn: 323572
2018-01-27 06:17:40 +08:00
; AVX512BW-NEXT: vpternlogq $15, %zmm0, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrlw $4, %zmm0, %zmm0
; AVX512BW-NEXT: vpandq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpshufb %zmm0, %zmm3, %zmm0
; AVX512BW-NEXT: vpaddb %zmm2, %zmm0, %zmm0
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35965 llvm-svn: 309926
2017-08-03 16:50:18 +08:00
; AVX512BW-NEXT: vpxor %xmm1, %xmm1, %xmm1
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpunpckhdq {{.*#+}} zmm2 = zmm0[2],zmm1[2],zmm0[3],zmm1[3],zmm0[6],zmm1[6],zmm0[7],zmm1[7],zmm0[10],zmm1[10],zmm0[11],zmm1[11],zmm0[14],zmm1[14],zmm0[15],zmm1[15]
; AVX512BW-NEXT: vpsadbw %zmm1, %zmm2, %zmm2
; AVX512BW-NEXT: vpunpckldq {{.*#+}} zmm0 = zmm0[0],zmm1[0],zmm0[1],zmm1[1],zmm0[4],zmm1[4],zmm0[5],zmm1[5],zmm0[8],zmm1[8],zmm0[9],zmm1[9],zmm0[12],zmm1[12],zmm0[13],zmm1[13]
; AVX512BW-NEXT: vpsadbw %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpackuswb %zmm2, %zmm0, %zmm0
; AVX512BW-NEXT: retq
;
; AVX512DQ-LABEL: testv16i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512DQ: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $1, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $2, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $4, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $8, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $16, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86] Use vpternlog to implement vector not under AVX512. Previously we had to materialize all 1s in a register using vpternlog or pcmpeq and then xor with that. By using vpternlog directly we can do it in one operation. This is implemented using isel patterns, but we should maybe consider creating a generalized vpternlog combiner. llvm-svn: 323572
2018-01-27 06:17:40 +08:00
; AVX512DQ-NEXT: vpternlogq $15, %zmm0, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512DQ-NEXT: vpand %ymm2, %ymm1, %ymm3
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512DQ-NEXT: vpshufb %ymm3, %ymm4, %ymm3
; AVX512DQ-NEXT: vpsrlw $4, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm2, %ymm1, %ymm1
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm4, %ymm1
; AVX512DQ-NEXT: vpaddb %ymm3, %ymm1, %ymm1
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX512DQ-NEXT: vpxor %xmm3, %xmm3, %xmm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpunpckhdq {{.*#+}} ymm5 = ymm1[2],ymm3[2],ymm1[3],ymm3[3],ymm1[6],ymm3[6],ymm1[7],ymm3[7]
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm5, %ymm5
; AVX512DQ-NEXT: vpunpckldq {{.*#+}} ymm1 = ymm1[0],ymm3[0],ymm1[1],ymm3[1],ymm1[4],ymm3[4],ymm1[5],ymm3[5]
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm1, %ymm1
; AVX512DQ-NEXT: vpackuswb %ymm5, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm2, %ymm0, %ymm5
; AVX512DQ-NEXT: vpshufb %ymm5, %ymm4, %ymm5
; AVX512DQ-NEXT: vpsrlw $4, %ymm0, %ymm0
; AVX512DQ-NEXT: vpand %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm4, %ymm0
; AVX512DQ-NEXT: vpaddb %ymm5, %ymm0, %ymm0
; AVX512DQ-NEXT: vpunpckhdq {{.*#+}} ymm2 = ymm0[2],ymm3[2],ymm0[3],ymm3[3],ymm0[6],ymm3[6],ymm0[7],ymm3[7]
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm2, %ymm2
; AVX512DQ-NEXT: vpunpckldq {{.*#+}} ymm0 = ymm0[0],ymm3[0],ymm0[1],ymm3[1],ymm0[4],ymm3[4],ymm0[5],ymm3[5]
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512DQ-NEXT: retq
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
%out = call <16 x i32> @llvm.ctlz.v16i32(<16 x i32> %in, i1 0)
ret <16 x i32> %out
}
define <16 x i32> @testv16i32u(<16 x i32> %in) nounwind {
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CD-LABEL: testv16i32u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CD: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CD-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CD-NEXT: retq
;
; AVX512CDBW-LABEL: testv16i32u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CDBW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CDBW-NEXT: retq
;
; AVX512BW-LABEL: testv16i32u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512BW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $1, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $2, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $4, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $8, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrld $16, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512BW-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512BW-NEXT: vpandnq %zmm1, %zmm0, %zmm2
[AVX-512] Remove patterns that select vmovdqu8/16 for unmasked loads. Prefer vmovdqa64/vmovdqu64 instead. These were taking priority over the aligned load instructions since there is no vmovda8/16. I don't think there is really a difference between aligned and unaligned on newer cpus so I don't think it matters which instructions we use. But with this change we reduce the size of the isel table a little and we allow the aligned information to pass through to the evex->vec pass and produce the same output has avx/avx2 in some cases. I also generally dislike patterns rooted in a bitcast which these were. Differential Revision: https://reviews.llvm.org/D35977 llvm-svn: 309589
2017-08-01 01:35:44 +08:00
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpshufb %zmm2, %zmm3, %zmm2
[X86] Use vpternlog to implement vector not under AVX512. Previously we had to materialize all 1s in a register using vpternlog or pcmpeq and then xor with that. By using vpternlog directly we can do it in one operation. This is implemented using isel patterns, but we should maybe consider creating a generalized vpternlog combiner. llvm-svn: 323572
2018-01-27 06:17:40 +08:00
; AVX512BW-NEXT: vpternlogq $15, %zmm0, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrlw $4, %zmm0, %zmm0
; AVX512BW-NEXT: vpandq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpshufb %zmm0, %zmm3, %zmm0
; AVX512BW-NEXT: vpaddb %zmm2, %zmm0, %zmm0
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35965 llvm-svn: 309926
2017-08-03 16:50:18 +08:00
; AVX512BW-NEXT: vpxor %xmm1, %xmm1, %xmm1
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpunpckhdq {{.*#+}} zmm2 = zmm0[2],zmm1[2],zmm0[3],zmm1[3],zmm0[6],zmm1[6],zmm0[7],zmm1[7],zmm0[10],zmm1[10],zmm0[11],zmm1[11],zmm0[14],zmm1[14],zmm0[15],zmm1[15]
; AVX512BW-NEXT: vpsadbw %zmm1, %zmm2, %zmm2
; AVX512BW-NEXT: vpunpckldq {{.*#+}} zmm0 = zmm0[0],zmm1[0],zmm0[1],zmm1[1],zmm0[4],zmm1[4],zmm0[5],zmm1[5],zmm0[8],zmm1[8],zmm0[9],zmm1[9],zmm0[12],zmm1[12],zmm0[13],zmm1[13]
; AVX512BW-NEXT: vpsadbw %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpackuswb %zmm2, %zmm0, %zmm0
; AVX512BW-NEXT: retq
;
; AVX512DQ-LABEL: testv16i32u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512DQ: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $1, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $2, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $4, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $8, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrld $16, %zmm0, %zmm1
[X86] Stop promoting vector and/or/xor/andn to vXi64. These promotions add additional bitcasts to the SelectionDAG that can pessimize computeKnownBits/computeNumSignBits. It also seems to interfere with broadcast formation. This patch removes the promotion and adds isel patterns instead. The increased table size is more than I would like, but hopefully we can find some canonicalizations or other tricks to start pruning out patterns going forward. Differential Revision: https://reviews.llvm.org/D53268 llvm-svn: 345408
2018-10-27 01:21:26 +08:00
; AVX512DQ-NEXT: vpord %zmm1, %zmm0, %zmm0
[X86] Use vpternlog to implement vector not under AVX512. Previously we had to materialize all 1s in a register using vpternlog or pcmpeq and then xor with that. By using vpternlog directly we can do it in one operation. This is implemented using isel patterns, but we should maybe consider creating a generalized vpternlog combiner. llvm-svn: 323572
2018-01-27 06:17:40 +08:00
; AVX512DQ-NEXT: vpternlogq $15, %zmm0, %zmm0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512DQ-NEXT: vpand %ymm2, %ymm1, %ymm3
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512DQ-NEXT: vpshufb %ymm3, %ymm4, %ymm3
; AVX512DQ-NEXT: vpsrlw $4, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm2, %ymm1, %ymm1
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm4, %ymm1
; AVX512DQ-NEXT: vpaddb %ymm3, %ymm1, %ymm1
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX512DQ-NEXT: vpxor %xmm3, %xmm3, %xmm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpunpckhdq {{.*#+}} ymm5 = ymm1[2],ymm3[2],ymm1[3],ymm3[3],ymm1[6],ymm3[6],ymm1[7],ymm3[7]
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm5, %ymm5
; AVX512DQ-NEXT: vpunpckldq {{.*#+}} ymm1 = ymm1[0],ymm3[0],ymm1[1],ymm3[1],ymm1[4],ymm3[4],ymm1[5],ymm3[5]
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm1, %ymm1
; AVX512DQ-NEXT: vpackuswb %ymm5, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm2, %ymm0, %ymm5
; AVX512DQ-NEXT: vpshufb %ymm5, %ymm4, %ymm5
; AVX512DQ-NEXT: vpsrlw $4, %ymm0, %ymm0
; AVX512DQ-NEXT: vpand %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm4, %ymm0
; AVX512DQ-NEXT: vpaddb %ymm5, %ymm0, %ymm0
; AVX512DQ-NEXT: vpunpckhdq {{.*#+}} ymm2 = ymm0[2],ymm3[2],ymm0[3],ymm3[3],ymm0[6],ymm3[6],ymm0[7],ymm3[7]
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm2, %ymm2
; AVX512DQ-NEXT: vpunpckldq {{.*#+}} ymm0 = ymm0[0],ymm3[0],ymm0[1],ymm3[1],ymm0[4],ymm3[4],ymm0[5],ymm3[5]
; AVX512DQ-NEXT: vpsadbw %ymm3, %ymm0, %ymm0
; AVX512DQ-NEXT: vpackuswb %ymm2, %ymm0, %ymm0
; AVX512DQ-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512DQ-NEXT: retq
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
%out = call <16 x i32> @llvm.ctlz.v16i32(<16 x i32> %in, i1 -1)
ret <16 x i32> %out
}
define <32 x i16> @testv32i16(<32 x i16> %in) nounwind {
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-LABEL: testv32i16:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CD: # %bb.0:
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxwd {{.*#+}} zmm0 = ymm0[0],zero,ymm0[1],zero,ymm0[2],zero,ymm0[3],zero,ymm0[4],zero,ymm0[5],zero,ymm0[6],zero,ymm0[7],zero,ymm0[8],zero,ymm0[9],zero,ymm0[10],zero,ymm0[11],zero,ymm0[12],zero,ymm0[13],zero,ymm0[14],zero,ymm0[15],zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CD-NEXT: vpmovdw %zmm0, %ymm0
; AVX512CD-NEXT: vmovdqa {{.*#+}} ymm2 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; AVX512CD-NEXT: vpsubw %ymm2, %ymm0, %ymm0
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxwd {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm1, %zmm1
; AVX512CD-NEXT: vpmovdw %zmm1, %ymm1
; AVX512CD-NEXT: vpsubw %ymm2, %ymm1, %ymm1
; AVX512CD-NEXT: retq
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
;
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-LABEL: testv32i16:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CDBW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512CDBW-NEXT: vpmovzxwd {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero
; AVX512CDBW-NEXT: vplzcntd %zmm1, %zmm1
; AVX512CDBW-NEXT: vpmovdw %zmm1, %ymm1
; AVX512CDBW-NEXT: vmovdqa {{.*#+}} ymm2 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; AVX512CDBW-NEXT: vpsubw %ymm2, %ymm1, %ymm1
; AVX512CDBW-NEXT: vpmovzxwd {{.*#+}} zmm0 = ymm0[0],zero,ymm0[1],zero,ymm0[2],zero,ymm0[3],zero,ymm0[4],zero,ymm0[5],zero,ymm0[6],zero,ymm0[7],zero,ymm0[8],zero,ymm0[9],zero,ymm0[10],zero,ymm0[11],zero,ymm0[12],zero,ymm0[13],zero,ymm0[14],zero,ymm0[15],zero
; AVX512CDBW-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CDBW-NEXT: vpmovdw %zmm0, %ymm0
; AVX512CDBW-NEXT: vpsubw %ymm2, %ymm0, %ymm0
; AVX512CDBW-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512CDBW-NEXT: retq
;
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
; AVX512BW-LABEL: testv32i16:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512BW: # %bb.0:
[X86] Move VPTESTM matching from the isel table to custom code in X86ISelDAGToDAG. We had many tablegen patterns for these instructions. And due to the commutability of the patterns, tablegen expands them to even more patterns. All together VPTESTMD patterns accounted for more the 50K of the 610K isel table. This had gotten bad when we stopped canonicalizing AND to vXi64. This required a pattern for every combination of bitcast input type. This change moves the matching to custom code where it is easier to look through the bitcasts without being concerned with the specific types. The test changes are because we are now stricter with one use checks as its required to make load folding legal. We now require the AND and any BITCAST to only have a single use. This prevents forming VPTESTM and a VPAND with the same inputs. We now support broadcast loads for 128/256 patterns without VLX. We'll widen to 512-bit like and still fold the broadcast since the amount of memory read doesn't change. There are a few tests that got slightly longer because are now prefering load + VPTESTM over XOR+VPCMPEQ for (seteq (load), allzeros). Previously we were able to share the XOR with multiple VPTESTM instructions. llvm-svn: 358359
2019-04-15 02:26:11 +08:00
; AVX512BW-NEXT: vpsrlw $4, %zmm0, %zmm1
; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0]
; AVX512BW-NEXT: vpshufb %zmm1, %zmm2, %zmm3
; AVX512BW-NEXT: vpshufb %zmm0, %zmm2, %zmm2
; AVX512BW-NEXT: vptestnmb %zmm1, %zmm1, %k0
; AVX512BW-NEXT: vpmovm2b %k0, %zmm1
; AVX512BW-NEXT: vpandq %zmm1, %zmm2, %zmm1
; AVX512BW-NEXT: vpaddb %zmm3, %zmm1, %zmm1
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512BW-NEXT: vptestnmb %zmm0, %zmm0, %k0
[X86] Support v32i16/v64i8 CTLZ using lookup table. Had to tweak the setcc's used by the code to use a vXi1 result type with a sign extend back to vector size. llvm-svn: 318871
2017-11-23 04:05:57 +08:00
; AVX512BW-NEXT: vpmovm2b %k0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrlw $8, %zmm0, %zmm0
[X86] Support v32i16/v64i8 CTLZ using lookup table. Had to tweak the setcc's used by the code to use a vXi1 result type with a sign extend back to vector size. llvm-svn: 318871
2017-11-23 04:05:57 +08:00
; AVX512BW-NEXT: vpandq %zmm0, %zmm1, %zmm0
; AVX512BW-NEXT: vpsrlw $8, %zmm1, %zmm1
; AVX512BW-NEXT: vpaddw %zmm0, %zmm1, %zmm0
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
; AVX512BW-NEXT: retq
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
;
; AVX512DQ-LABEL: testv32i16:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512DQ: # %bb.0:
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm2 = [4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0]
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm2, %ymm3
; AVX512DQ-NEXT: vpsrlw $4, %ymm0, %ymm4
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm5 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512DQ-NEXT: vpand %ymm5, %ymm4, %ymm4
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX512DQ-NEXT: vpxor %xmm6, %xmm6, %xmm6
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpcmpeqb %ymm6, %ymm4, %ymm7
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpand %ymm7, %ymm3, %ymm3
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm4, %ymm2, %ymm4
; AVX512DQ-NEXT: vpaddb %ymm4, %ymm3, %ymm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpcmpeqb %ymm6, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512DQ-NEXT: vpand %ymm0, %ymm3, %ymm0
; AVX512DQ-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX512DQ-NEXT: vpaddw %ymm0, %ymm3, %ymm0
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm2, %ymm3
; AVX512DQ-NEXT: vpsrlw $4, %ymm1, %ymm4
; AVX512DQ-NEXT: vpand %ymm5, %ymm4, %ymm4
; AVX512DQ-NEXT: vpcmpeqb %ymm6, %ymm4, %ymm5
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpand %ymm5, %ymm3, %ymm3
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm4, %ymm2, %ymm2
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512DQ-NEXT: vpcmpeqb %ymm6, %ymm1, %ymm1
; AVX512DQ-NEXT: vpsrlw $8, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm1, %ymm2, %ymm1
; AVX512DQ-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512DQ-NEXT: vpaddw %ymm1, %ymm2, %ymm1
; AVX512DQ-NEXT: retq
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
%out = call <32 x i16> @llvm.ctlz.v32i16(<32 x i16> %in, i1 0)
ret <32 x i16> %out
}
define <32 x i16> @testv32i16u(<32 x i16> %in) nounwind {
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-LABEL: testv32i16u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CD: # %bb.0:
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxwd {{.*#+}} zmm0 = ymm0[0],zero,ymm0[1],zero,ymm0[2],zero,ymm0[3],zero,ymm0[4],zero,ymm0[5],zero,ymm0[6],zero,ymm0[7],zero,ymm0[8],zero,ymm0[9],zero,ymm0[10],zero,ymm0[11],zero,ymm0[12],zero,ymm0[13],zero,ymm0[14],zero,ymm0[15],zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CD-NEXT: vpmovdw %zmm0, %ymm0
; AVX512CD-NEXT: vmovdqa {{.*#+}} ymm2 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; AVX512CD-NEXT: vpsubw %ymm2, %ymm0, %ymm0
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxwd {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm1, %zmm1
; AVX512CD-NEXT: vpmovdw %zmm1, %ymm1
; AVX512CD-NEXT: vpsubw %ymm2, %ymm1, %ymm1
; AVX512CD-NEXT: retq
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
;
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-LABEL: testv32i16u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CDBW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512CDBW-NEXT: vpmovzxwd {{.*#+}} zmm1 = ymm1[0],zero,ymm1[1],zero,ymm1[2],zero,ymm1[3],zero,ymm1[4],zero,ymm1[5],zero,ymm1[6],zero,ymm1[7],zero,ymm1[8],zero,ymm1[9],zero,ymm1[10],zero,ymm1[11],zero,ymm1[12],zero,ymm1[13],zero,ymm1[14],zero,ymm1[15],zero
; AVX512CDBW-NEXT: vplzcntd %zmm1, %zmm1
; AVX512CDBW-NEXT: vpmovdw %zmm1, %ymm1
; AVX512CDBW-NEXT: vmovdqa {{.*#+}} ymm2 = [16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16]
; AVX512CDBW-NEXT: vpsubw %ymm2, %ymm1, %ymm1
; AVX512CDBW-NEXT: vpmovzxwd {{.*#+}} zmm0 = ymm0[0],zero,ymm0[1],zero,ymm0[2],zero,ymm0[3],zero,ymm0[4],zero,ymm0[5],zero,ymm0[6],zero,ymm0[7],zero,ymm0[8],zero,ymm0[9],zero,ymm0[10],zero,ymm0[11],zero,ymm0[12],zero,ymm0[13],zero,ymm0[14],zero,ymm0[15],zero
; AVX512CDBW-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CDBW-NEXT: vpmovdw %zmm0, %ymm0
; AVX512CDBW-NEXT: vpsubw %ymm2, %ymm0, %ymm0
; AVX512CDBW-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512CDBW-NEXT: retq
;
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
; AVX512BW-LABEL: testv32i16u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512BW: # %bb.0:
[X86] Move VPTESTM matching from the isel table to custom code in X86ISelDAGToDAG. We had many tablegen patterns for these instructions. And due to the commutability of the patterns, tablegen expands them to even more patterns. All together VPTESTMD patterns accounted for more the 50K of the 610K isel table. This had gotten bad when we stopped canonicalizing AND to vXi64. This required a pattern for every combination of bitcast input type. This change moves the matching to custom code where it is easier to look through the bitcasts without being concerned with the specific types. The test changes are because we are now stricter with one use checks as its required to make load folding legal. We now require the AND and any BITCAST to only have a single use. This prevents forming VPTESTM and a VPAND with the same inputs. We now support broadcast loads for 128/256 patterns without VLX. We'll widen to 512-bit like and still fold the broadcast since the amount of memory read doesn't change. There are a few tests that got slightly longer because are now prefering load + VPTESTM over XOR+VPCMPEQ for (seteq (load), allzeros). Previously we were able to share the XOR with multiple VPTESTM instructions. llvm-svn: 358359
2019-04-15 02:26:11 +08:00
; AVX512BW-NEXT: vpsrlw $4, %zmm0, %zmm1
; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0]
; AVX512BW-NEXT: vpshufb %zmm1, %zmm2, %zmm3
; AVX512BW-NEXT: vpshufb %zmm0, %zmm2, %zmm2
; AVX512BW-NEXT: vptestnmb %zmm1, %zmm1, %k0
; AVX512BW-NEXT: vpmovm2b %k0, %zmm1
; AVX512BW-NEXT: vpandq %zmm1, %zmm2, %zmm1
; AVX512BW-NEXT: vpaddb %zmm3, %zmm1, %zmm1
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512BW-NEXT: vptestnmb %zmm0, %zmm0, %k0
[X86] Support v32i16/v64i8 CTLZ using lookup table. Had to tweak the setcc's used by the code to use a vXi1 result type with a sign extend back to vector size. llvm-svn: 318871
2017-11-23 04:05:57 +08:00
; AVX512BW-NEXT: vpmovm2b %k0, %zmm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512BW-NEXT: vpsrlw $8, %zmm0, %zmm0
[X86] Support v32i16/v64i8 CTLZ using lookup table. Had to tweak the setcc's used by the code to use a vXi1 result type with a sign extend back to vector size. llvm-svn: 318871
2017-11-23 04:05:57 +08:00
; AVX512BW-NEXT: vpandq %zmm0, %zmm1, %zmm0
; AVX512BW-NEXT: vpsrlw $8, %zmm1, %zmm1
; AVX512BW-NEXT: vpaddw %zmm0, %zmm1, %zmm0
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
; AVX512BW-NEXT: retq
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
;
; AVX512DQ-LABEL: testv32i16u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512DQ: # %bb.0:
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm2 = [4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0]
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm2, %ymm3
; AVX512DQ-NEXT: vpsrlw $4, %ymm0, %ymm4
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm5 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512DQ-NEXT: vpand %ymm5, %ymm4, %ymm4
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX512DQ-NEXT: vpxor %xmm6, %xmm6, %xmm6
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpcmpeqb %ymm6, %ymm4, %ymm7
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpand %ymm7, %ymm3, %ymm3
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm4, %ymm2, %ymm4
; AVX512DQ-NEXT: vpaddb %ymm4, %ymm3, %ymm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpcmpeqb %ymm6, %ymm0, %ymm0
; AVX512DQ-NEXT: vpsrlw $8, %ymm0, %ymm0
; AVX512DQ-NEXT: vpand %ymm0, %ymm3, %ymm0
; AVX512DQ-NEXT: vpsrlw $8, %ymm3, %ymm3
; AVX512DQ-NEXT: vpaddw %ymm0, %ymm3, %ymm0
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm2, %ymm3
; AVX512DQ-NEXT: vpsrlw $4, %ymm1, %ymm4
; AVX512DQ-NEXT: vpand %ymm5, %ymm4, %ymm4
; AVX512DQ-NEXT: vpcmpeqb %ymm6, %ymm4, %ymm5
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpand %ymm5, %ymm3, %ymm3
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm4, %ymm2, %ymm2
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpaddb %ymm2, %ymm3, %ymm2
; AVX512DQ-NEXT: vpcmpeqb %ymm6, %ymm1, %ymm1
; AVX512DQ-NEXT: vpsrlw $8, %ymm1, %ymm1
; AVX512DQ-NEXT: vpand %ymm1, %ymm2, %ymm1
; AVX512DQ-NEXT: vpsrlw $8, %ymm2, %ymm2
; AVX512DQ-NEXT: vpaddw %ymm1, %ymm2, %ymm1
; AVX512DQ-NEXT: retq
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
%out = call <32 x i16> @llvm.ctlz.v32i16(<32 x i16> %in, i1 -1)
ret <32 x i16> %out
}
define <64 x i8> @testv64i8(<64 x i8> %in) nounwind {
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-LABEL: testv64i8:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CD: # %bb.0:
[AVX512] Add ExeDomain to vector extend and truncate instructions. llvm-svn: 276394
2016-07-22 13:46:44 +08:00
; AVX512CD-NEXT: vextracti128 $1, %ymm0, %xmm2
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxbd {{.*#+}} zmm2 = xmm2[0],zero,zero,zero,xmm2[1],zero,zero,zero,xmm2[2],zero,zero,zero,xmm2[3],zero,zero,zero,xmm2[4],zero,zero,zero,xmm2[5],zero,zero,zero,xmm2[6],zero,zero,zero,xmm2[7],zero,zero,zero,xmm2[8],zero,zero,zero,xmm2[9],zero,zero,zero,xmm2[10],zero,zero,zero,xmm2[11],zero,zero,zero,xmm2[12],zero,zero,zero,xmm2[13],zero,zero,zero,xmm2[14],zero,zero,zero,xmm2[15],zero,zero,zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm2, %zmm2
; AVX512CD-NEXT: vpmovdb %zmm2, %xmm2
; AVX512CD-NEXT: vmovdqa {{.*#+}} xmm3 = [24,24,24,24,24,24,24,24,24,24,24,24,24,24,24,24]
; AVX512CD-NEXT: vpsubb %xmm3, %xmm2, %xmm2
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxbd {{.*#+}} zmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero,xmm0[2],zero,zero,zero,xmm0[3],zero,zero,zero,xmm0[4],zero,zero,zero,xmm0[5],zero,zero,zero,xmm0[6],zero,zero,zero,xmm0[7],zero,zero,zero,xmm0[8],zero,zero,zero,xmm0[9],zero,zero,zero,xmm0[10],zero,zero,zero,xmm0[11],zero,zero,zero,xmm0[12],zero,zero,zero,xmm0[13],zero,zero,zero,xmm0[14],zero,zero,zero,xmm0[15],zero,zero,zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CD-NEXT: vpmovdb %zmm0, %xmm0
; AVX512CD-NEXT: vpsubb %xmm3, %xmm0, %xmm0
; AVX512CD-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
[AVX512] Add ExeDomain to vector extend and truncate instructions. llvm-svn: 276394
2016-07-22 13:46:44 +08:00
; AVX512CD-NEXT: vextracti128 $1, %ymm1, %xmm2
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxbd {{.*#+}} zmm2 = xmm2[0],zero,zero,zero,xmm2[1],zero,zero,zero,xmm2[2],zero,zero,zero,xmm2[3],zero,zero,zero,xmm2[4],zero,zero,zero,xmm2[5],zero,zero,zero,xmm2[6],zero,zero,zero,xmm2[7],zero,zero,zero,xmm2[8],zero,zero,zero,xmm2[9],zero,zero,zero,xmm2[10],zero,zero,zero,xmm2[11],zero,zero,zero,xmm2[12],zero,zero,zero,xmm2[13],zero,zero,zero,xmm2[14],zero,zero,zero,xmm2[15],zero,zero,zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm2, %zmm2
; AVX512CD-NEXT: vpmovdb %zmm2, %xmm2
; AVX512CD-NEXT: vpsubb %xmm3, %xmm2, %xmm2
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxbd {{.*#+}} zmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero,xmm1[2],zero,zero,zero,xmm1[3],zero,zero,zero,xmm1[4],zero,zero,zero,xmm1[5],zero,zero,zero,xmm1[6],zero,zero,zero,xmm1[7],zero,zero,zero,xmm1[8],zero,zero,zero,xmm1[9],zero,zero,zero,xmm1[10],zero,zero,zero,xmm1[11],zero,zero,zero,xmm1[12],zero,zero,zero,xmm1[13],zero,zero,zero,xmm1[14],zero,zero,zero,xmm1[15],zero,zero,zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm1, %zmm1
; AVX512CD-NEXT: vpmovdb %zmm1, %xmm1
; AVX512CD-NEXT: vpsubb %xmm3, %xmm1, %xmm1
; AVX512CD-NEXT: vinserti128 $1, %xmm2, %ymm1, %ymm1
; AVX512CD-NEXT: retq
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
;
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-LABEL: testv64i8:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CDBW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512CDBW-NEXT: vextracti128 $1, %ymm1, %xmm2
; AVX512CDBW-NEXT: vpmovzxbd {{.*#+}} zmm2 = xmm2[0],zero,zero,zero,xmm2[1],zero,zero,zero,xmm2[2],zero,zero,zero,xmm2[3],zero,zero,zero,xmm2[4],zero,zero,zero,xmm2[5],zero,zero,zero,xmm2[6],zero,zero,zero,xmm2[7],zero,zero,zero,xmm2[8],zero,zero,zero,xmm2[9],zero,zero,zero,xmm2[10],zero,zero,zero,xmm2[11],zero,zero,zero,xmm2[12],zero,zero,zero,xmm2[13],zero,zero,zero,xmm2[14],zero,zero,zero,xmm2[15],zero,zero,zero
; AVX512CDBW-NEXT: vplzcntd %zmm2, %zmm2
; AVX512CDBW-NEXT: vpmovdb %zmm2, %xmm2
; AVX512CDBW-NEXT: vmovdqa {{.*#+}} xmm3 = [24,24,24,24,24,24,24,24,24,24,24,24,24,24,24,24]
; AVX512CDBW-NEXT: vpsubb %xmm3, %xmm2, %xmm2
; AVX512CDBW-NEXT: vpmovzxbd {{.*#+}} zmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero,xmm1[2],zero,zero,zero,xmm1[3],zero,zero,zero,xmm1[4],zero,zero,zero,xmm1[5],zero,zero,zero,xmm1[6],zero,zero,zero,xmm1[7],zero,zero,zero,xmm1[8],zero,zero,zero,xmm1[9],zero,zero,zero,xmm1[10],zero,zero,zero,xmm1[11],zero,zero,zero,xmm1[12],zero,zero,zero,xmm1[13],zero,zero,zero,xmm1[14],zero,zero,zero,xmm1[15],zero,zero,zero
; AVX512CDBW-NEXT: vplzcntd %zmm1, %zmm1
; AVX512CDBW-NEXT: vpmovdb %zmm1, %xmm1
; AVX512CDBW-NEXT: vpsubb %xmm3, %xmm1, %xmm1
; AVX512CDBW-NEXT: vinserti128 $1, %xmm2, %ymm1, %ymm1
; AVX512CDBW-NEXT: vextracti128 $1, %ymm0, %xmm2
; AVX512CDBW-NEXT: vpmovzxbd {{.*#+}} zmm2 = xmm2[0],zero,zero,zero,xmm2[1],zero,zero,zero,xmm2[2],zero,zero,zero,xmm2[3],zero,zero,zero,xmm2[4],zero,zero,zero,xmm2[5],zero,zero,zero,xmm2[6],zero,zero,zero,xmm2[7],zero,zero,zero,xmm2[8],zero,zero,zero,xmm2[9],zero,zero,zero,xmm2[10],zero,zero,zero,xmm2[11],zero,zero,zero,xmm2[12],zero,zero,zero,xmm2[13],zero,zero,zero,xmm2[14],zero,zero,zero,xmm2[15],zero,zero,zero
; AVX512CDBW-NEXT: vplzcntd %zmm2, %zmm2
; AVX512CDBW-NEXT: vpmovdb %zmm2, %xmm2
; AVX512CDBW-NEXT: vpsubb %xmm3, %xmm2, %xmm2
; AVX512CDBW-NEXT: vpmovzxbd {{.*#+}} zmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero,xmm0[2],zero,zero,zero,xmm0[3],zero,zero,zero,xmm0[4],zero,zero,zero,xmm0[5],zero,zero,zero,xmm0[6],zero,zero,zero,xmm0[7],zero,zero,zero,xmm0[8],zero,zero,zero,xmm0[9],zero,zero,zero,xmm0[10],zero,zero,zero,xmm0[11],zero,zero,zero,xmm0[12],zero,zero,zero,xmm0[13],zero,zero,zero,xmm0[14],zero,zero,zero,xmm0[15],zero,zero,zero
; AVX512CDBW-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CDBW-NEXT: vpmovdb %zmm0, %xmm0
; AVX512CDBW-NEXT: vpsubb %xmm3, %xmm0, %xmm0
; AVX512CDBW-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
; AVX512CDBW-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512CDBW-NEXT: retq
;
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
; AVX512BW-LABEL: testv64i8:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512BW: # %bb.0:
[X86] Move VPTESTM matching from the isel table to custom code in X86ISelDAGToDAG. We had many tablegen patterns for these instructions. And due to the commutability of the patterns, tablegen expands them to even more patterns. All together VPTESTMD patterns accounted for more the 50K of the 610K isel table. This had gotten bad when we stopped canonicalizing AND to vXi64. This required a pattern for every combination of bitcast input type. This change moves the matching to custom code where it is easier to look through the bitcasts without being concerned with the specific types. The test changes are because we are now stricter with one use checks as its required to make load folding legal. We now require the AND and any BITCAST to only have a single use. This prevents forming VPTESTM and a VPAND with the same inputs. We now support broadcast loads for 128/256 patterns without VLX. We'll widen to 512-bit like and still fold the broadcast since the amount of memory read doesn't change. There are a few tests that got slightly longer because are now prefering load + VPTESTM over XOR+VPCMPEQ for (seteq (load), allzeros). Previously we were able to share the XOR with multiple VPTESTM instructions. llvm-svn: 358359
2019-04-15 02:26:11 +08:00
; AVX512BW-NEXT: vpsrlw $4, %zmm0, %zmm1
; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0]
; AVX512BW-NEXT: vpshufb %zmm1, %zmm2, %zmm3
; AVX512BW-NEXT: vpshufb %zmm0, %zmm2, %zmm0
; AVX512BW-NEXT: vptestnmb %zmm1, %zmm1, %k0
; AVX512BW-NEXT: vpmovm2b %k0, %zmm1
; AVX512BW-NEXT: vpandq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpaddb %zmm3, %zmm0, %zmm0
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
; AVX512BW-NEXT: retq
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
;
; AVX512DQ-LABEL: testv64i8:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512DQ: # %bb.0:
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm2 = [4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0]
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm2, %ymm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrlw $4, %ymm0, %ymm0
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm4 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512DQ-NEXT: vpand %ymm4, %ymm0, %ymm0
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX512DQ-NEXT: vpxor %xmm5, %xmm5, %xmm5
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpcmpeqb %ymm5, %ymm0, %ymm6
; AVX512DQ-NEXT: vpand %ymm6, %ymm3, %ymm3
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm2, %ymm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpaddb %ymm0, %ymm3, %ymm0
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm2, %ymm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrlw $4, %ymm1, %ymm1
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpand %ymm4, %ymm1, %ymm1
; AVX512DQ-NEXT: vpcmpeqb %ymm5, %ymm1, %ymm4
; AVX512DQ-NEXT: vpand %ymm4, %ymm3, %ymm3
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm2, %ymm1
; AVX512DQ-NEXT: vpaddb %ymm1, %ymm3, %ymm1
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: retq
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
%out = call <64 x i8> @llvm.ctlz.v64i8(<64 x i8> %in, i1 0)
ret <64 x i8> %out
}
define <64 x i8> @testv64i8u(<64 x i8> %in) nounwind {
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-LABEL: testv64i8u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CD: # %bb.0:
[AVX512] Add ExeDomain to vector extend and truncate instructions. llvm-svn: 276394
2016-07-22 13:46:44 +08:00
; AVX512CD-NEXT: vextracti128 $1, %ymm0, %xmm2
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxbd {{.*#+}} zmm2 = xmm2[0],zero,zero,zero,xmm2[1],zero,zero,zero,xmm2[2],zero,zero,zero,xmm2[3],zero,zero,zero,xmm2[4],zero,zero,zero,xmm2[5],zero,zero,zero,xmm2[6],zero,zero,zero,xmm2[7],zero,zero,zero,xmm2[8],zero,zero,zero,xmm2[9],zero,zero,zero,xmm2[10],zero,zero,zero,xmm2[11],zero,zero,zero,xmm2[12],zero,zero,zero,xmm2[13],zero,zero,zero,xmm2[14],zero,zero,zero,xmm2[15],zero,zero,zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm2, %zmm2
; AVX512CD-NEXT: vpmovdb %zmm2, %xmm2
; AVX512CD-NEXT: vmovdqa {{.*#+}} xmm3 = [24,24,24,24,24,24,24,24,24,24,24,24,24,24,24,24]
; AVX512CD-NEXT: vpsubb %xmm3, %xmm2, %xmm2
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxbd {{.*#+}} zmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero,xmm0[2],zero,zero,zero,xmm0[3],zero,zero,zero,xmm0[4],zero,zero,zero,xmm0[5],zero,zero,zero,xmm0[6],zero,zero,zero,xmm0[7],zero,zero,zero,xmm0[8],zero,zero,zero,xmm0[9],zero,zero,zero,xmm0[10],zero,zero,zero,xmm0[11],zero,zero,zero,xmm0[12],zero,zero,zero,xmm0[13],zero,zero,zero,xmm0[14],zero,zero,zero,xmm0[15],zero,zero,zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CD-NEXT: vpmovdb %zmm0, %xmm0
; AVX512CD-NEXT: vpsubb %xmm3, %xmm0, %xmm0
; AVX512CD-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
[AVX512] Add ExeDomain to vector extend and truncate instructions. llvm-svn: 276394
2016-07-22 13:46:44 +08:00
; AVX512CD-NEXT: vextracti128 $1, %ymm1, %xmm2
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxbd {{.*#+}} zmm2 = xmm2[0],zero,zero,zero,xmm2[1],zero,zero,zero,xmm2[2],zero,zero,zero,xmm2[3],zero,zero,zero,xmm2[4],zero,zero,zero,xmm2[5],zero,zero,zero,xmm2[6],zero,zero,zero,xmm2[7],zero,zero,zero,xmm2[8],zero,zero,zero,xmm2[9],zero,zero,zero,xmm2[10],zero,zero,zero,xmm2[11],zero,zero,zero,xmm2[12],zero,zero,zero,xmm2[13],zero,zero,zero,xmm2[14],zero,zero,zero,xmm2[15],zero,zero,zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm2, %zmm2
; AVX512CD-NEXT: vpmovdb %zmm2, %xmm2
; AVX512CD-NEXT: vpsubb %xmm3, %xmm2, %xmm2
[X86][AVX512] Added support for VPMOVZX shuffle decoding. llvm-svn: 260007
2016-02-07 03:51:21 +08:00
; AVX512CD-NEXT: vpmovzxbd {{.*#+}} zmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero,xmm1[2],zero,zero,zero,xmm1[3],zero,zero,zero,xmm1[4],zero,zero,zero,xmm1[5],zero,zero,zero,xmm1[6],zero,zero,zero,xmm1[7],zero,zero,zero,xmm1[8],zero,zero,zero,xmm1[9],zero,zero,zero,xmm1[10],zero,zero,zero,xmm1[11],zero,zero,zero,xmm1[12],zero,zero,zero,xmm1[13],zero,zero,zero,xmm1[14],zero,zero,zero,xmm1[15],zero,zero,zero
[X86][AVX512] Fixed prefix ordering for lzcnt tests. Let AVX512 targets share the same CHECKs. llvm-svn: 260000
2016-02-07 02:07:19 +08:00
; AVX512CD-NEXT: vplzcntd %zmm1, %zmm1
; AVX512CD-NEXT: vpmovdb %zmm1, %xmm1
; AVX512CD-NEXT: vpsubb %xmm3, %xmm1, %xmm1
; AVX512CD-NEXT: vinserti128 $1, %xmm2, %ymm1, %ymm1
; AVX512CD-NEXT: retq
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
;
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-LABEL: testv64i8u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512CDBW: # %bb.0:
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512CDBW-NEXT: vextracti64x4 $1, %zmm0, %ymm1
; AVX512CDBW-NEXT: vextracti128 $1, %ymm1, %xmm2
; AVX512CDBW-NEXT: vpmovzxbd {{.*#+}} zmm2 = xmm2[0],zero,zero,zero,xmm2[1],zero,zero,zero,xmm2[2],zero,zero,zero,xmm2[3],zero,zero,zero,xmm2[4],zero,zero,zero,xmm2[5],zero,zero,zero,xmm2[6],zero,zero,zero,xmm2[7],zero,zero,zero,xmm2[8],zero,zero,zero,xmm2[9],zero,zero,zero,xmm2[10],zero,zero,zero,xmm2[11],zero,zero,zero,xmm2[12],zero,zero,zero,xmm2[13],zero,zero,zero,xmm2[14],zero,zero,zero,xmm2[15],zero,zero,zero
; AVX512CDBW-NEXT: vplzcntd %zmm2, %zmm2
; AVX512CDBW-NEXT: vpmovdb %zmm2, %xmm2
; AVX512CDBW-NEXT: vmovdqa {{.*#+}} xmm3 = [24,24,24,24,24,24,24,24,24,24,24,24,24,24,24,24]
; AVX512CDBW-NEXT: vpsubb %xmm3, %xmm2, %xmm2
; AVX512CDBW-NEXT: vpmovzxbd {{.*#+}} zmm1 = xmm1[0],zero,zero,zero,xmm1[1],zero,zero,zero,xmm1[2],zero,zero,zero,xmm1[3],zero,zero,zero,xmm1[4],zero,zero,zero,xmm1[5],zero,zero,zero,xmm1[6],zero,zero,zero,xmm1[7],zero,zero,zero,xmm1[8],zero,zero,zero,xmm1[9],zero,zero,zero,xmm1[10],zero,zero,zero,xmm1[11],zero,zero,zero,xmm1[12],zero,zero,zero,xmm1[13],zero,zero,zero,xmm1[14],zero,zero,zero,xmm1[15],zero,zero,zero
; AVX512CDBW-NEXT: vplzcntd %zmm1, %zmm1
; AVX512CDBW-NEXT: vpmovdb %zmm1, %xmm1
; AVX512CDBW-NEXT: vpsubb %xmm3, %xmm1, %xmm1
; AVX512CDBW-NEXT: vinserti128 $1, %xmm2, %ymm1, %ymm1
; AVX512CDBW-NEXT: vextracti128 $1, %ymm0, %xmm2
; AVX512CDBW-NEXT: vpmovzxbd {{.*#+}} zmm2 = xmm2[0],zero,zero,zero,xmm2[1],zero,zero,zero,xmm2[2],zero,zero,zero,xmm2[3],zero,zero,zero,xmm2[4],zero,zero,zero,xmm2[5],zero,zero,zero,xmm2[6],zero,zero,zero,xmm2[7],zero,zero,zero,xmm2[8],zero,zero,zero,xmm2[9],zero,zero,zero,xmm2[10],zero,zero,zero,xmm2[11],zero,zero,zero,xmm2[12],zero,zero,zero,xmm2[13],zero,zero,zero,xmm2[14],zero,zero,zero,xmm2[15],zero,zero,zero
; AVX512CDBW-NEXT: vplzcntd %zmm2, %zmm2
; AVX512CDBW-NEXT: vpmovdb %zmm2, %xmm2
; AVX512CDBW-NEXT: vpsubb %xmm3, %xmm2, %xmm2
; AVX512CDBW-NEXT: vpmovzxbd {{.*#+}} zmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero,xmm0[2],zero,zero,zero,xmm0[3],zero,zero,zero,xmm0[4],zero,zero,zero,xmm0[5],zero,zero,zero,xmm0[6],zero,zero,zero,xmm0[7],zero,zero,zero,xmm0[8],zero,zero,zero,xmm0[9],zero,zero,zero,xmm0[10],zero,zero,zero,xmm0[11],zero,zero,zero,xmm0[12],zero,zero,zero,xmm0[13],zero,zero,zero,xmm0[14],zero,zero,zero,xmm0[15],zero,zero,zero
; AVX512CDBW-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CDBW-NEXT: vpmovdb %zmm0, %xmm0
; AVX512CDBW-NEXT: vpsubb %xmm3, %xmm0, %xmm0
; AVX512CDBW-NEXT: vinserti128 $1, %xmm2, %ymm0, %ymm0
; AVX512CDBW-NEXT: vinserti64x4 $1, %ymm1, %zmm0, %zmm0
; AVX512CDBW-NEXT: retq
;
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
; AVX512BW-LABEL: testv64i8u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512BW: # %bb.0:
[X86] Move VPTESTM matching from the isel table to custom code in X86ISelDAGToDAG. We had many tablegen patterns for these instructions. And due to the commutability of the patterns, tablegen expands them to even more patterns. All together VPTESTMD patterns accounted for more the 50K of the 610K isel table. This had gotten bad when we stopped canonicalizing AND to vXi64. This required a pattern for every combination of bitcast input type. This change moves the matching to custom code where it is easier to look through the bitcasts without being concerned with the specific types. The test changes are because we are now stricter with one use checks as its required to make load folding legal. We now require the AND and any BITCAST to only have a single use. This prevents forming VPTESTM and a VPAND with the same inputs. We now support broadcast loads for 128/256 patterns without VLX. We'll widen to 512-bit like and still fold the broadcast since the amount of memory read doesn't change. There are a few tests that got slightly longer because are now prefering load + VPTESTM over XOR+VPCMPEQ for (seteq (load), allzeros). Previously we were able to share the XOR with multiple VPTESTM instructions. llvm-svn: 358359
2019-04-15 02:26:11 +08:00
; AVX512BW-NEXT: vpsrlw $4, %zmm0, %zmm1
; AVX512BW-NEXT: vpandq {{.*}}(%rip), %zmm1, %zmm1
; AVX512BW-NEXT: vmovdqa64 {{.*#+}} zmm2 = [4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0]
; AVX512BW-NEXT: vpshufb %zmm1, %zmm2, %zmm3
; AVX512BW-NEXT: vpshufb %zmm0, %zmm2, %zmm0
; AVX512BW-NEXT: vptestnmb %zmm1, %zmm1, %k0
; AVX512BW-NEXT: vpmovm2b %k0, %zmm1
; AVX512BW-NEXT: vpandq %zmm1, %zmm0, %zmm0
; AVX512BW-NEXT: vpaddb %zmm3, %zmm0, %zmm0
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
; AVX512BW-NEXT: retq
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
;
; AVX512DQ-LABEL: testv64i8u:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512DQ: # %bb.0:
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm2 = [4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0,4,3,2,2,1,1,1,1,0,0,0,0,0,0,0,0]
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm2, %ymm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrlw $4, %ymm0, %ymm0
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vmovdqa {{.*#+}} ymm4 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512DQ-NEXT: vpand %ymm4, %ymm0, %ymm0
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX512DQ-NEXT: vpxor %xmm5, %xmm5, %xmm5
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpcmpeqb %ymm5, %ymm0, %ymm6
; AVX512DQ-NEXT: vpand %ymm6, %ymm3, %ymm3
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm0, %ymm2, %ymm0
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpaddb %ymm0, %ymm3, %ymm0
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm2, %ymm3
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: vpsrlw $4, %ymm1, %ymm1
[X86][SSE] Remove unnecessary bit-and in pshufb vector ctlz (PR39703) SSE PSHUFB vector ctlz lowering works at the i4 nibble level. As detailed in PR39703, we were masking the lower nibble off but we only actually use it in the case where the upper nibble is known to be zero, making it safe to remove the mask and save an instruction. Differential Revision: https://reviews.llvm.org/D54707 llvm-svn: 347242
2018-11-20 02:40:59 +08:00
; AVX512DQ-NEXT: vpand %ymm4, %ymm1, %ymm1
; AVX512DQ-NEXT: vpcmpeqb %ymm5, %ymm1, %ymm4
; AVX512DQ-NEXT: vpand %ymm4, %ymm3, %ymm3
; AVX512DQ-NEXT: vpshufb %ymm1, %ymm2, %ymm1
; AVX512DQ-NEXT: vpaddb %ymm1, %ymm3, %ymm1
[X86][AVX512] Improve support and testing for CTLZ of 512-bit vectors without CDI llvm-svn: 302233
2017-05-05 21:31:52 +08:00
; AVX512DQ-NEXT: retq
[X86] Updated vector lzcnt tests. Added missing vec512 tests. llvm-svn: 246927
2015-09-05 19:56:30 +08:00
%out = call <64 x i8> @llvm.ctlz.v64i8(<64 x i8> %in, i1 -1)
ret <64 x i8> %out
}
declare <8 x i64> @llvm.ctlz.v8i64(<8 x i64>, i1)
declare <16 x i32> @llvm.ctlz.v16i32(<16 x i32>, i1)
declare <32 x i16> @llvm.ctlz.v32i16(<32 x i16>, i1)
declare <64 x i8> @llvm.ctlz.v64i8(<64 x i8>, i1)