maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/vector-tzcnt-128.ll

1569 lines
60 KiB
LLVM
Raw Normal View History

[X86] Updated test checks script to generalise LCPI symbol refs The script now replace '.LCPI888_8' style asm symbols with the {{\.LCPI.*}} re pattern - this helps stop hardcoded symbols in 32-bit x86 tests changing with every edit of the file Refreshed some tests to demonstrate the new check llvm-svn: 272488
2016-06-12 04:39:21 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[X86] Updated vector tzcnt tests. Added vec512 tests. llvm-svn: 246922
2015-09-05 18:19:07 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE2
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse3 | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE3
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+ssse3 | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSSE3
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse4.1 | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE41
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX1
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx2 | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX2
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512cd,+avx512vl | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX512CDVL
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512cd,-avx512vl | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX512CD
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512vpopcntdq | FileCheck %s --check-prefix=ALL --check-prefix=AVX512VPOPCNTDQ
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
;
; Just one 32-bit run to make sure we do reasonable things for i64 tzcnt.
; RUN: llc < %s -mtriple=i686-unknown-unknown -mattr=+sse4.1 | FileCheck %s --check-prefix=ALL --check-prefix=X32-SSE --check-prefix=X32-SSE41
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <2 x i64> @testv2i64(<2 x i64> %in) nounwind {
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-LABEL: testv2i64:
; SSE2: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; SSE2-NEXT: pxor %xmm1, %xmm1
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: psubq %xmm0, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: psubq {{.*}}(%rip), %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psrlq $1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psubq %xmm0, %xmm2
; SSE2-NEXT: movdqa {{.*#+}} xmm0 = [3689348814741910323,3689348814741910323]
; SSE2-NEXT: movdqa %xmm2, %xmm3
; SSE2-NEXT: pand %xmm0, %xmm3
; SSE2-NEXT: psrlq $2, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: paddq %xmm3, %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psrlq $4, %xmm0
; SSE2-NEXT: paddq %xmm2, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psadbw %xmm1, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-NEXT: retq
;
; SSE3-LABEL: testv2i64:
; SSE3: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; SSE3-NEXT: pxor %xmm1, %xmm1
; SSE3-NEXT: pxor %xmm2, %xmm2
; SSE3-NEXT: psubq %xmm0, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: psubq {{.*}}(%rip), %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psrlq $1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psubq %xmm0, %xmm2
; SSE3-NEXT: movdqa {{.*#+}} xmm0 = [3689348814741910323,3689348814741910323]
; SSE3-NEXT: movdqa %xmm2, %xmm3
; SSE3-NEXT: pand %xmm0, %xmm3
; SSE3-NEXT: psrlq $2, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: paddq %xmm3, %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psrlq $4, %xmm0
; SSE3-NEXT: paddq %xmm2, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psadbw %xmm1, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE3-NEXT: retq
;
; SSSE3-LABEL: testv2i64:
; SSSE3: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; SSSE3-NEXT: pxor %xmm1, %xmm1
; SSSE3-NEXT: pxor %xmm2, %xmm2
; SSSE3-NEXT: psubq %xmm0, %xmm2
; SSSE3-NEXT: pand %xmm0, %xmm2
; SSSE3-NEXT: psubq {{.*}}(%rip), %xmm2
; SSSE3-NEXT: movdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSSE3-NEXT: movdqa %xmm2, %xmm4
; SSSE3-NEXT: pand %xmm3, %xmm4
; SSSE3-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSSE3-NEXT: movdqa %xmm0, %xmm5
; SSSE3-NEXT: pshufb %xmm4, %xmm5
; SSSE3-NEXT: psrlw $4, %xmm2
; SSSE3-NEXT: pand %xmm3, %xmm2
; SSSE3-NEXT: pshufb %xmm2, %xmm0
; SSSE3-NEXT: paddb %xmm5, %xmm0
; SSSE3-NEXT: psadbw %xmm1, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSSE3-NEXT: retq
;
; SSE41-LABEL: testv2i64:
; SSE41: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; SSE41-NEXT: pxor %xmm1, %xmm1
; SSE41-NEXT: pxor %xmm2, %xmm2
; SSE41-NEXT: psubq %xmm0, %xmm2
; SSE41-NEXT: pand %xmm0, %xmm2
; SSE41-NEXT: psubq {{.*}}(%rip), %xmm2
; SSE41-NEXT: movdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSE41-NEXT: movdqa %xmm2, %xmm4
; SSE41-NEXT: pand %xmm3, %xmm4
; SSE41-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSE41-NEXT: movdqa %xmm0, %xmm5
; SSE41-NEXT: pshufb %xmm4, %xmm5
; SSE41-NEXT: psrlw $4, %xmm2
; SSE41-NEXT: pand %xmm3, %xmm2
; SSE41-NEXT: pshufb %xmm2, %xmm0
; SSE41-NEXT: paddb %xmm5, %xmm0
; SSE41-NEXT: psadbw %xmm1, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE41-NEXT: retq
;
[X86][AVX] Regenerate tzcnt tests. The EVEX -> VEX fix means that AVX/AVX512 code is more likely the same now. llvm-svn: 291241
2017-01-06 23:54:23 +08:00
; AVX-LABEL: testv2i64:
; AVX: # BB#0:
; AVX-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX-NEXT: vpsubq %xmm0, %xmm1, %xmm2
; AVX-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX-NEXT: vpsubq {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: testv2i64:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubq %xmm0, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsubq {{.*}}(%rip), %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpopcntq %zmm0, %zmm0
; AVX512VPOPCNTDQ-NEXT: # kill: %XMM0<def> %XMM0<kill> %ZMM0<kill>
; AVX512VPOPCNTDQ-NEXT: vzeroupper
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: testv2i64:
; X32-SSE: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; X32-SSE-NEXT: pxor %xmm1, %xmm1
; X32-SSE-NEXT: pxor %xmm2, %xmm2
; X32-SSE-NEXT: psubq %xmm0, %xmm2
; X32-SSE-NEXT: pand %xmm0, %xmm2
; X32-SSE-NEXT: psubq {{\.LCPI.*}}, %xmm2
; X32-SSE-NEXT: movdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; X32-SSE-NEXT: movdqa %xmm2, %xmm4
; X32-SSE-NEXT: pand %xmm3, %xmm4
; X32-SSE-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; X32-SSE-NEXT: movdqa %xmm0, %xmm5
; X32-SSE-NEXT: pshufb %xmm4, %xmm5
; X32-SSE-NEXT: psrlw $4, %xmm2
; X32-SSE-NEXT: pand %xmm3, %xmm2
; X32-SSE-NEXT: pshufb %xmm2, %xmm0
; X32-SSE-NEXT: paddb %xmm5, %xmm0
; X32-SSE-NEXT: psadbw %xmm1, %xmm0
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: retl
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
%out = call <2 x i64> @llvm.cttz.v2i64(<2 x i64> %in, i1 0)
ret <2 x i64> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <2 x i64> @testv2i64u(<2 x i64> %in) nounwind {
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-LABEL: testv2i64u:
; SSE2: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; SSE2-NEXT: pxor %xmm1, %xmm1
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: psubq %xmm0, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: psubq {{.*}}(%rip), %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psrlq $1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psubq %xmm0, %xmm2
; SSE2-NEXT: movdqa {{.*#+}} xmm0 = [3689348814741910323,3689348814741910323]
; SSE2-NEXT: movdqa %xmm2, %xmm3
; SSE2-NEXT: pand %xmm0, %xmm3
; SSE2-NEXT: psrlq $2, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: paddq %xmm3, %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psrlq $4, %xmm0
; SSE2-NEXT: paddq %xmm2, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psadbw %xmm1, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-NEXT: retq
;
; SSE3-LABEL: testv2i64u:
; SSE3: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; SSE3-NEXT: pxor %xmm1, %xmm1
; SSE3-NEXT: pxor %xmm2, %xmm2
; SSE3-NEXT: psubq %xmm0, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: psubq {{.*}}(%rip), %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psrlq $1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psubq %xmm0, %xmm2
; SSE3-NEXT: movdqa {{.*#+}} xmm0 = [3689348814741910323,3689348814741910323]
; SSE3-NEXT: movdqa %xmm2, %xmm3
; SSE3-NEXT: pand %xmm0, %xmm3
; SSE3-NEXT: psrlq $2, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: paddq %xmm3, %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psrlq $4, %xmm0
; SSE3-NEXT: paddq %xmm2, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psadbw %xmm1, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE3-NEXT: retq
;
; SSSE3-LABEL: testv2i64u:
; SSSE3: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; SSSE3-NEXT: pxor %xmm1, %xmm1
; SSSE3-NEXT: pxor %xmm2, %xmm2
; SSSE3-NEXT: psubq %xmm0, %xmm2
; SSSE3-NEXT: pand %xmm0, %xmm2
; SSSE3-NEXT: psubq {{.*}}(%rip), %xmm2
; SSSE3-NEXT: movdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSSE3-NEXT: movdqa %xmm2, %xmm4
; SSSE3-NEXT: pand %xmm3, %xmm4
; SSSE3-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSSE3-NEXT: movdqa %xmm0, %xmm5
; SSSE3-NEXT: pshufb %xmm4, %xmm5
; SSSE3-NEXT: psrlw $4, %xmm2
; SSSE3-NEXT: pand %xmm3, %xmm2
; SSSE3-NEXT: pshufb %xmm2, %xmm0
; SSSE3-NEXT: paddb %xmm5, %xmm0
; SSSE3-NEXT: psadbw %xmm1, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSSE3-NEXT: retq
;
; SSE41-LABEL: testv2i64u:
; SSE41: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; SSE41-NEXT: pxor %xmm1, %xmm1
; SSE41-NEXT: pxor %xmm2, %xmm2
; SSE41-NEXT: psubq %xmm0, %xmm2
; SSE41-NEXT: pand %xmm0, %xmm2
; SSE41-NEXT: psubq {{.*}}(%rip), %xmm2
; SSE41-NEXT: movdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSE41-NEXT: movdqa %xmm2, %xmm4
; SSE41-NEXT: pand %xmm3, %xmm4
; SSE41-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSE41-NEXT: movdqa %xmm0, %xmm5
; SSE41-NEXT: pshufb %xmm4, %xmm5
; SSE41-NEXT: psrlw $4, %xmm2
; SSE41-NEXT: pand %xmm3, %xmm2
; SSE41-NEXT: pshufb %xmm2, %xmm0
; SSE41-NEXT: paddb %xmm5, %xmm0
; SSE41-NEXT: psadbw %xmm1, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE41-NEXT: retq
;
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX1-LABEL: testv2i64u:
; AVX1: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; AVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX1-NEXT: vpsubq %xmm0, %xmm1, %xmm2
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpsubq {{.*}}(%rip), %xmm0, %xmm0
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX1-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX1-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX1-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX1-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: testv2i64u:
; AVX2: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; AVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX2-NEXT: vpsubq %xmm0, %xmm1, %xmm2
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vpsubq {{.*}}(%rip), %xmm0, %xmm0
; AVX2-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX2-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX2-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX2-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX2-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX2-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX2-NEXT: retq
;
; AVX512CDVL-LABEL: testv2i64u:
; AVX512CDVL: # BB#0:
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vpxor %xmm1, %xmm1, %xmm1
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vpsubq %xmm0, %xmm1, %xmm1
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vpand %xmm1, %xmm0, %xmm0
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vplzcntq %xmm0, %xmm0
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vmovdqa {{.*#+}} xmm1 = [63,63]
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vpsubq %xmm0, %xmm1, %xmm0
; AVX512CDVL-NEXT: retq
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
;
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
; AVX512CD-LABEL: testv2i64u:
; AVX512CD: # BB#0:
; AVX512CD-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512CD-NEXT: vpsubq %xmm0, %xmm1, %xmm1
; AVX512CD-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512CD-NEXT: vplzcntq %zmm0, %zmm0
; AVX512CD-NEXT: vmovdqa {{.*#+}} xmm1 = [63,63]
; AVX512CD-NEXT: vpsubq %xmm0, %xmm1, %xmm0
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512CD-NEXT: vzeroupper
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
; AVX512CD-NEXT: retq
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: testv2i64u:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubq %xmm0, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsubq {{.*}}(%rip), %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpopcntq %zmm0, %zmm0
; AVX512VPOPCNTDQ-NEXT: # kill: %XMM0<def> %XMM0<kill> %ZMM0<kill>
; AVX512VPOPCNTDQ-NEXT: vzeroupper
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: testv2i64u:
; X32-SSE: # BB#0:
[X86][SSE] Don't decide when to scalarize CTTZ/CTLZ for performance at lowering - this is what cost models are for Improved CTTZ/CTLZ costings will be added shortly llvm-svn: 277713
2016-08-04 18:14:39 +08:00
; X32-SSE-NEXT: pxor %xmm1, %xmm1
; X32-SSE-NEXT: pxor %xmm2, %xmm2
; X32-SSE-NEXT: psubq %xmm0, %xmm2
; X32-SSE-NEXT: pand %xmm0, %xmm2
; X32-SSE-NEXT: psubq {{\.LCPI.*}}, %xmm2
; X32-SSE-NEXT: movdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; X32-SSE-NEXT: movdqa %xmm2, %xmm4
; X32-SSE-NEXT: pand %xmm3, %xmm4
; X32-SSE-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; X32-SSE-NEXT: movdqa %xmm0, %xmm5
; X32-SSE-NEXT: pshufb %xmm4, %xmm5
; X32-SSE-NEXT: psrlw $4, %xmm2
; X32-SSE-NEXT: pand %xmm3, %xmm2
; X32-SSE-NEXT: pshufb %xmm2, %xmm0
; X32-SSE-NEXT: paddb %xmm5, %xmm0
; X32-SSE-NEXT: psadbw %xmm1, %xmm0
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: retl
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
%out = call <2 x i64> @llvm.cttz.v2i64(<2 x i64> %in, i1 -1)
ret <2 x i64> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <4 x i32> @testv4i32(<4 x i32> %in) nounwind {
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-LABEL: testv4i32:
; SSE2: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE2-NEXT: pxor %xmm1, %xmm1
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: psubd %xmm0, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: psubd {{.*}}(%rip), %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psrld $1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psubd %xmm0, %xmm2
; SSE2-NEXT: movdqa {{.*#+}} xmm0 = [858993459,858993459,858993459,858993459]
; SSE2-NEXT: movdqa %xmm2, %xmm3
; SSE2-NEXT: pand %xmm0, %xmm3
; SSE2-NEXT: psrld $2, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: paddd %xmm3, %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psrld $4, %xmm0
; SSE2-NEXT: paddd %xmm2, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: movdqa %xmm0, %xmm2
; SSE2-NEXT: punpckhdq {{.*#+}} xmm2 = xmm2[2],xmm1[2],xmm2[3],xmm1[3]
; SSE2-NEXT: psadbw %xmm1, %xmm2
; SSE2-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
; SSE2-NEXT: psadbw %xmm1, %xmm0
; SSE2-NEXT: packuswb %xmm2, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-NEXT: retq
;
; SSE3-LABEL: testv4i32:
; SSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE3-NEXT: pxor %xmm1, %xmm1
; SSE3-NEXT: pxor %xmm2, %xmm2
; SSE3-NEXT: psubd %xmm0, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: psubd {{.*}}(%rip), %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psrld $1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psubd %xmm0, %xmm2
; SSE3-NEXT: movdqa {{.*#+}} xmm0 = [858993459,858993459,858993459,858993459]
; SSE3-NEXT: movdqa %xmm2, %xmm3
; SSE3-NEXT: pand %xmm0, %xmm3
; SSE3-NEXT: psrld $2, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: paddd %xmm3, %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psrld $4, %xmm0
; SSE3-NEXT: paddd %xmm2, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: movdqa %xmm0, %xmm2
; SSE3-NEXT: punpckhdq {{.*#+}} xmm2 = xmm2[2],xmm1[2],xmm2[3],xmm1[3]
; SSE3-NEXT: psadbw %xmm1, %xmm2
; SSE3-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
; SSE3-NEXT: psadbw %xmm1, %xmm0
; SSE3-NEXT: packuswb %xmm2, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE3-NEXT: retq
;
; SSSE3-LABEL: testv4i32:
; SSSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSSE3-NEXT: pxor %xmm1, %xmm1
; SSSE3-NEXT: pxor %xmm2, %xmm2
; SSSE3-NEXT: psubd %xmm0, %xmm2
; SSSE3-NEXT: pand %xmm0, %xmm2
; SSSE3-NEXT: psubd {{.*}}(%rip), %xmm2
; SSSE3-NEXT: movdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSSE3-NEXT: movdqa %xmm2, %xmm4
; SSSE3-NEXT: pand %xmm3, %xmm4
; SSSE3-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSSE3-NEXT: movdqa %xmm0, %xmm5
; SSSE3-NEXT: pshufb %xmm4, %xmm5
; SSSE3-NEXT: psrlw $4, %xmm2
; SSSE3-NEXT: pand %xmm3, %xmm2
; SSSE3-NEXT: pshufb %xmm2, %xmm0
; SSSE3-NEXT: paddb %xmm5, %xmm0
; SSSE3-NEXT: movdqa %xmm0, %xmm2
; SSSE3-NEXT: punpckhdq {{.*#+}} xmm2 = xmm2[2],xmm1[2],xmm2[3],xmm1[3]
; SSSE3-NEXT: psadbw %xmm1, %xmm2
; SSSE3-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
; SSSE3-NEXT: psadbw %xmm1, %xmm0
; SSSE3-NEXT: packuswb %xmm2, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSSE3-NEXT: retq
;
; SSE41-LABEL: testv4i32:
; SSE41: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: pxor %xmm1, %xmm1
; SSE41-NEXT: pxor %xmm2, %xmm2
; SSE41-NEXT: psubd %xmm0, %xmm2
; SSE41-NEXT: pand %xmm0, %xmm2
; SSE41-NEXT: psubd {{.*}}(%rip), %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; SSE41-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSE41-NEXT: movdqa %xmm2, %xmm3
; SSE41-NEXT: pand %xmm0, %xmm3
; SSE41-NEXT: movdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSE41-NEXT: movdqa %xmm4, %xmm5
; SSE41-NEXT: pshufb %xmm3, %xmm5
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: psrlw $4, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; SSE41-NEXT: pand %xmm0, %xmm2
; SSE41-NEXT: pshufb %xmm2, %xmm4
; SSE41-NEXT: paddb %xmm5, %xmm4
; SSE41-NEXT: pmovzxdq {{.*#+}} xmm0 = xmm4[0],zero,xmm4[1],zero
; SSE41-NEXT: punpckhdq {{.*#+}} xmm4 = xmm4[2],xmm1[2],xmm4[3],xmm1[3]
; SSE41-NEXT: psadbw %xmm1, %xmm4
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: psadbw %xmm1, %xmm0
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; SSE41-NEXT: packuswb %xmm4, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE41-NEXT: retq
;
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; AVX1-LABEL: testv4i32:
; AVX1: # BB#0:
; AVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX1-NEXT: vpsubd %xmm0, %xmm1, %xmm2
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpsubd {{.*}}(%rip), %xmm0, %xmm0
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX1-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX1-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX1-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX1-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpunpckhdq {{.*#+}} xmm2 = xmm0[2],xmm1[2],xmm0[3],xmm1[3]
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
; AVX1-NEXT: vpsadbw %xmm1, %xmm2, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; AVX1-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
; AVX1-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; AVX1-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: testv4i32:
; AVX2: # BB#0:
; AVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX2-NEXT: vpsubd %xmm0, %xmm1, %xmm2
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vpbroadcastd {{.*}}(%rip), %xmm2
; AVX2-NEXT: vpsubd %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX2-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX2-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX2-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX2-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX2-NEXT: vpunpckhdq {{.*#+}} xmm2 = xmm0[2],xmm1[2],xmm0[3],xmm1[3]
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
; AVX2-NEXT: vpsadbw %xmm1, %xmm2, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; AVX2-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
; AVX2-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; AVX2-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; AVX2-NEXT: retq
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
;
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-LABEL: testv4i32:
; AVX512CDVL: # BB#0:
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vpxor %xmm1, %xmm1, %xmm1
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vpsubd %xmm0, %xmm1, %xmm2
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vpand %xmm2, %xmm0, %xmm0
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vpsubd {{.*}}(%rip){1to4}, %xmm0, %xmm0
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512CDVL-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX512CDVL-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX512CDVL-NEXT: vpsrlw $4, %xmm0, %xmm0
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vpand %xmm2, %xmm0, %xmm0
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX512CDVL-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX512CDVL-NEXT: vpunpckhdq {{.*#+}} xmm2 = xmm0[2],xmm1[2],xmm0[3],xmm1[3]
; AVX512CDVL-NEXT: vpsadbw %xmm1, %xmm2, %xmm2
; AVX512CDVL-NEXT: vpunpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
; AVX512CDVL-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
; AVX512CDVL-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; AVX512CDVL-NEXT: retq
;
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
; AVX512CD-LABEL: testv4i32:
; AVX512CD: # BB#0:
; AVX512CD-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512CD-NEXT: vpsubd %xmm0, %xmm1, %xmm2
; AVX512CD-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX512CD-NEXT: vpbroadcastd {{.*}}(%rip), %xmm2
; AVX512CD-NEXT: vpsubd %xmm2, %xmm0, %xmm0
; AVX512CD-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512CD-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX512CD-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512CD-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX512CD-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX512CD-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX512CD-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX512CD-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX512CD-NEXT: vpunpckhdq {{.*#+}} xmm2 = xmm0[2],xmm1[2],xmm0[3],xmm1[3]
; AVX512CD-NEXT: vpsadbw %xmm1, %xmm2, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; AVX512CD-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
; AVX512CD-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
; AVX512CD-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; AVX512CD-NEXT: retq
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: testv4i32:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubd %xmm0, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpbroadcastd {{.*}}(%rip), %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubd %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpopcntd %zmm0, %zmm0
; AVX512VPOPCNTDQ-NEXT: # kill: %XMM0<def> %XMM0<kill> %ZMM0<kill>
; AVX512VPOPCNTDQ-NEXT: vzeroupper
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: testv4i32:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: pxor %xmm1, %xmm1
; X32-SSE-NEXT: pxor %xmm2, %xmm2
; X32-SSE-NEXT: psubd %xmm0, %xmm2
; X32-SSE-NEXT: pand %xmm0, %xmm2
[X86] Updated test checks script to generalise LCPI symbol refs The script now replace '.LCPI888_8' style asm symbols with the {{\.LCPI.*}} re pattern - this helps stop hardcoded symbols in 32-bit x86 tests changing with every edit of the file Refreshed some tests to demonstrate the new check llvm-svn: 272488
2016-06-12 04:39:21 +08:00
; X32-SSE-NEXT: psubd {{\.LCPI.*}}, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; X32-SSE-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; X32-SSE-NEXT: movdqa %xmm2, %xmm3
; X32-SSE-NEXT: pand %xmm0, %xmm3
; X32-SSE-NEXT: movdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; X32-SSE-NEXT: movdqa %xmm4, %xmm5
; X32-SSE-NEXT: pshufb %xmm3, %xmm5
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: psrlw $4, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; X32-SSE-NEXT: pand %xmm0, %xmm2
; X32-SSE-NEXT: pshufb %xmm2, %xmm4
; X32-SSE-NEXT: paddb %xmm5, %xmm4
; X32-SSE-NEXT: pmovzxdq {{.*#+}} xmm0 = xmm4[0],zero,xmm4[1],zero
; X32-SSE-NEXT: punpckhdq {{.*#+}} xmm4 = xmm4[2],xmm1[2],xmm4[3],xmm1[3]
; X32-SSE-NEXT: psadbw %xmm1, %xmm4
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: psadbw %xmm1, %xmm0
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; X32-SSE-NEXT: packuswb %xmm4, %xmm0
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: retl
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
%out = call <4 x i32> @llvm.cttz.v4i32(<4 x i32> %in, i1 0)
ret <4 x i32> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <4 x i32> @testv4i32u(<4 x i32> %in) nounwind {
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-LABEL: testv4i32u:
; SSE2: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE2-NEXT: pxor %xmm1, %xmm1
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: psubd %xmm0, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: psubd {{.*}}(%rip), %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psrld $1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psubd %xmm0, %xmm2
; SSE2-NEXT: movdqa {{.*#+}} xmm0 = [858993459,858993459,858993459,858993459]
; SSE2-NEXT: movdqa %xmm2, %xmm3
; SSE2-NEXT: pand %xmm0, %xmm3
; SSE2-NEXT: psrld $2, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: paddd %xmm3, %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psrld $4, %xmm0
; SSE2-NEXT: paddd %xmm2, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: movdqa %xmm0, %xmm2
; SSE2-NEXT: punpckhdq {{.*#+}} xmm2 = xmm2[2],xmm1[2],xmm2[3],xmm1[3]
; SSE2-NEXT: psadbw %xmm1, %xmm2
; SSE2-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
; SSE2-NEXT: psadbw %xmm1, %xmm0
; SSE2-NEXT: packuswb %xmm2, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-NEXT: retq
;
; SSE3-LABEL: testv4i32u:
; SSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE3-NEXT: pxor %xmm1, %xmm1
; SSE3-NEXT: pxor %xmm2, %xmm2
; SSE3-NEXT: psubd %xmm0, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: psubd {{.*}}(%rip), %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psrld $1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psubd %xmm0, %xmm2
; SSE3-NEXT: movdqa {{.*#+}} xmm0 = [858993459,858993459,858993459,858993459]
; SSE3-NEXT: movdqa %xmm2, %xmm3
; SSE3-NEXT: pand %xmm0, %xmm3
; SSE3-NEXT: psrld $2, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: paddd %xmm3, %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psrld $4, %xmm0
; SSE3-NEXT: paddd %xmm2, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: movdqa %xmm0, %xmm2
; SSE3-NEXT: punpckhdq {{.*#+}} xmm2 = xmm2[2],xmm1[2],xmm2[3],xmm1[3]
; SSE3-NEXT: psadbw %xmm1, %xmm2
; SSE3-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
; SSE3-NEXT: psadbw %xmm1, %xmm0
; SSE3-NEXT: packuswb %xmm2, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE3-NEXT: retq
;
; SSSE3-LABEL: testv4i32u:
; SSSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSSE3-NEXT: pxor %xmm1, %xmm1
; SSSE3-NEXT: pxor %xmm2, %xmm2
; SSSE3-NEXT: psubd %xmm0, %xmm2
; SSSE3-NEXT: pand %xmm0, %xmm2
; SSSE3-NEXT: psubd {{.*}}(%rip), %xmm2
; SSSE3-NEXT: movdqa {{.*#+}} xmm3 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSSE3-NEXT: movdqa %xmm2, %xmm4
; SSSE3-NEXT: pand %xmm3, %xmm4
; SSSE3-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSSE3-NEXT: movdqa %xmm0, %xmm5
; SSSE3-NEXT: pshufb %xmm4, %xmm5
; SSSE3-NEXT: psrlw $4, %xmm2
; SSSE3-NEXT: pand %xmm3, %xmm2
; SSSE3-NEXT: pshufb %xmm2, %xmm0
; SSSE3-NEXT: paddb %xmm5, %xmm0
; SSSE3-NEXT: movdqa %xmm0, %xmm2
; SSSE3-NEXT: punpckhdq {{.*#+}} xmm2 = xmm2[2],xmm1[2],xmm2[3],xmm1[3]
; SSSE3-NEXT: psadbw %xmm1, %xmm2
; SSSE3-NEXT: punpckldq {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
; SSSE3-NEXT: psadbw %xmm1, %xmm0
; SSSE3-NEXT: packuswb %xmm2, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSSE3-NEXT: retq
;
; SSE41-LABEL: testv4i32u:
; SSE41: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: pxor %xmm1, %xmm1
; SSE41-NEXT: pxor %xmm2, %xmm2
; SSE41-NEXT: psubd %xmm0, %xmm2
; SSE41-NEXT: pand %xmm0, %xmm2
; SSE41-NEXT: psubd {{.*}}(%rip), %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; SSE41-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSE41-NEXT: movdqa %xmm2, %xmm3
; SSE41-NEXT: pand %xmm0, %xmm3
; SSE41-NEXT: movdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSE41-NEXT: movdqa %xmm4, %xmm5
; SSE41-NEXT: pshufb %xmm3, %xmm5
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: psrlw $4, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; SSE41-NEXT: pand %xmm0, %xmm2
; SSE41-NEXT: pshufb %xmm2, %xmm4
; SSE41-NEXT: paddb %xmm5, %xmm4
; SSE41-NEXT: pmovzxdq {{.*#+}} xmm0 = xmm4[0],zero,xmm4[1],zero
; SSE41-NEXT: punpckhdq {{.*#+}} xmm4 = xmm4[2],xmm1[2],xmm4[3],xmm1[3]
; SSE41-NEXT: psadbw %xmm1, %xmm4
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: psadbw %xmm1, %xmm0
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; SSE41-NEXT: packuswb %xmm4, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE41-NEXT: retq
;
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; AVX1-LABEL: testv4i32u:
; AVX1: # BB#0:
; AVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX1-NEXT: vpsubd %xmm0, %xmm1, %xmm2
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpsubd {{.*}}(%rip), %xmm0, %xmm0
; AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX1-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX1-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX1-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX1-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX1-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vpunpckhdq {{.*#+}} xmm2 = xmm0[2],xmm1[2],xmm0[3],xmm1[3]
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
; AVX1-NEXT: vpsadbw %xmm1, %xmm2, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; AVX1-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
; AVX1-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; AVX1-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: testv4i32u:
; AVX2: # BB#0:
; AVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX2-NEXT: vpsubd %xmm0, %xmm1, %xmm2
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vpbroadcastd {{.*}}(%rip), %xmm2
; AVX2-NEXT: vpsubd %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vmovdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm3
; AVX2-NEXT: vmovdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX2-NEXT: vpshufb %xmm3, %xmm4, %xmm3
; AVX2-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX2-NEXT: vpand %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vpshufb %xmm0, %xmm4, %xmm0
; AVX2-NEXT: vpaddb %xmm3, %xmm0, %xmm0
; AVX2-NEXT: vpunpckhdq {{.*#+}} xmm2 = xmm0[2],xmm1[2],xmm0[3],xmm1[3]
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
; AVX2-NEXT: vpsadbw %xmm1, %xmm2, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; AVX2-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
; AVX2-NEXT: vpsadbw %xmm1, %xmm0, %xmm0
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; AVX2-NEXT: vpackuswb %xmm2, %xmm0, %xmm0
; AVX2-NEXT: retq
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
;
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-LABEL: testv4i32u:
; AVX512CDVL: # BB#0:
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vpxor %xmm1, %xmm1, %xmm1
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vpsubd %xmm0, %xmm1, %xmm1
This is a large patch for X86 AVX-512 of an optimization for reducing code size by encoding EVEX AVX-512 instructions using the shorter VEX encoding when possible. There are cases of AVX-512 instructions that have two possible encodings. This is the case with instructions that use vector registers with low indexes of 0 - 15 and do not use the zmm registers or the mask k registers. The EVEX encoding prefix requires 4 bytes whereas the VEX prefix can take only up to 3 bytes. Consequently, using the VEX encoding for these instructions results in a code size reduction of ~2 bytes even though it is compiled with the AVX-512 features enabled. Reviewers: Craig Topper, Zvi Rackoover, Elena Demikhovsky Differential Revision: https://reviews.llvm.org/D27901 llvm-svn: 290663
2016-12-28 18:12:48 +08:00
; AVX512CDVL-NEXT: vpand %xmm1, %xmm0, %xmm0
[AVX512] Add avx512cd+vl runs to vector-tzcnt-128/256 tests to show using the vplzcntd/q instructions. llvm-svn: 266860
2016-04-20 13:19:01 +08:00
; AVX512CDVL-NEXT: vplzcntd %xmm0, %xmm0
; AVX512CDVL-NEXT: vpbroadcastd {{.*}}(%rip), %xmm1
; AVX512CDVL-NEXT: vpsubd %xmm0, %xmm1, %xmm0
; AVX512CDVL-NEXT: retq
;
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
; AVX512CD-LABEL: testv4i32u:
; AVX512CD: # BB#0:
; AVX512CD-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512CD-NEXT: vpsubd %xmm0, %xmm1, %xmm1
; AVX512CD-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512CD-NEXT: vplzcntd %zmm0, %zmm0
; AVX512CD-NEXT: vpbroadcastd {{.*}}(%rip), %xmm1
; AVX512CD-NEXT: vpsubd %xmm0, %xmm1, %xmm0
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512CD-NEXT: vzeroupper
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
; AVX512CD-NEXT: retq
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: testv4i32u:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubd %xmm0, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpbroadcastd {{.*}}(%rip), %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubd %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpopcntd %zmm0, %zmm0
; AVX512VPOPCNTDQ-NEXT: # kill: %XMM0<def> %XMM0<kill> %ZMM0<kill>
; AVX512VPOPCNTDQ-NEXT: vzeroupper
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: testv4i32u:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: pxor %xmm1, %xmm1
; X32-SSE-NEXT: pxor %xmm2, %xmm2
; X32-SSE-NEXT: psubd %xmm0, %xmm2
; X32-SSE-NEXT: pand %xmm0, %xmm2
[X86] Updated test checks script to generalise LCPI symbol refs The script now replace '.LCPI888_8' style asm symbols with the {{\.LCPI.*}} re pattern - this helps stop hardcoded symbols in 32-bit x86 tests changing with every edit of the file Refreshed some tests to demonstrate the new check llvm-svn: 272488
2016-06-12 04:39:21 +08:00
; X32-SSE-NEXT: psubd {{\.LCPI.*}}, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; X32-SSE-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; X32-SSE-NEXT: movdqa %xmm2, %xmm3
; X32-SSE-NEXT: pand %xmm0, %xmm3
; X32-SSE-NEXT: movdqa {{.*#+}} xmm4 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; X32-SSE-NEXT: movdqa %xmm4, %xmm5
; X32-SSE-NEXT: pshufb %xmm3, %xmm5
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: psrlw $4, %xmm2
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; X32-SSE-NEXT: pand %xmm0, %xmm2
; X32-SSE-NEXT: pshufb %xmm2, %xmm4
; X32-SSE-NEXT: paddb %xmm5, %xmm4
; X32-SSE-NEXT: pmovzxdq {{.*#+}} xmm0 = xmm4[0],zero,xmm4[1],zero
; X32-SSE-NEXT: punpckhdq {{.*#+}} xmm4 = xmm4[2],xmm1[2],xmm4[3],xmm1[3]
; X32-SSE-NEXT: psadbw %xmm1, %xmm4
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: psadbw %xmm1, %xmm0
[X86][SSE] Add initial support for combining target shuffles to (V)PMOVZX. We can only handle 128-bit vectors until we support target shuffle inputs of different size to the output. llvm-svn: 288140
2016-11-29 22:18:51 +08:00
; X32-SSE-NEXT: packuswb %xmm4, %xmm0
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: retl
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
%out = call <4 x i32> @llvm.cttz.v4i32(<4 x i32> %in, i1 -1)
ret <4 x i32> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <8 x i16> @testv8i16(<8 x i16> %in) nounwind {
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-LABEL: testv8i16:
; SSE2: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE2-NEXT: pxor %xmm1, %xmm1
; SSE2-NEXT: psubw %xmm0, %xmm1
; SSE2-NEXT: pand %xmm0, %xmm1
; SSE2-NEXT: psubw {{.*}}(%rip), %xmm1
; SSE2-NEXT: movdqa %xmm1, %xmm0
; SSE2-NEXT: psrlw $1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psubw %xmm0, %xmm1
; SSE2-NEXT: movdqa {{.*#+}} xmm0 = [13107,13107,13107,13107,13107,13107,13107,13107]
; SSE2-NEXT: movdqa %xmm1, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: psrlw $2, %xmm1
; SSE2-NEXT: pand %xmm0, %xmm1
; SSE2-NEXT: paddw %xmm2, %xmm1
; SSE2-NEXT: movdqa %xmm1, %xmm2
; SSE2-NEXT: psrlw $4, %xmm2
; SSE2-NEXT: paddw %xmm1, %xmm2
; SSE2-NEXT: pand {{.*}}(%rip), %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psllw $8, %xmm0
; SSE2-NEXT: paddb %xmm2, %xmm0
; SSE2-NEXT: psrlw $8, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-NEXT: retq
;
; SSE3-LABEL: testv8i16:
; SSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE3-NEXT: pxor %xmm1, %xmm1
; SSE3-NEXT: psubw %xmm0, %xmm1
; SSE3-NEXT: pand %xmm0, %xmm1
; SSE3-NEXT: psubw {{.*}}(%rip), %xmm1
; SSE3-NEXT: movdqa %xmm1, %xmm0
; SSE3-NEXT: psrlw $1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psubw %xmm0, %xmm1
; SSE3-NEXT: movdqa {{.*#+}} xmm0 = [13107,13107,13107,13107,13107,13107,13107,13107]
; SSE3-NEXT: movdqa %xmm1, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: psrlw $2, %xmm1
; SSE3-NEXT: pand %xmm0, %xmm1
; SSE3-NEXT: paddw %xmm2, %xmm1
; SSE3-NEXT: movdqa %xmm1, %xmm2
; SSE3-NEXT: psrlw $4, %xmm2
; SSE3-NEXT: paddw %xmm1, %xmm2
; SSE3-NEXT: pand {{.*}}(%rip), %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psllw $8, %xmm0
; SSE3-NEXT: paddb %xmm2, %xmm0
; SSE3-NEXT: psrlw $8, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE3-NEXT: retq
;
; SSSE3-LABEL: testv8i16:
; SSSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSSE3-NEXT: pxor %xmm1, %xmm1
; SSSE3-NEXT: psubw %xmm0, %xmm1
; SSSE3-NEXT: pand %xmm0, %xmm1
; SSSE3-NEXT: psubw {{.*}}(%rip), %xmm1
; SSSE3-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSSE3-NEXT: movdqa %xmm1, %xmm2
; SSSE3-NEXT: pand %xmm0, %xmm2
; SSSE3-NEXT: movdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSSE3-NEXT: movdqa %xmm3, %xmm4
; SSSE3-NEXT: pshufb %xmm2, %xmm4
; SSSE3-NEXT: psrlw $4, %xmm1
; SSSE3-NEXT: pand %xmm0, %xmm1
; SSSE3-NEXT: pshufb %xmm1, %xmm3
; SSSE3-NEXT: paddb %xmm4, %xmm3
; SSSE3-NEXT: movdqa %xmm3, %xmm0
; SSSE3-NEXT: psllw $8, %xmm0
; SSSE3-NEXT: paddb %xmm3, %xmm0
; SSSE3-NEXT: psrlw $8, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSSE3-NEXT: retq
;
; SSE41-LABEL: testv8i16:
; SSE41: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: pxor %xmm1, %xmm1
; SSE41-NEXT: psubw %xmm0, %xmm1
; SSE41-NEXT: pand %xmm0, %xmm1
; SSE41-NEXT: psubw {{.*}}(%rip), %xmm1
; SSE41-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSE41-NEXT: movdqa %xmm1, %xmm2
; SSE41-NEXT: pand %xmm0, %xmm2
; SSE41-NEXT: movdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSE41-NEXT: movdqa %xmm3, %xmm4
; SSE41-NEXT: pshufb %xmm2, %xmm4
; SSE41-NEXT: psrlw $4, %xmm1
; SSE41-NEXT: pand %xmm0, %xmm1
; SSE41-NEXT: pshufb %xmm1, %xmm3
; SSE41-NEXT: paddb %xmm4, %xmm3
; SSE41-NEXT: movdqa %xmm3, %xmm0
; SSE41-NEXT: psllw $8, %xmm0
; SSE41-NEXT: paddb %xmm3, %xmm0
; SSE41-NEXT: psrlw $8, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE41-NEXT: retq
;
[X86][AVX] Regenerate tzcnt tests. The EVEX -> VEX fix means that AVX/AVX512 code is more likely the same now. llvm-svn: 291241
2017-01-06 23:54:23 +08:00
; AVX-LABEL: testv8i16:
; AVX: # BB#0:
; AVX-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX-NEXT: vpsubw %xmm0, %xmm1, %xmm1
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX-NEXT: vpsubw {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: vmovdqa {{.*#+}} xmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm2
; AVX-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX-NEXT: vpshufb %xmm2, %xmm3, %xmm2
; AVX-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX-NEXT: vpshufb %xmm0, %xmm3, %xmm0
; AVX-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX-NEXT: vpsllw $8, %xmm0, %xmm1
; AVX-NEXT: vpaddb %xmm0, %xmm1, %xmm0
; AVX-NEXT: vpsrlw $8, %xmm0, %xmm0
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: testv8i16:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubw %xmm0, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsubw {{.*}}(%rip), %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vmovdqa {{.*#+}} xmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm2
; AVX512VPOPCNTDQ-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512VPOPCNTDQ-NEXT: vpshufb %xmm2, %xmm3, %xmm2
; AVX512VPOPCNTDQ-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpshufb %xmm0, %xmm3, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsllw $8, %xmm0, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpaddb %xmm0, %xmm1, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsrlw $8, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: testv8i16:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: pxor %xmm1, %xmm1
; X32-SSE-NEXT: psubw %xmm0, %xmm1
; X32-SSE-NEXT: pand %xmm0, %xmm1
[X86] Updated test checks script to generalise LCPI symbol refs The script now replace '.LCPI888_8' style asm symbols with the {{\.LCPI.*}} re pattern - this helps stop hardcoded symbols in 32-bit x86 tests changing with every edit of the file Refreshed some tests to demonstrate the new check llvm-svn: 272488
2016-06-12 04:39:21 +08:00
; X32-SSE-NEXT: psubw {{\.LCPI.*}}, %xmm1
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; X32-SSE-NEXT: movdqa %xmm1, %xmm2
; X32-SSE-NEXT: pand %xmm0, %xmm2
; X32-SSE-NEXT: movdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; X32-SSE-NEXT: movdqa %xmm3, %xmm4
; X32-SSE-NEXT: pshufb %xmm2, %xmm4
; X32-SSE-NEXT: psrlw $4, %xmm1
; X32-SSE-NEXT: pand %xmm0, %xmm1
; X32-SSE-NEXT: pshufb %xmm1, %xmm3
; X32-SSE-NEXT: paddb %xmm4, %xmm3
; X32-SSE-NEXT: movdqa %xmm3, %xmm0
; X32-SSE-NEXT: psllw $8, %xmm0
; X32-SSE-NEXT: paddb %xmm3, %xmm0
; X32-SSE-NEXT: psrlw $8, %xmm0
; X32-SSE-NEXT: retl
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
%out = call <8 x i16> @llvm.cttz.v8i16(<8 x i16> %in, i1 0)
ret <8 x i16> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <8 x i16> @testv8i16u(<8 x i16> %in) nounwind {
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-LABEL: testv8i16u:
; SSE2: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE2-NEXT: pxor %xmm1, %xmm1
; SSE2-NEXT: psubw %xmm0, %xmm1
; SSE2-NEXT: pand %xmm0, %xmm1
; SSE2-NEXT: psubw {{.*}}(%rip), %xmm1
; SSE2-NEXT: movdqa %xmm1, %xmm0
; SSE2-NEXT: psrlw $1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psubw %xmm0, %xmm1
; SSE2-NEXT: movdqa {{.*#+}} xmm0 = [13107,13107,13107,13107,13107,13107,13107,13107]
; SSE2-NEXT: movdqa %xmm1, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: psrlw $2, %xmm1
; SSE2-NEXT: pand %xmm0, %xmm1
; SSE2-NEXT: paddw %xmm2, %xmm1
; SSE2-NEXT: movdqa %xmm1, %xmm2
; SSE2-NEXT: psrlw $4, %xmm2
; SSE2-NEXT: paddw %xmm1, %xmm2
; SSE2-NEXT: pand {{.*}}(%rip), %xmm2
; SSE2-NEXT: movdqa %xmm2, %xmm0
; SSE2-NEXT: psllw $8, %xmm0
; SSE2-NEXT: paddb %xmm2, %xmm0
; SSE2-NEXT: psrlw $8, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-NEXT: retq
;
; SSE3-LABEL: testv8i16u:
; SSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE3-NEXT: pxor %xmm1, %xmm1
; SSE3-NEXT: psubw %xmm0, %xmm1
; SSE3-NEXT: pand %xmm0, %xmm1
; SSE3-NEXT: psubw {{.*}}(%rip), %xmm1
; SSE3-NEXT: movdqa %xmm1, %xmm0
; SSE3-NEXT: psrlw $1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psubw %xmm0, %xmm1
; SSE3-NEXT: movdqa {{.*#+}} xmm0 = [13107,13107,13107,13107,13107,13107,13107,13107]
; SSE3-NEXT: movdqa %xmm1, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: psrlw $2, %xmm1
; SSE3-NEXT: pand %xmm0, %xmm1
; SSE3-NEXT: paddw %xmm2, %xmm1
; SSE3-NEXT: movdqa %xmm1, %xmm2
; SSE3-NEXT: psrlw $4, %xmm2
; SSE3-NEXT: paddw %xmm1, %xmm2
; SSE3-NEXT: pand {{.*}}(%rip), %xmm2
; SSE3-NEXT: movdqa %xmm2, %xmm0
; SSE3-NEXT: psllw $8, %xmm0
; SSE3-NEXT: paddb %xmm2, %xmm0
; SSE3-NEXT: psrlw $8, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE3-NEXT: retq
;
; SSSE3-LABEL: testv8i16u:
; SSSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSSE3-NEXT: pxor %xmm1, %xmm1
; SSSE3-NEXT: psubw %xmm0, %xmm1
; SSSE3-NEXT: pand %xmm0, %xmm1
; SSSE3-NEXT: psubw {{.*}}(%rip), %xmm1
; SSSE3-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSSE3-NEXT: movdqa %xmm1, %xmm2
; SSSE3-NEXT: pand %xmm0, %xmm2
; SSSE3-NEXT: movdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSSE3-NEXT: movdqa %xmm3, %xmm4
; SSSE3-NEXT: pshufb %xmm2, %xmm4
; SSSE3-NEXT: psrlw $4, %xmm1
; SSSE3-NEXT: pand %xmm0, %xmm1
; SSSE3-NEXT: pshufb %xmm1, %xmm3
; SSSE3-NEXT: paddb %xmm4, %xmm3
; SSSE3-NEXT: movdqa %xmm3, %xmm0
; SSSE3-NEXT: psllw $8, %xmm0
; SSSE3-NEXT: paddb %xmm3, %xmm0
; SSSE3-NEXT: psrlw $8, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSSE3-NEXT: retq
;
; SSE41-LABEL: testv8i16u:
; SSE41: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: pxor %xmm1, %xmm1
; SSE41-NEXT: psubw %xmm0, %xmm1
; SSE41-NEXT: pand %xmm0, %xmm1
; SSE41-NEXT: psubw {{.*}}(%rip), %xmm1
; SSE41-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSE41-NEXT: movdqa %xmm1, %xmm2
; SSE41-NEXT: pand %xmm0, %xmm2
; SSE41-NEXT: movdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSE41-NEXT: movdqa %xmm3, %xmm4
; SSE41-NEXT: pshufb %xmm2, %xmm4
; SSE41-NEXT: psrlw $4, %xmm1
; SSE41-NEXT: pand %xmm0, %xmm1
; SSE41-NEXT: pshufb %xmm1, %xmm3
; SSE41-NEXT: paddb %xmm4, %xmm3
; SSE41-NEXT: movdqa %xmm3, %xmm0
; SSE41-NEXT: psllw $8, %xmm0
; SSE41-NEXT: paddb %xmm3, %xmm0
; SSE41-NEXT: psrlw $8, %xmm0
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
; SSE41-NEXT: retq
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
;
[X86][AVX] Regenerate tzcnt tests. The EVEX -> VEX fix means that AVX/AVX512 code is more likely the same now. llvm-svn: 291241
2017-01-06 23:54:23 +08:00
; AVX-LABEL: testv8i16u:
; AVX: # BB#0:
; AVX-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX-NEXT: vpsubw %xmm0, %xmm1, %xmm1
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX-NEXT: vpsubw {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: vmovdqa {{.*#+}} xmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm2
; AVX-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX-NEXT: vpshufb %xmm2, %xmm3, %xmm2
; AVX-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX-NEXT: vpshufb %xmm0, %xmm3, %xmm0
; AVX-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX-NEXT: vpsllw $8, %xmm0, %xmm1
; AVX-NEXT: vpaddb %xmm0, %xmm1, %xmm0
; AVX-NEXT: vpsrlw $8, %xmm0, %xmm0
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: testv8i16u:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubw %xmm0, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsubw {{.*}}(%rip), %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vmovdqa {{.*#+}} xmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm2
; AVX512VPOPCNTDQ-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512VPOPCNTDQ-NEXT: vpshufb %xmm2, %xmm3, %xmm2
; AVX512VPOPCNTDQ-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpshufb %xmm0, %xmm3, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsllw $8, %xmm0, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpaddb %xmm0, %xmm1, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsrlw $8, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: testv8i16u:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: pxor %xmm1, %xmm1
; X32-SSE-NEXT: psubw %xmm0, %xmm1
; X32-SSE-NEXT: pand %xmm0, %xmm1
[X86] Updated test checks script to generalise LCPI symbol refs The script now replace '.LCPI888_8' style asm symbols with the {{\.LCPI.*}} re pattern - this helps stop hardcoded symbols in 32-bit x86 tests changing with every edit of the file Refreshed some tests to demonstrate the new check llvm-svn: 272488
2016-06-12 04:39:21 +08:00
; X32-SSE-NEXT: psubw {{\.LCPI.*}}, %xmm1
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: movdqa {{.*#+}} xmm0 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; X32-SSE-NEXT: movdqa %xmm1, %xmm2
; X32-SSE-NEXT: pand %xmm0, %xmm2
; X32-SSE-NEXT: movdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; X32-SSE-NEXT: movdqa %xmm3, %xmm4
; X32-SSE-NEXT: pshufb %xmm2, %xmm4
; X32-SSE-NEXT: psrlw $4, %xmm1
; X32-SSE-NEXT: pand %xmm0, %xmm1
; X32-SSE-NEXT: pshufb %xmm1, %xmm3
; X32-SSE-NEXT: paddb %xmm4, %xmm3
; X32-SSE-NEXT: movdqa %xmm3, %xmm0
; X32-SSE-NEXT: psllw $8, %xmm0
; X32-SSE-NEXT: paddb %xmm3, %xmm0
; X32-SSE-NEXT: psrlw $8, %xmm0
; X32-SSE-NEXT: retl
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
%out = call <8 x i16> @llvm.cttz.v8i16(<8 x i16> %in, i1 -1)
ret <8 x i16> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <16 x i8> @testv16i8(<16 x i8> %in) nounwind {
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-LABEL: testv16i8:
; SSE2: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE2-NEXT: pxor %xmm1, %xmm1
; SSE2-NEXT: psubb %xmm0, %xmm1
; SSE2-NEXT: pand %xmm0, %xmm1
; SSE2-NEXT: psubb {{.*}}(%rip), %xmm1
; SSE2-NEXT: movdqa %xmm1, %xmm0
; SSE2-NEXT: psrlw $1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psubb %xmm0, %xmm1
; SSE2-NEXT: movdqa {{.*#+}} xmm0 = [51,51,51,51,51,51,51,51,51,51,51,51,51,51,51,51]
; SSE2-NEXT: movdqa %xmm1, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: psrlw $2, %xmm1
; SSE2-NEXT: pand %xmm0, %xmm1
; SSE2-NEXT: paddb %xmm2, %xmm1
; SSE2-NEXT: movdqa %xmm1, %xmm0
; SSE2-NEXT: psrlw $4, %xmm0
; SSE2-NEXT: paddb %xmm1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-NEXT: retq
;
; SSE3-LABEL: testv16i8:
; SSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE3-NEXT: pxor %xmm1, %xmm1
; SSE3-NEXT: psubb %xmm0, %xmm1
; SSE3-NEXT: pand %xmm0, %xmm1
; SSE3-NEXT: psubb {{.*}}(%rip), %xmm1
; SSE3-NEXT: movdqa %xmm1, %xmm0
; SSE3-NEXT: psrlw $1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psubb %xmm0, %xmm1
; SSE3-NEXT: movdqa {{.*#+}} xmm0 = [51,51,51,51,51,51,51,51,51,51,51,51,51,51,51,51]
; SSE3-NEXT: movdqa %xmm1, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: psrlw $2, %xmm1
; SSE3-NEXT: pand %xmm0, %xmm1
; SSE3-NEXT: paddb %xmm2, %xmm1
; SSE3-NEXT: movdqa %xmm1, %xmm0
; SSE3-NEXT: psrlw $4, %xmm0
; SSE3-NEXT: paddb %xmm1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE3-NEXT: retq
;
; SSSE3-LABEL: testv16i8:
; SSSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSSE3-NEXT: pxor %xmm1, %xmm1
; SSSE3-NEXT: psubb %xmm0, %xmm1
; SSSE3-NEXT: pand %xmm0, %xmm1
; SSSE3-NEXT: psubb {{.*}}(%rip), %xmm1
; SSSE3-NEXT: movdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSSE3-NEXT: movdqa %xmm1, %xmm3
; SSSE3-NEXT: pand %xmm2, %xmm3
; SSSE3-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSSE3-NEXT: movdqa %xmm0, %xmm4
; SSSE3-NEXT: pshufb %xmm3, %xmm4
; SSSE3-NEXT: psrlw $4, %xmm1
; SSSE3-NEXT: pand %xmm2, %xmm1
; SSSE3-NEXT: pshufb %xmm1, %xmm0
; SSSE3-NEXT: paddb %xmm4, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSSE3-NEXT: retq
;
; SSE41-LABEL: testv16i8:
; SSE41: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: pxor %xmm1, %xmm1
; SSE41-NEXT: psubb %xmm0, %xmm1
; SSE41-NEXT: pand %xmm0, %xmm1
; SSE41-NEXT: psubb {{.*}}(%rip), %xmm1
; SSE41-NEXT: movdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSE41-NEXT: movdqa %xmm1, %xmm3
; SSE41-NEXT: pand %xmm2, %xmm3
; SSE41-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSE41-NEXT: movdqa %xmm0, %xmm4
; SSE41-NEXT: pshufb %xmm3, %xmm4
; SSE41-NEXT: psrlw $4, %xmm1
; SSE41-NEXT: pand %xmm2, %xmm1
; SSE41-NEXT: pshufb %xmm1, %xmm0
; SSE41-NEXT: paddb %xmm4, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE41-NEXT: retq
;
[X86][AVX] Regenerate tzcnt tests. The EVEX -> VEX fix means that AVX/AVX512 code is more likely the same now. llvm-svn: 291241
2017-01-06 23:54:23 +08:00
; AVX-LABEL: testv16i8:
; AVX: # BB#0:
; AVX-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX-NEXT: vpsubb %xmm0, %xmm1, %xmm1
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX-NEXT: vpsubb {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: vmovdqa {{.*#+}} xmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm2
; AVX-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX-NEXT: vpshufb %xmm2, %xmm3, %xmm2
; AVX-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX-NEXT: vpshufb %xmm0, %xmm3, %xmm0
; AVX-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: testv16i8:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubb %xmm0, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsubb {{.*}}(%rip), %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vmovdqa {{.*#+}} xmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm2
; AVX512VPOPCNTDQ-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512VPOPCNTDQ-NEXT: vpshufb %xmm2, %xmm3, %xmm2
; AVX512VPOPCNTDQ-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpshufb %xmm0, %xmm3, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: testv16i8:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: pxor %xmm1, %xmm1
; X32-SSE-NEXT: psubb %xmm0, %xmm1
; X32-SSE-NEXT: pand %xmm0, %xmm1
[X86] Updated test checks script to generalise LCPI symbol refs The script now replace '.LCPI888_8' style asm symbols with the {{\.LCPI.*}} re pattern - this helps stop hardcoded symbols in 32-bit x86 tests changing with every edit of the file Refreshed some tests to demonstrate the new check llvm-svn: 272488
2016-06-12 04:39:21 +08:00
; X32-SSE-NEXT: psubb {{\.LCPI.*}}, %xmm1
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: movdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; X32-SSE-NEXT: movdqa %xmm1, %xmm3
; X32-SSE-NEXT: pand %xmm2, %xmm3
; X32-SSE-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; X32-SSE-NEXT: movdqa %xmm0, %xmm4
; X32-SSE-NEXT: pshufb %xmm3, %xmm4
; X32-SSE-NEXT: psrlw $4, %xmm1
; X32-SSE-NEXT: pand %xmm2, %xmm1
; X32-SSE-NEXT: pshufb %xmm1, %xmm0
; X32-SSE-NEXT: paddb %xmm4, %xmm0
; X32-SSE-NEXT: retl
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
%out = call <16 x i8> @llvm.cttz.v16i8(<16 x i8> %in, i1 0)
ret <16 x i8> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <16 x i8> @testv16i8u(<16 x i8> %in) nounwind {
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-LABEL: testv16i8u:
; SSE2: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE2-NEXT: pxor %xmm1, %xmm1
; SSE2-NEXT: psubb %xmm0, %xmm1
; SSE2-NEXT: pand %xmm0, %xmm1
; SSE2-NEXT: psubb {{.*}}(%rip), %xmm1
; SSE2-NEXT: movdqa %xmm1, %xmm0
; SSE2-NEXT: psrlw $1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: psubb %xmm0, %xmm1
; SSE2-NEXT: movdqa {{.*#+}} xmm0 = [51,51,51,51,51,51,51,51,51,51,51,51,51,51,51,51]
; SSE2-NEXT: movdqa %xmm1, %xmm2
; SSE2-NEXT: pand %xmm0, %xmm2
; SSE2-NEXT: psrlw $2, %xmm1
; SSE2-NEXT: pand %xmm0, %xmm1
; SSE2-NEXT: paddb %xmm2, %xmm1
; SSE2-NEXT: movdqa %xmm1, %xmm0
; SSE2-NEXT: psrlw $4, %xmm0
; SSE2-NEXT: paddb %xmm1, %xmm0
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE2-NEXT: retq
;
; SSE3-LABEL: testv16i8u:
; SSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE3-NEXT: pxor %xmm1, %xmm1
; SSE3-NEXT: psubb %xmm0, %xmm1
; SSE3-NEXT: pand %xmm0, %xmm1
; SSE3-NEXT: psubb {{.*}}(%rip), %xmm1
; SSE3-NEXT: movdqa %xmm1, %xmm0
; SSE3-NEXT: psrlw $1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
; SSE3-NEXT: psubb %xmm0, %xmm1
; SSE3-NEXT: movdqa {{.*#+}} xmm0 = [51,51,51,51,51,51,51,51,51,51,51,51,51,51,51,51]
; SSE3-NEXT: movdqa %xmm1, %xmm2
; SSE3-NEXT: pand %xmm0, %xmm2
; SSE3-NEXT: psrlw $2, %xmm1
; SSE3-NEXT: pand %xmm0, %xmm1
; SSE3-NEXT: paddb %xmm2, %xmm1
; SSE3-NEXT: movdqa %xmm1, %xmm0
; SSE3-NEXT: psrlw $4, %xmm0
; SSE3-NEXT: paddb %xmm1, %xmm0
; SSE3-NEXT: pand {{.*}}(%rip), %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE3-NEXT: retq
;
; SSSE3-LABEL: testv16i8u:
; SSSE3: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSSE3-NEXT: pxor %xmm1, %xmm1
; SSSE3-NEXT: psubb %xmm0, %xmm1
; SSSE3-NEXT: pand %xmm0, %xmm1
; SSSE3-NEXT: psubb {{.*}}(%rip), %xmm1
; SSSE3-NEXT: movdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSSE3-NEXT: movdqa %xmm1, %xmm3
; SSSE3-NEXT: pand %xmm2, %xmm3
; SSSE3-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSSE3-NEXT: movdqa %xmm0, %xmm4
; SSSE3-NEXT: pshufb %xmm3, %xmm4
; SSSE3-NEXT: psrlw $4, %xmm1
; SSSE3-NEXT: pand %xmm2, %xmm1
; SSSE3-NEXT: pshufb %xmm1, %xmm0
; SSSE3-NEXT: paddb %xmm4, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSSE3-NEXT: retq
;
; SSE41-LABEL: testv16i8u:
; SSE41: # BB#0:
[X86][SSE] Vectorize CTTZ + CTTZ_ZERO_UNDEF Now that we have fast vector CTPOP implementations we can use this to speed up vector CTTZ using the pattern (cttz(x) = ctpop((x & -x) - 1)) Additionally, for AVX512CD that provides lzcnt instructions we can use the pattern (cttz_undef(x) = (width - 1) - ctlz(x & -x)) Differential Revision: http://reviews.llvm.org/D12663 llvm-svn: 248091
2015-09-19 21:22:57 +08:00
; SSE41-NEXT: pxor %xmm1, %xmm1
; SSE41-NEXT: psubb %xmm0, %xmm1
; SSE41-NEXT: pand %xmm0, %xmm1
; SSE41-NEXT: psubb {{.*}}(%rip), %xmm1
; SSE41-NEXT: movdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; SSE41-NEXT: movdqa %xmm1, %xmm3
; SSE41-NEXT: pand %xmm2, %xmm3
; SSE41-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; SSE41-NEXT: movdqa %xmm0, %xmm4
; SSE41-NEXT: pshufb %xmm3, %xmm4
; SSE41-NEXT: psrlw $4, %xmm1
; SSE41-NEXT: pand %xmm2, %xmm1
; SSE41-NEXT: pshufb %xmm1, %xmm0
; SSE41-NEXT: paddb %xmm4, %xmm0
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
; SSE41-NEXT: retq
;
[X86][AVX] Regenerate tzcnt tests. The EVEX -> VEX fix means that AVX/AVX512 code is more likely the same now. llvm-svn: 291241
2017-01-06 23:54:23 +08:00
; AVX-LABEL: testv16i8u:
; AVX: # BB#0:
; AVX-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX-NEXT: vpsubb %xmm0, %xmm1, %xmm1
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX-NEXT: vpsubb {{.*}}(%rip), %xmm0, %xmm0
; AVX-NEXT: vmovdqa {{.*#+}} xmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm2
; AVX-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX-NEXT: vpshufb %xmm2, %xmm3, %xmm2
; AVX-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX-NEXT: vpshufb %xmm0, %xmm3, %xmm0
; AVX-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: testv16i8u:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpsubb %xmm0, %xmm1, %xmm1
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpsubb {{.*}}(%rip), %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vmovdqa {{.*#+}} xmm1 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm2
; AVX512VPOPCNTDQ-NEXT: vmovdqa {{.*#+}} xmm3 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; AVX512VPOPCNTDQ-NEXT: vpshufb %xmm2, %xmm3, %xmm2
; AVX512VPOPCNTDQ-NEXT: vpsrlw $4, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpand %xmm1, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpshufb %xmm0, %xmm3, %xmm0
; AVX512VPOPCNTDQ-NEXT: vpaddb %xmm2, %xmm0, %xmm0
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: testv16i8u:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: pxor %xmm1, %xmm1
; X32-SSE-NEXT: psubb %xmm0, %xmm1
; X32-SSE-NEXT: pand %xmm0, %xmm1
[X86] Updated test checks script to generalise LCPI symbol refs The script now replace '.LCPI888_8' style asm symbols with the {{\.LCPI.*}} re pattern - this helps stop hardcoded symbols in 32-bit x86 tests changing with every edit of the file Refreshed some tests to demonstrate the new check llvm-svn: 272488
2016-06-12 04:39:21 +08:00
; X32-SSE-NEXT: psubb {{\.LCPI.*}}, %xmm1
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: movdqa {{.*#+}} xmm2 = [15,15,15,15,15,15,15,15,15,15,15,15,15,15,15,15]
; X32-SSE-NEXT: movdqa %xmm1, %xmm3
; X32-SSE-NEXT: pand %xmm2, %xmm3
; X32-SSE-NEXT: movdqa {{.*#+}} xmm0 = [0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4]
; X32-SSE-NEXT: movdqa %xmm0, %xmm4
; X32-SSE-NEXT: pshufb %xmm3, %xmm4
; X32-SSE-NEXT: psrlw $4, %xmm1
; X32-SSE-NEXT: pand %xmm2, %xmm1
; X32-SSE-NEXT: pshufb %xmm1, %xmm0
; X32-SSE-NEXT: paddb %xmm4, %xmm0
; X32-SSE-NEXT: retl
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
%out = call <16 x i8> @llvm.cttz.v16i8(<16 x i8> %in, i1 -1)
ret <16 x i8> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <2 x i64> @foldv2i64() nounwind {
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-LABEL: foldv2i64:
; SSE: # BB#0:
; SSE-NEXT: movl $8, %eax
[X86][SSE2] Fix asm string for movq (Move Quadword) instruction. Replace "mov{d|q}" with "movq". Differential Revision: https://reviews.llvm.org/D32220 llvm-svn: 301386
2017-04-26 15:08:44 +08:00
; SSE-NEXT: movq %rax, %xmm0
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-NEXT: retq
;
; AVX-LABEL: foldv2i64:
; AVX: # BB#0:
; AVX-NEXT: movl $8, %eax
; AVX-NEXT: vmovq %rax, %xmm0
; AVX-NEXT: retq
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: foldv2i64:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: movl $8, %eax
; AVX512VPOPCNTDQ-NEXT: vmovq %rax, %xmm0
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: foldv2i64:
; X32-SSE: # BB#0:
[DAGCombine] Allow vector constant folding of any value type before type legalization The patch comes in 2 parts: 1 - it makes use of the SelectionDAG::NewNodesMustHaveLegalTypes flag to tell when it can safely constant fold illegal types. 2 - it correctly resets SelectionDAG::NewNodesMustHaveLegalTypes at the start of each call to SelectionDAGISel::CodeGenAndEmitDAG so all the pre-legalization stages can make use of it - not just the first basic block that gets handled. Fix for PR30760 Differential Revision: https://reviews.llvm.org/D29568 llvm-svn: 294749
2017-02-10 22:37:25 +08:00
; X32-SSE-NEXT: movl $8, %eax
; X32-SSE-NEXT: movd %eax, %xmm0
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: retl
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
%out = call <2 x i64> @llvm.cttz.v2i64(<2 x i64> <i64 256, i64 -1>, i1 0)
ret <2 x i64> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <2 x i64> @foldv2i64u() nounwind {
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-LABEL: foldv2i64u:
; SSE: # BB#0:
; SSE-NEXT: movl $8, %eax
[X86][SSE2] Fix asm string for movq (Move Quadword) instruction. Replace "mov{d|q}" with "movq". Differential Revision: https://reviews.llvm.org/D32220 llvm-svn: 301386
2017-04-26 15:08:44 +08:00
; SSE-NEXT: movq %rax, %xmm0
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-NEXT: retq
;
; AVX-LABEL: foldv2i64u:
; AVX: # BB#0:
; AVX-NEXT: movl $8, %eax
; AVX-NEXT: vmovq %rax, %xmm0
; AVX-NEXT: retq
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: foldv2i64u:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: movl $8, %eax
; AVX512VPOPCNTDQ-NEXT: vmovq %rax, %xmm0
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: foldv2i64u:
; X32-SSE: # BB#0:
[DAGCombine] Allow vector constant folding of any value type before type legalization The patch comes in 2 parts: 1 - it makes use of the SelectionDAG::NewNodesMustHaveLegalTypes flag to tell when it can safely constant fold illegal types. 2 - it correctly resets SelectionDAG::NewNodesMustHaveLegalTypes at the start of each call to SelectionDAGISel::CodeGenAndEmitDAG so all the pre-legalization stages can make use of it - not just the first basic block that gets handled. Fix for PR30760 Differential Revision: https://reviews.llvm.org/D29568 llvm-svn: 294749
2017-02-10 22:37:25 +08:00
; X32-SSE-NEXT: movl $8, %eax
; X32-SSE-NEXT: movd %eax, %xmm0
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-NEXT: retl
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
%out = call <2 x i64> @llvm.cttz.v2i64(<2 x i64> <i64 256, i64 -1>, i1 -1)
ret <2 x i64> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <4 x i32> @foldv4i32() nounwind {
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-LABEL: foldv4i32:
; SSE: # BB#0:
; SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,32,0]
; SSE-NEXT: retq
;
[AVX-512] Fix duplicate column in AVX512 execution dependency table that was preventing VMOVDQU32/VMOVDQA32 from being recognized. Fix a bug in the code that stops execution dependency fix from turning operations on 32-bit integer element types into operations on 64-bit integer element types. llvm-svn: 277327
2016-08-01 15:55:33 +08:00
; AVX-LABEL: foldv4i32:
; AVX: # BB#0:
; AVX-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,32,0]
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: foldv4i32:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,32,0]
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: foldv4i32:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,32,0]
; X32-SSE-NEXT: retl
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
%out = call <4 x i32> @llvm.cttz.v4i32(<4 x i32> <i32 256, i32 -1, i32 0, i32 255>, i1 0)
ret <4 x i32> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <4 x i32> @foldv4i32u() nounwind {
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-LABEL: foldv4i32u:
; SSE: # BB#0:
; SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,32,0]
; SSE-NEXT: retq
;
[AVX-512] Fix duplicate column in AVX512 execution dependency table that was preventing VMOVDQU32/VMOVDQA32 from being recognized. Fix a bug in the code that stops execution dependency fix from turning operations on 32-bit integer element types into operations on 64-bit integer element types. llvm-svn: 277327
2016-08-01 15:55:33 +08:00
; AVX-LABEL: foldv4i32u:
; AVX: # BB#0:
; AVX-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,32,0]
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: foldv4i32u:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,32,0]
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: foldv4i32u:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,32,0]
; X32-SSE-NEXT: retl
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
%out = call <4 x i32> @llvm.cttz.v4i32(<4 x i32> <i32 256, i32 -1, i32 0, i32 255>, i1 -1)
ret <4 x i32> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <8 x i16> @foldv8i16() nounwind {
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-LABEL: foldv8i16:
; SSE: # BB#0:
; SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,16,0,16,0,3,3]
; SSE-NEXT: retq
;
[AVX-512] Fix duplicate column in AVX512 execution dependency table that was preventing VMOVDQU32/VMOVDQA32 from being recognized. Fix a bug in the code that stops execution dependency fix from turning operations on 32-bit integer element types into operations on 64-bit integer element types. llvm-svn: 277327
2016-08-01 15:55:33 +08:00
; AVX-LABEL: foldv8i16:
; AVX: # BB#0:
; AVX-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,16,0,16,0,3,3]
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: foldv8i16:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,16,0,16,0,3,3]
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: foldv8i16:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,16,0,16,0,3,3]
; X32-SSE-NEXT: retl
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
%out = call <8 x i16> @llvm.cttz.v8i16(<8 x i16> <i16 256, i16 -1, i16 0, i16 255, i16 -65536, i16 7, i16 24, i16 88>, i1 0)
ret <8 x i16> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <8 x i16> @foldv8i16u() nounwind {
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-LABEL: foldv8i16u:
; SSE: # BB#0:
; SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,16,0,16,0,3,3]
; SSE-NEXT: retq
;
[AVX-512] Fix duplicate column in AVX512 execution dependency table that was preventing VMOVDQU32/VMOVDQA32 from being recognized. Fix a bug in the code that stops execution dependency fix from turning operations on 32-bit integer element types into operations on 64-bit integer element types. llvm-svn: 277327
2016-08-01 15:55:33 +08:00
; AVX-LABEL: foldv8i16u:
; AVX: # BB#0:
; AVX-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,16,0,16,0,3,3]
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: foldv8i16u:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,16,0,16,0,3,3]
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: foldv8i16u:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,16,0,16,0,3,3]
; X32-SSE-NEXT: retl
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
%out = call <8 x i16> @llvm.cttz.v8i16(<8 x i16> <i16 256, i16 -1, i16 0, i16 255, i16 -65536, i16 7, i16 24, i16 88>, i1 -1)
ret <8 x i16> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <16 x i8> @foldv16i8() nounwind {
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-LABEL: foldv16i8:
; SSE: # BB#0:
; SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,8,0,8,0,3,3,1,1,0,1,2,3,4,5]
; SSE-NEXT: retq
;
[AVX-512] Fix duplicate column in AVX512 execution dependency table that was preventing VMOVDQU32/VMOVDQA32 from being recognized. Fix a bug in the code that stops execution dependency fix from turning operations on 32-bit integer element types into operations on 64-bit integer element types. llvm-svn: 277327
2016-08-01 15:55:33 +08:00
; AVX-LABEL: foldv16i8:
; AVX: # BB#0:
; AVX-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,8,0,8,0,3,3,1,1,0,1,2,3,4,5]
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: foldv16i8:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,8,0,8,0,3,3,1,1,0,1,2,3,4,5]
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: foldv16i8:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,8,0,8,0,3,3,1,1,0,1,2,3,4,5]
; X32-SSE-NEXT: retl
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
%out = call <16 x i8> @llvm.cttz.v16i8(<16 x i8> <i8 256, i8 -1, i8 0, i8 255, i8 -65536, i8 7, i8 24, i8 88, i8 -2, i8 254, i8 1, i8 2, i8 4, i8 8, i8 16, i8 32>, i1 0)
ret <16 x i8> %out
}
[X86][SSE] Tidied up vector CTLZ/CTTZ. NFCI. llvm-svn: 242645
2015-07-20 01:09:43 +08:00
define <16 x i8> @foldv16i8u() nounwind {
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
; SSE-LABEL: foldv16i8u:
; SSE: # BB#0:
; SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,8,0,8,0,3,3,1,1,0,1,2,3,4,5]
; SSE-NEXT: retq
;
[AVX-512] Fix duplicate column in AVX512 execution dependency table that was preventing VMOVDQU32/VMOVDQA32 from being recognized. Fix a bug in the code that stops execution dependency fix from turning operations on 32-bit integer element types into operations on 64-bit integer element types. llvm-svn: 277327
2016-08-01 15:55:33 +08:00
; AVX-LABEL: foldv16i8u:
; AVX: # BB#0:
; AVX-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,8,0,8,0,3,3,1,1,0,1,2,3,4,5]
; AVX-NEXT: retq
[AVX512] Teach lowering to use vplzcntd/q to implement 128/256-bit CTTZ_ZERO_UNDEF even without VLX support. We can just extend to 512-bits and extract like we do for CTLZ. llvm-svn: 267100
2016-04-22 11:22:38 +08:00
;
[X86] Adding vpopcntd and vpopcntq instructions AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the LLVM side of the addition of two new intrinsic based instructions (vpopcntd and vpopcntq). Differential Revision: https://reviews.llvm.org/D33169 llvm-svn: 303858
2017-05-25 21:45:23 +08:00
; AVX512VPOPCNTDQ-LABEL: foldv16i8u:
; AVX512VPOPCNTDQ: # BB#0:
; AVX512VPOPCNTDQ-NEXT: vmovaps {{.*#+}} xmm0 = [8,0,8,0,8,0,3,3,1,1,0,1,2,3,4,5]
; AVX512VPOPCNTDQ-NEXT: retq
;
[X86][SSE] Added 32-bit tests for vector lzcnt/tzcnt tests v2i64 tests are particularly bad on 32-bit targets. llvm-svn: 265789
2016-04-08 23:01:31 +08:00
; X32-SSE-LABEL: foldv16i8u:
; X32-SSE: # BB#0:
; X32-SSE-NEXT: movaps {{.*#+}} xmm0 = [8,0,8,0,8,0,3,3,1,1,0,1,2,3,4,5]
; X32-SSE-NEXT: retl
[DAGCombiner] Added CTTZ vector constant folding support. llvm-svn: 239293
2015-06-08 17:57:09 +08:00
%out = call <16 x i8> @llvm.cttz.v16i8(<16 x i8> <i8 256, i8 -1, i8 0, i8 255, i8 -65536, i8 7, i8 24, i8 88, i8 -2, i8 254, i8 1, i8 2, i8 4, i8 8, i8 16, i8 32>, i1 -1)
ret <16 x i8> %out
}
[X86] Added tzcnt vector tests. llvm-svn: 239264
2015-06-08 05:01:34 +08:00
declare <2 x i64> @llvm.cttz.v2i64(<2 x i64>, i1)
declare <4 x i32> @llvm.cttz.v4i32(<4 x i32>, i1)
declare <8 x i16> @llvm.cttz.v8i16(<8 x i16>, i1)
declare <16 x i8> @llvm.cttz.v16i8(<16 x i8>, i1)