maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/masked_load.ll

2125 lines
82 KiB
LLVM
Raw Normal View History

VirtRegMap: Replace some identity copies with KILL instructions. An identity COPY like this: %AL = COPY %AL, %EAX<imp-def> has no semantic effect, but encodes liveness information: Further users of %EAX only depend on this instruction even though it does not define the full register. Replace the COPY with a KILL instruction in those cases to maintain this liveness information. (This reverts a small part of r238588 but this time adds a comment explaining why a KILL instruction is useful). llvm-svn: 274952
2016-07-09 08:19:07 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
Recommit r354363 "[X86][SSE] Generalize X86ISD::BLENDI support to more value types" And its follow ups r354511, r354640. A follow patch will fix the issue that caused it to be reverted. llvm-svn: 354737
2019-02-24 05:41:42 +08:00
; RUN: llc < %s -disable-peephole -mtriple=x86_64-apple-darwin -mattr=sse2 | FileCheck %s --check-prefixes=SSE,SSE2
; RUN: llc < %s -disable-peephole -mtriple=x86_64-apple-darwin -mattr=sse4.2 | FileCheck %s --check-prefixes=SSE,SSE42
; RUN: llc < %s -disable-peephole -mtriple=x86_64-apple-darwin -mattr=avx | FileCheck %s --check-prefixes=AVX,AVX1OR2,AVX1
; RUN: llc < %s -disable-peephole -mtriple=x86_64-apple-darwin -mattr=avx2 | FileCheck %s --check-prefixes=AVX,AVX1OR2,AVX2
; RUN: llc < %s -disable-peephole -mtriple=x86_64-apple-darwin -mattr=avx512f | FileCheck %s --check-prefixes=AVX,AVX512,AVX512F
; RUN: llc < %s -disable-peephole -mtriple=x86_64-apple-darwin -mattr=avx512f,avx512bw,avx512vl | FileCheck %s --check-prefixes=AVX,AVX512,AVX512VLBW
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <1 x double> @load_v1f64_v1i64(<1 x i64> %trigger, <1 x double>* %addr, <1 x double> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: load_v1f64_v1i64:
; SSE: ## %bb.0:
; SSE-NEXT: testq %rdi, %rdi
; SSE-NEXT: jne LBB0_2
; SSE-NEXT: ## %bb.1: ## %cond.load
; SSE-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; SSE-NEXT: LBB0_2: ## %else
; SSE-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX-LABEL: load_v1f64_v1i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: ## %bb.0:
[X86] Update TTI to report that v1iX/v1fX types aren't legal for masked gather/scatter/load/store. The type legalizer will try to scalarize these operations if it sees them, but there is no handling for scalarizing them. This leads to a fatal error. With this change they will now be scalarized by the mem intrinsic scalarizing pass before SelectionDAG. llvm-svn: 318380
2017-11-16 14:02:05 +08:00
; AVX-NEXT: testq %rdi, %rdi
[ScalarizeMaskedMemIntrin] When expanding masked loads, start with the passthru value and insert each conditional load result over their element. Previously we started with undef and did one final merge at the end with a select. llvm-svn: 343271
2018-09-28 05:28:52 +08:00
; AVX-NEXT: jne LBB0_2
; AVX-NEXT: ## %bb.1: ## %cond.load
; AVX-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
; AVX-NEXT: LBB0_2: ## %else
[X86] Update TTI to report that v1iX/v1fX types aren't legal for masked gather/scatter/load/store. The type legalizer will try to scalarize these operations if it sees them, but there is no handling for scalarizing them. This leads to a fatal error. With this change they will now be scalarized by the mem intrinsic scalarizing pass before SelectionDAG. llvm-svn: 318380
2017-11-16 14:02:05 +08:00
; AVX-NEXT: retq
%mask = icmp eq <1 x i64> %trigger, zeroinitializer
%res = call <1 x double> @llvm.masked.load.v1f64.p0v1f64(<1 x double>* %addr, i32 4, <1 x i1>%mask, <1 x double>%dst)
ret <1 x double> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <2 x double> @load_v2f64_v2i64(<2 x i64> %trigger, <2 x double>* %addr, <2 x double> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_v2f64_v2i64:
; SSE2: ## %bb.0:
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: pcmpeqd %xmm0, %xmm2
; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm2[1,0,3,2]
; SSE2-NEXT: pand %xmm2, %xmm0
; SSE2-NEXT: movd %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB1_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE2-NEXT: LBB1_2: ## %else
; SSE2-NEXT: pextrw $4, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB1_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
; SSE2-NEXT: LBB1_4: ## %else2
; SSE2-NEXT: movapd %xmm1, %xmm0
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_v2f64_v2i64:
; SSE42: ## %bb.0:
; SSE42-NEXT: pxor %xmm2, %xmm2
; SSE42-NEXT: pcmpeqq %xmm0, %xmm2
; SSE42-NEXT: pextrb $0, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB1_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE42-NEXT: LBB1_2: ## %else
; SSE42-NEXT: pextrb $8, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB1_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: movhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
; SSE42-NEXT: LBB1_4: ## %else2
; SSE42-NEXT: movapd %xmm1, %xmm0
; SSE42-NEXT: retq
;
; AVX1OR2-LABEL: load_v2f64_v2i64:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1OR2-NEXT: vpcmpeqq %xmm2, %xmm0, %xmm0
; AVX1OR2-NEXT: vmaskmovpd (%rdi), %xmm0, %xmm2
; AVX1OR2-NEXT: vblendvpd %xmm0, %xmm2, %xmm1, %xmm0
; AVX1OR2-NEXT: retq
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_v2f64_v2i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm1 killed $xmm1 def $zmm1
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 def $zmm0
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmq %zmm0, %zmm0, %k0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: kshiftlw $14, %k0, %k0
; AVX512F-NEXT: kshiftrw $14, %k0, %k1
; AVX512F-NEXT: vblendmpd (%rdi), %zmm1, %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 killed $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_v2f64_v2i64:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vptestnmq %xmm0, %xmm0, %k1
; AVX512VLBW-NEXT: vblendmpd (%rdi), %xmm1, %xmm0 {%k1}
; AVX512VLBW-NEXT: retq
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
%mask = icmp eq <2 x i64> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <2 x double> @llvm.masked.load.v2f64.p0v2f64(<2 x double>* %addr, i32 4, <2 x i1>%mask, <2 x double>%dst)
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
ret <2 x double> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <4 x float> @load_v4f32_v4i32(<4 x i32> %trigger, <4 x float>* %addr, <4 x float> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_v4f32_v4i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: pcmpeqd %xmm0, %xmm2
; SSE2-NEXT: movd %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB2_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm1 = xmm3[0],xmm1[1,2,3]
; SSE2-NEXT: LBB2_2: ## %else
; SSE2-NEXT: pextrw $2, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB2_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm1[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[2,0],xmm1[2,3]
; SSE2-NEXT: movaps %xmm2, %xmm1
; SSE2-NEXT: LBB2_4: ## %else2
; SSE2-NEXT: xorps %xmm2, %xmm2
; SSE2-NEXT: pcmpeqd %xmm2, %xmm0
; SSE2-NEXT: pextrw $4, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB2_6
; SSE2-NEXT: ## %bb.5: ## %cond.load4
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm1[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0,2]
; SSE2-NEXT: LBB2_6: ## %else5
; SSE2-NEXT: pextrw $6, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB2_8
; SSE2-NEXT: ## %bb.7: ## %cond.load7
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm0[2,0]
; SSE2-NEXT: LBB2_8: ## %else8
; SSE2-NEXT: movaps %xmm1, %xmm0
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_v4f32_v4i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: pxor %xmm2, %xmm2
; SSE42-NEXT: pcmpeqd %xmm0, %xmm2
; SSE42-NEXT: pextrb $0, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB2_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE42-NEXT: blendps {{.*#+}} xmm1 = xmm3[0],xmm1[1,2,3]
; SSE42-NEXT: LBB2_2: ## %else
; SSE42-NEXT: pextrb $4, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB2_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: insertps {{.*#+}} xmm1 = xmm1[0],mem[0],xmm1[2,3]
; SSE42-NEXT: LBB2_4: ## %else2
; SSE42-NEXT: pxor %xmm2, %xmm2
; SSE42-NEXT: pcmpeqd %xmm2, %xmm0
; SSE42-NEXT: pextrb $8, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB2_6
; SSE42-NEXT: ## %bb.5: ## %cond.load4
; SSE42-NEXT: insertps {{.*#+}} xmm1 = xmm1[0,1],mem[0],xmm1[3]
; SSE42-NEXT: LBB2_6: ## %else5
; SSE42-NEXT: pextrb $12, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB2_8
; SSE42-NEXT: ## %bb.7: ## %cond.load7
; SSE42-NEXT: insertps {{.*#+}} xmm1 = xmm1[0,1,2],mem[0]
; SSE42-NEXT: LBB2_8: ## %else8
; SSE42-NEXT: movaps %xmm1, %xmm0
; SSE42-NEXT: retq
;
; AVX1OR2-LABEL: load_v4f32_v4i32:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1OR2-NEXT: vpcmpeqd %xmm2, %xmm0, %xmm0
; AVX1OR2-NEXT: vmaskmovps (%rdi), %xmm0, %xmm2
; AVX1OR2-NEXT: vblendvps %xmm0, %xmm2, %xmm1, %xmm0
; AVX1OR2-NEXT: retq
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_v4f32_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm1 killed $xmm1 def $zmm1
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 def $zmm0
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmd %zmm0, %zmm0, %k0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: kshiftlw $12, %k0, %k0
; AVX512F-NEXT: kshiftrw $12, %k0, %k1
; AVX512F-NEXT: vblendmps (%rdi), %zmm1, %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 killed $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_v4f32_v4i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vptestnmd %xmm0, %xmm0, %k1
; AVX512VLBW-NEXT: vblendmps (%rdi), %xmm1, %xmm0 {%k1}
; AVX512VLBW-NEXT: retq
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
%mask = icmp eq <4 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x float> @llvm.masked.load.v4f32.p0v4f32(<4 x float>* %addr, i32 4, <4 x i1>%mask, <4 x float>%dst)
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
ret <4 x float> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <4 x i32> @load_v4i32_v4i32(<4 x i32> %trigger, <4 x i32>* %addr, <4 x i32> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_v4i32_v4i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: pcmpeqd %xmm0, %xmm2
; SSE2-NEXT: movd %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB3_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm1 = xmm3[0],xmm1[1,2,3]
; SSE2-NEXT: LBB3_2: ## %else
; SSE2-NEXT: pextrw $2, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB3_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm1[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[2,0],xmm1[2,3]
; SSE2-NEXT: movaps %xmm2, %xmm1
; SSE2-NEXT: LBB3_4: ## %else2
; SSE2-NEXT: xorps %xmm2, %xmm2
; SSE2-NEXT: pcmpeqd %xmm2, %xmm0
; SSE2-NEXT: pextrw $4, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB3_6
; SSE2-NEXT: ## %bb.5: ## %cond.load4
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm1[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0,2]
; SSE2-NEXT: LBB3_6: ## %else5
; SSE2-NEXT: pextrw $6, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB3_8
; SSE2-NEXT: ## %bb.7: ## %cond.load7
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm0[2,0]
; SSE2-NEXT: LBB3_8: ## %else8
; SSE2-NEXT: movaps %xmm1, %xmm0
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_v4i32_v4i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: pxor %xmm2, %xmm2
; SSE42-NEXT: pcmpeqd %xmm0, %xmm2
; SSE42-NEXT: pextrb $0, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB3_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: pinsrd $0, (%rdi), %xmm1
; SSE42-NEXT: LBB3_2: ## %else
; SSE42-NEXT: pextrb $4, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB3_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: pinsrd $1, 4(%rdi), %xmm1
; SSE42-NEXT: LBB3_4: ## %else2
; SSE42-NEXT: pxor %xmm2, %xmm2
; SSE42-NEXT: pcmpeqd %xmm2, %xmm0
; SSE42-NEXT: pextrb $8, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB3_6
; SSE42-NEXT: ## %bb.5: ## %cond.load4
; SSE42-NEXT: pinsrd $2, 8(%rdi), %xmm1
; SSE42-NEXT: LBB3_6: ## %else5
; SSE42-NEXT: pextrb $12, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB3_8
; SSE42-NEXT: ## %bb.7: ## %cond.load7
; SSE42-NEXT: pinsrd $3, 12(%rdi), %xmm1
; SSE42-NEXT: LBB3_8: ## %else8
; SSE42-NEXT: movdqa %xmm1, %xmm0
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_v4i32_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpcmpeqd %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vmaskmovps (%rdi), %xmm0, %xmm2
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX1-NEXT: vblendvps %xmm0, %xmm2, %xmm1, %xmm0
; AVX1-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_v4i32_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX2-NEXT: vpcmpeqd %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vpmaskmovd (%rdi), %xmm0, %xmm2
; AVX2-NEXT: vblendvps %xmm0, %xmm2, %xmm1, %xmm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_v4i32_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm1 killed $xmm1 def $zmm1
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 def $zmm0
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmd %zmm0, %zmm0, %k0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: kshiftlw $12, %k0, %k0
; AVX512F-NEXT: kshiftrw $12, %k0, %k1
; AVX512F-NEXT: vpblendmd (%rdi), %zmm1, %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 killed $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_v4i32_v4i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vptestnmd %xmm0, %xmm0, %k1
; AVX512VLBW-NEXT: vpblendmd (%rdi), %xmm1, %xmm0 {%k1}
; AVX512VLBW-NEXT: retq
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
%mask = icmp eq <4 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x i32> @llvm.masked.load.v4i32.p0v4i32(<4 x i32>* %addr, i32 4, <4 x i1>%mask, <4 x i32>%dst)
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
ret <4 x i32> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <4 x double> @load_v4f64_v4i32(<4 x i32> %trigger, <4 x double>* %addr, <4 x double> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_v4f64_v4i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: pxor %xmm3, %xmm3
; SSE2-NEXT: pcmpeqd %xmm0, %xmm3
; SSE2-NEXT: movd %xmm3, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB4_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE2-NEXT: LBB4_2: ## %else
; SSE2-NEXT: pextrw $2, %xmm3, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB4_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
; SSE2-NEXT: LBB4_4: ## %else2
; SSE2-NEXT: pxor %xmm3, %xmm3
; SSE2-NEXT: pcmpeqd %xmm3, %xmm0
; SSE2-NEXT: pextrw $4, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB4_6
; SSE2-NEXT: ## %bb.5: ## %cond.load4
; SSE2-NEXT: movlpd {{.*#+}} xmm2 = mem[0],xmm2[1]
; SSE2-NEXT: LBB4_6: ## %else5
; SSE2-NEXT: pextrw $6, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB4_8
; SSE2-NEXT: ## %bb.7: ## %cond.load7
; SSE2-NEXT: movhpd {{.*#+}} xmm2 = xmm2[0],mem[0]
; SSE2-NEXT: LBB4_8: ## %else8
; SSE2-NEXT: movapd %xmm1, %xmm0
; SSE2-NEXT: movapd %xmm2, %xmm1
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_v4f64_v4i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: pxor %xmm3, %xmm3
; SSE42-NEXT: pcmpeqd %xmm0, %xmm3
; SSE42-NEXT: pextrb $0, %xmm3, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB4_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE42-NEXT: LBB4_2: ## %else
; SSE42-NEXT: pextrb $4, %xmm3, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB4_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: movhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
; SSE42-NEXT: LBB4_4: ## %else2
; SSE42-NEXT: pxor %xmm3, %xmm3
; SSE42-NEXT: pcmpeqd %xmm3, %xmm0
; SSE42-NEXT: pextrb $8, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB4_6
; SSE42-NEXT: ## %bb.5: ## %cond.load4
; SSE42-NEXT: movlpd {{.*#+}} xmm2 = mem[0],xmm2[1]
; SSE42-NEXT: LBB4_6: ## %else5
; SSE42-NEXT: pextrb $12, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB4_8
; SSE42-NEXT: ## %bb.7: ## %cond.load7
; SSE42-NEXT: movhpd {{.*#+}} xmm2 = xmm2[0],mem[0]
; SSE42-NEXT: LBB4_8: ## %else8
; SSE42-NEXT: movapd %xmm1, %xmm0
; SSE42-NEXT: movapd %xmm2, %xmm1
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_v4f64_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpcmpeqd %xmm2, %xmm0, %xmm0
; AVX1-NEXT: vpmovsxdq %xmm0, %xmm2
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX1-NEXT: vpshufd {{.*#+}} xmm0 = xmm0[2,3,0,1]
; AVX1-NEXT: vpmovsxdq %xmm0, %xmm0
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm2, %ymm0
; AVX1-NEXT: vmaskmovpd (%rdi), %ymm0, %ymm2
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX1-NEXT: vblendvpd %ymm0, %ymm2, %ymm1, %ymm0
; AVX1-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_v4f64_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX2-NEXT: vpcmpeqd %xmm2, %xmm0, %xmm0
; AVX2-NEXT: vpmovsxdq %xmm0, %ymm0
; AVX2-NEXT: vmaskmovpd (%rdi), %ymm0, %ymm2
; AVX2-NEXT: vblendvpd %ymm0, %ymm2, %ymm1, %ymm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_v4f64_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm1 killed $ymm1 def $zmm1
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 def $zmm0
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmd %zmm0, %zmm0, %k0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: kshiftlw $12, %k0, %k0
; AVX512F-NEXT: kshiftrw $12, %k0, %k1
; AVX512F-NEXT: vblendmpd (%rdi), %zmm1, %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_v4f64_v4i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vptestnmd %xmm0, %xmm0, %k1
; AVX512VLBW-NEXT: vblendmpd (%rdi), %ymm1, %ymm0 {%k1}
; AVX512VLBW-NEXT: retq
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
%mask = icmp eq <4 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x double> @llvm.masked.load.v4f64.p0v4f64(<4 x double>* %addr, i32 32, <4 x i1>%mask, <4 x double>%dst)
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
ret <4 x double> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <4 x double> @load_zero_v4f64_v4i32(<4 x i32> %trigger, <4 x double>* %addr, <4 x double> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_zero_v4f64_v4i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: movdqa %xmm0, %xmm2
; SSE2-NEXT: pxor %xmm0, %xmm0
; SSE2-NEXT: movdqa %xmm2, %xmm3
; SSE2-NEXT: pcmpeqd %xmm0, %xmm3
; SSE2-NEXT: movd %xmm3, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: xorpd %xmm1, %xmm1
; SSE2-NEXT: je LBB5_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movq {{.*#+}} xmm0 = mem[0],zero
; SSE2-NEXT: xorpd %xmm1, %xmm1
; SSE2-NEXT: LBB5_2: ## %else
; SSE2-NEXT: pextrw $2, %xmm3, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB5_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
; SSE2-NEXT: LBB5_4: ## %else2
; SSE2-NEXT: pxor %xmm3, %xmm3
; SSE2-NEXT: pcmpeqd %xmm3, %xmm2
; SSE2-NEXT: pextrw $4, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB5_6
; SSE2-NEXT: ## %bb.5: ## %cond.load4
; SSE2-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE2-NEXT: LBB5_6: ## %else5
; SSE2-NEXT: pextrw $6, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB5_8
; SSE2-NEXT: ## %bb.7: ## %cond.load7
; SSE2-NEXT: movhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
; SSE2-NEXT: LBB5_8: ## %else8
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_zero_v4f64_v4i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: movdqa %xmm0, %xmm2
; SSE42-NEXT: pxor %xmm0, %xmm0
; SSE42-NEXT: movdqa %xmm2, %xmm3
; SSE42-NEXT: pcmpeqd %xmm0, %xmm3
; SSE42-NEXT: pextrb $0, %xmm3, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: xorpd %xmm1, %xmm1
; SSE42-NEXT: je LBB5_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movq {{.*#+}} xmm0 = mem[0],zero
; SSE42-NEXT: xorpd %xmm1, %xmm1
; SSE42-NEXT: LBB5_2: ## %else
; SSE42-NEXT: pextrb $4, %xmm3, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB5_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: movhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
; SSE42-NEXT: LBB5_4: ## %else2
; SSE42-NEXT: pxor %xmm3, %xmm3
; SSE42-NEXT: pcmpeqd %xmm3, %xmm2
; SSE42-NEXT: pextrb $8, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB5_6
; SSE42-NEXT: ## %bb.5: ## %cond.load4
; SSE42-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE42-NEXT: LBB5_6: ## %else5
; SSE42-NEXT: pextrb $12, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB5_8
; SSE42-NEXT: ## %bb.7: ## %cond.load7
; SSE42-NEXT: movhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
; SSE42-NEXT: LBB5_8: ## %else8
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_zero_v4f64_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX1-NEXT: vpcmpeqd %xmm1, %xmm0, %xmm0
; AVX1-NEXT: vpmovsxdq %xmm0, %xmm1
; AVX1-NEXT: vpshufd {{.*#+}} xmm0 = xmm0[2,3,0,1]
; AVX1-NEXT: vpmovsxdq %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; AVX1-NEXT: vmaskmovpd (%rdi), %ymm0, %ymm0
; AVX1-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_zero_v4f64_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX2-NEXT: vpcmpeqd %xmm1, %xmm0, %xmm0
; AVX2-NEXT: vpmovsxdq %xmm0, %ymm0
; AVX2-NEXT: vmaskmovpd (%rdi), %ymm0, %ymm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_zero_v4f64_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 def $zmm0
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmd %zmm0, %zmm0, %k0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: kshiftlw $12, %k0, %k0
; AVX512F-NEXT: kshiftrw $12, %k0, %k1
; AVX512F-NEXT: vmovupd (%rdi), %zmm0 {%k1} {z}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_zero_v4f64_v4i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vptestnmd %xmm0, %xmm0, %k1
; AVX512VLBW-NEXT: vmovapd (%rdi), %ymm0 {%k1} {z}
; AVX512VLBW-NEXT: retq
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
%mask = icmp eq <4 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x double> @llvm.masked.load.v4f64.p0v4f64(<4 x double>* %addr, i32 32, <4 x i1>%mask, <4 x double>zeroinitializer)
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
ret <4 x double> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <8 x float> @load_v8f32_v8i32(<8 x i32> %trigger, <8 x float>* %addr, <8 x float> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_v8f32_v8i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: pxor %xmm4, %xmm4
; SSE2-NEXT: pcmpeqd %xmm0, %xmm4
; SSE2-NEXT: movdqa %xmm4, %xmm5
; SSE2-NEXT: packssdw %xmm0, %xmm5
; SSE2-NEXT: movd %xmm5, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB6_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movss {{.*#+}} xmm5 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm2 = xmm5[0],xmm2[1,2,3]
; SSE2-NEXT: LBB6_2: ## %else
[DAGCombiner] after simplifying demanded elements of vector operand of extract, revisit the extract; 2nd try This is a retry of rL349051 (reverted at rL349056). I changed the check for dead-ness from number of uses to an opcode test for DELETED_NODE based on existing similar code. Differential Revision: https://reviews.llvm.org/D55655 llvm-svn: 349058
2018-12-14 01:05:01 +08:00
; SSE2-NEXT: psrlq $16, %xmm4
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-NEXT: movd %xmm4, %eax
; SSE2-NEXT: shrl $16, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB6_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movss {{.*#+}} xmm4 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[0,0],xmm2[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[2,0],xmm2[2,3]
; SSE2-NEXT: movaps %xmm4, %xmm2
; SSE2-NEXT: LBB6_4: ## %else2
; SSE2-NEXT: xorps %xmm4, %xmm4
; SSE2-NEXT: pcmpeqd %xmm4, %xmm0
; SSE2-NEXT: pextrw $4, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB6_6
; SSE2-NEXT: ## %bb.5: ## %cond.load4
; SSE2-NEXT: movss {{.*#+}} xmm4 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm4 = xmm4[0,0],xmm2[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,1],xmm4[0,2]
; SSE2-NEXT: LBB6_6: ## %else5
; SSE2-NEXT: pextrw $6, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB6_8
; SSE2-NEXT: ## %bb.7: ## %cond.load7
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm2[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,1],xmm0[2,0]
; SSE2-NEXT: LBB6_8: ## %else8
; SSE2-NEXT: xorps %xmm0, %xmm0
; SSE2-NEXT: pcmpeqd %xmm1, %xmm0
; SSE2-NEXT: pextrw $0, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB6_10
; SSE2-NEXT: ## %bb.9: ## %cond.load10
; SSE2-NEXT: movss {{.*#+}} xmm4 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm3 = xmm4[0],xmm3[1,2,3]
; SSE2-NEXT: LBB6_10: ## %else11
; SSE2-NEXT: pextrw $2, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB6_12
; SSE2-NEXT: ## %bb.11: ## %cond.load13
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm3[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[2,0],xmm3[2,3]
; SSE2-NEXT: movaps %xmm0, %xmm3
; SSE2-NEXT: LBB6_12: ## %else14
; SSE2-NEXT: xorps %xmm0, %xmm0
; SSE2-NEXT: pcmpeqd %xmm0, %xmm1
; SSE2-NEXT: pextrw $4, %xmm1, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB6_14
; SSE2-NEXT: ## %bb.13: ## %cond.load16
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm3[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,1],xmm0[0,2]
; SSE2-NEXT: LBB6_14: ## %else17
; SSE2-NEXT: pextrw $6, %xmm1, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB6_16
; SSE2-NEXT: ## %bb.15: ## %cond.load19
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm3[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,1],xmm0[2,0]
; SSE2-NEXT: LBB6_16: ## %else20
; SSE2-NEXT: movaps %xmm2, %xmm0
; SSE2-NEXT: movaps %xmm3, %xmm1
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_v8f32_v8i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: pxor %xmm4, %xmm4
; SSE42-NEXT: pcmpeqd %xmm0, %xmm4
; SSE42-NEXT: pextrb $0, %xmm4, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB6_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movss {{.*#+}} xmm5 = mem[0],zero,zero,zero
; SSE42-NEXT: blendps {{.*#+}} xmm2 = xmm5[0],xmm2[1,2,3]
; SSE42-NEXT: LBB6_2: ## %else
; SSE42-NEXT: pextrb $4, %xmm4, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB6_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: insertps {{.*#+}} xmm2 = xmm2[0],mem[0],xmm2[2,3]
; SSE42-NEXT: LBB6_4: ## %else2
; SSE42-NEXT: pxor %xmm4, %xmm4
; SSE42-NEXT: pcmpeqd %xmm4, %xmm0
; SSE42-NEXT: pextrb $8, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB6_6
; SSE42-NEXT: ## %bb.5: ## %cond.load4
; SSE42-NEXT: insertps {{.*#+}} xmm2 = xmm2[0,1],mem[0],xmm2[3]
; SSE42-NEXT: LBB6_6: ## %else5
; SSE42-NEXT: pextrb $12, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB6_8
; SSE42-NEXT: ## %bb.7: ## %cond.load7
; SSE42-NEXT: insertps {{.*#+}} xmm2 = xmm2[0,1,2],mem[0]
; SSE42-NEXT: LBB6_8: ## %else8
; SSE42-NEXT: pxor %xmm0, %xmm0
; SSE42-NEXT: pcmpeqd %xmm1, %xmm0
; SSE42-NEXT: pextrb $0, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB6_10
; SSE42-NEXT: ## %bb.9: ## %cond.load10
; SSE42-NEXT: movd {{.*#+}} xmm4 = mem[0],zero,zero,zero
; SSE42-NEXT: pblendw {{.*#+}} xmm3 = xmm4[0,1],xmm3[2,3,4,5,6,7]
; SSE42-NEXT: LBB6_10: ## %else11
; SSE42-NEXT: pextrb $4, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB6_12
; SSE42-NEXT: ## %bb.11: ## %cond.load13
; SSE42-NEXT: insertps {{.*#+}} xmm3 = xmm3[0],mem[0],xmm3[2,3]
; SSE42-NEXT: LBB6_12: ## %else14
; SSE42-NEXT: pxor %xmm0, %xmm0
; SSE42-NEXT: pcmpeqd %xmm0, %xmm1
; SSE42-NEXT: pextrb $8, %xmm1, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB6_14
; SSE42-NEXT: ## %bb.13: ## %cond.load16
; SSE42-NEXT: insertps {{.*#+}} xmm3 = xmm3[0,1],mem[0],xmm3[3]
; SSE42-NEXT: LBB6_14: ## %else17
; SSE42-NEXT: pextrb $12, %xmm1, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB6_16
; SSE42-NEXT: ## %bb.15: ## %cond.load19
; SSE42-NEXT: insertps {{.*#+}} xmm3 = xmm3[0,1,2],mem[0]
; SSE42-NEXT: LBB6_16: ## %else20
; SSE42-NEXT: movaps %xmm2, %xmm0
; SSE42-NEXT: movaps %xmm3, %xmm1
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_v8f32_v8i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm2
; AVX1-NEXT: vpxor %xmm3, %xmm3, %xmm3
; AVX1-NEXT: vpcmpeqd %xmm3, %xmm2, %xmm2
; AVX1-NEXT: vpcmpeqd %xmm3, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm2, %ymm0, %ymm0
; AVX1-NEXT: vmaskmovps (%rdi), %ymm0, %ymm2
; AVX1-NEXT: vblendvps %ymm0, %ymm2, %ymm1, %ymm0
; AVX1-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_v8f32_v8i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
[X86] SET0 to use XMM registers where possible PR26018 PR32862 Differential Revision: https://reviews.llvm.org/D35839 llvm-svn: 309298
2017-07-28 01:47:01 +08:00
; AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpcmpeqd %ymm2, %ymm0, %ymm0
; AVX2-NEXT: vmaskmovps (%rdi), %ymm0, %ymm2
; AVX2-NEXT: vblendvps %ymm0, %ymm2, %ymm1, %ymm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_v8f32_v8i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm1 killed $ymm1 def $zmm1
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 def $zmm0
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmd %zmm0, %zmm0, %k0
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: kshiftlw $8, %k0, %k0
; AVX512F-NEXT: kshiftrw $8, %k0, %k1
[AVX-512] Teach two address instruction pass to replace masked move instructions with blendm instructions when its beneficial. Isel now selects masked move instructions for vselect instead of blendm. But sometimes it beneficial to register allocation to remove the tied register constraint by using blendm instructions. This also picks up cases where the masked move was created due to a masked load intrinsic. Differential Revision: https://reviews.llvm.org/D28454 llvm-svn: 292005
2017-01-14 15:50:52 +08:00
; AVX512F-NEXT: vblendmps (%rdi), %zmm1, %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_v8f32_v8i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vptestnmd %ymm0, %ymm0, %k1
; AVX512VLBW-NEXT: vblendmps (%rdi), %ymm1, %ymm0 {%k1}
; AVX512VLBW-NEXT: retq
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
%mask = icmp eq <8 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <8 x float> @llvm.masked.load.v8f32.p0v8f32(<8 x float>* %addr, i32 32, <8 x i1>%mask, <8 x float>%dst)
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
ret <8 x float> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <8 x i32> @load_v8i32_v8i1(<8 x i1> %mask, <8 x i32>* %addr, <8 x i32> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_v8i32_v8i1:
; SSE2: ## %bb.0:
; SSE2-NEXT: movd %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB7_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm1 = xmm3[0],xmm1[1,2,3]
; SSE2-NEXT: LBB7_2: ## %else
; SSE2-NEXT: shrl $16, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB7_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm1[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[2,0],xmm1[2,3]
; SSE2-NEXT: movaps %xmm3, %xmm1
; SSE2-NEXT: LBB7_4: ## %else2
; SSE2-NEXT: pextrw $2, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB7_6
; SSE2-NEXT: ## %bb.5: ## %cond.load4
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm1[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm3[0,2]
; SSE2-NEXT: LBB7_6: ## %else5
; SSE2-NEXT: pextrw $3, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB7_8
; SSE2-NEXT: ## %bb.7: ## %cond.load7
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm1[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm3[2,0]
; SSE2-NEXT: LBB7_8: ## %else8
; SSE2-NEXT: pextrw $4, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB7_10
; SSE2-NEXT: ## %bb.9: ## %cond.load10
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm2 = xmm3[0],xmm2[1,2,3]
; SSE2-NEXT: LBB7_10: ## %else11
; SSE2-NEXT: pextrw $5, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB7_12
; SSE2-NEXT: ## %bb.11: ## %cond.load13
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm2[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[2,0],xmm2[2,3]
; SSE2-NEXT: movaps %xmm3, %xmm2
; SSE2-NEXT: LBB7_12: ## %else14
; SSE2-NEXT: pextrw $6, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB7_14
; SSE2-NEXT: ## %bb.13: ## %cond.load16
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm2[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,1],xmm3[0,2]
; SSE2-NEXT: LBB7_14: ## %else17
; SSE2-NEXT: pextrw $7, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB7_16
; SSE2-NEXT: ## %bb.15: ## %cond.load19
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm2[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,1],xmm0[2,0]
; SSE2-NEXT: LBB7_16: ## %else20
; SSE2-NEXT: movaps %xmm1, %xmm0
; SSE2-NEXT: movaps %xmm2, %xmm1
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_v8i32_v8i1:
; SSE42: ## %bb.0:
; SSE42-NEXT: pextrb $0, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB7_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: pinsrd $0, (%rdi), %xmm1
; SSE42-NEXT: LBB7_2: ## %else
; SSE42-NEXT: pextrb $2, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB7_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: pinsrd $1, 4(%rdi), %xmm1
; SSE42-NEXT: LBB7_4: ## %else2
; SSE42-NEXT: pextrb $4, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB7_6
; SSE42-NEXT: ## %bb.5: ## %cond.load4
; SSE42-NEXT: pinsrd $2, 8(%rdi), %xmm1
; SSE42-NEXT: LBB7_6: ## %else5
; SSE42-NEXT: pextrb $6, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB7_8
; SSE42-NEXT: ## %bb.7: ## %cond.load7
; SSE42-NEXT: pinsrd $3, 12(%rdi), %xmm1
; SSE42-NEXT: LBB7_8: ## %else8
; SSE42-NEXT: pextrb $8, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB7_10
; SSE42-NEXT: ## %bb.9: ## %cond.load10
; SSE42-NEXT: pinsrd $0, 16(%rdi), %xmm2
; SSE42-NEXT: LBB7_10: ## %else11
; SSE42-NEXT: pextrb $10, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB7_12
; SSE42-NEXT: ## %bb.11: ## %cond.load13
; SSE42-NEXT: pinsrd $1, 20(%rdi), %xmm2
; SSE42-NEXT: LBB7_12: ## %else14
; SSE42-NEXT: pextrb $12, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB7_14
; SSE42-NEXT: ## %bb.13: ## %cond.load16
; SSE42-NEXT: pinsrd $2, 24(%rdi), %xmm2
; SSE42-NEXT: LBB7_14: ## %else17
; SSE42-NEXT: pextrb $14, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB7_16
; SSE42-NEXT: ## %bb.15: ## %cond.load19
; SSE42-NEXT: pinsrd $3, 28(%rdi), %xmm2
; SSE42-NEXT: LBB7_16: ## %else20
; SSE42-NEXT: movdqa %xmm1, %xmm0
; SSE42-NEXT: movdqa %xmm2, %xmm1
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_v8i32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpmovzxwd {{.*#+}} xmm2 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
; AVX1-NEXT: vpslld $31, %xmm2, %xmm2
; AVX1-NEXT: vpsrad $31, %xmm2, %xmm2
; AVX1-NEXT: vpunpckhwd {{.*#+}} xmm0 = xmm0[4,4,5,5,6,6,7,7]
; AVX1-NEXT: vpslld $31, %xmm0, %xmm0
; AVX1-NEXT: vpsrad $31, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm2, %ymm0
; AVX1-NEXT: vmaskmovps (%rdi), %ymm0, %ymm2
; AVX1-NEXT: vblendvps %ymm0, %ymm2, %ymm1, %ymm0
; AVX1-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_v8i32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpmovzxwd {{.*#+}} ymm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero,xmm0[4],zero,xmm0[5],zero,xmm0[6],zero,xmm0[7],zero
; AVX2-NEXT: vpslld $31, %ymm0, %ymm0
; AVX2-NEXT: vpsrad $31, %ymm0, %ymm0
; AVX2-NEXT: vpmaskmovd (%rdi), %ymm0, %ymm2
; AVX2-NEXT: vblendvps %ymm0, %ymm2, %ymm1, %ymm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_v8i32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm1 killed $ymm1 def $zmm1
[X86] Turn X86ISD::VSEXT into X86ISD::VZEXT if the upper bits aren't demanded. This makes X86ISD::VSEXT more similar to ISD::SIGN_EXTEND and ISD::ZERO_EXTEND. I'm hoping to replace X86ISD::VSEXT/VZEXT with target independent nodes. Making the target specific nodes similar to the target independent nodes helps minimize test diffs in that patch. llvm-svn: 346539
2018-11-10 03:05:51 +08:00
; AVX512F-NEXT: vpmovzxwq {{.*#+}} zmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero,xmm0[2],zero,zero,zero,xmm0[3],zero,zero,zero,xmm0[4],zero,zero,zero,xmm0[5],zero,zero,zero,xmm0[6],zero,zero,zero,xmm0[7],zero,zero,zero
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: vpsllq $63, %zmm0, %zmm0
[X86][AVX512] Improve lowering of AVX512 test intrinsics Added TESTM and TESTNM to the list of instructions that already zeroing unused upper bits and does not need the redundant shift left and shift right instructions afterwards. Added a pattern for TESTM and TESTNM in iselLowering, so now icmp(neq,and(X,Y), 0) goes folds into TESTM and icmp(eq,and(X,Y), 0) goes folds into TESTNM This commit is a preparation for lowering the test and testn X86 intrinsics to IR. Differential Revision: https://reviews.llvm.org/D38732 llvm-svn: 317465
2017-11-06 17:22:38 +08:00
; AVX512F-NEXT: vptestmq %zmm0, %zmm0, %k1
[AVX-512] Teach two address instruction pass to replace masked move instructions with blendm instructions when its beneficial. Isel now selects masked move instructions for vselect instead of blendm. But sometimes it beneficial to register allocation to remove the tied register constraint by using blendm instructions. This also picks up cases where the masked move was created due to a masked load intrinsic. Differential Revision: https://reviews.llvm.org/D28454 llvm-svn: 292005
2017-01-14 15:50:52 +08:00
; AVX512F-NEXT: vpblendmd (%rdi), %zmm1, %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_v8i32_v8i1:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vpsllw $15, %xmm0, %xmm0
; AVX512VLBW-NEXT: vpmovw2m %xmm0, %k1
; AVX512VLBW-NEXT: vpblendmd (%rdi), %ymm1, %ymm0 {%k1}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <8 x i32> @llvm.masked.load.v8i32.p0v8i32(<8 x i32>* %addr, i32 4, <8 x i1>%mask, <8 x i32>%dst)
AVX-512: Fixed masked load / store instruction selection for KNL. Patterns were missing for KNL target for <8 x i32>, <8 x float> masked load/store. This intrinsic comes with all legal types: <8 x float> @llvm.masked.load.v8f32(<8 x float>* %addr, i32 align, <8 x i1> %mask, <8 x float> %passThru), but still requires lowering, because VMASKMOVPS, VMASKMOVDQU32 work with 512-bit vectors only. All data operands should be widened to 512-bit vector. The mask operand should be widened to v16i1 with zeroes. Differential Revision: http://reviews.llvm.org/D15265 llvm-svn: 254909
2015-12-07 21:39:24 +08:00
ret <8 x i32> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <8 x float> @load_zero_v8f32_v8i1(<8 x i1> %mask, <8 x float>* %addr) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_zero_v8f32_v8i1:
; SSE2: ## %bb.0:
; SSE2-NEXT: movdqa %xmm0, %xmm2
; SSE2-NEXT: movd %xmm0, %eax
; SSE2-NEXT: pxor %xmm0, %xmm0
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: xorps %xmm1, %xmm1
; SSE2-NEXT: je LBB8_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movd {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: xorps %xmm1, %xmm1
; SSE2-NEXT: LBB8_2: ## %else
; SSE2-NEXT: shrl $16, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB8_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm0[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[2,0],xmm0[2,3]
; SSE2-NEXT: movaps %xmm3, %xmm0
; SSE2-NEXT: LBB8_4: ## %else2
; SSE2-NEXT: pextrw $2, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB8_6
; SSE2-NEXT: ## %bb.5: ## %cond.load4
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm0[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,1],xmm3[0,2]
; SSE2-NEXT: LBB8_6: ## %else5
; SSE2-NEXT: pextrw $3, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB8_8
; SSE2-NEXT: ## %bb.7: ## %cond.load7
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm0[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,1],xmm3[2,0]
; SSE2-NEXT: LBB8_8: ## %else8
; SSE2-NEXT: pextrw $4, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB8_10
; SSE2-NEXT: ## %bb.9: ## %cond.load10
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm1 = xmm3[0],xmm1[1,2,3]
; SSE2-NEXT: LBB8_10: ## %else11
; SSE2-NEXT: pextrw $5, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB8_12
; SSE2-NEXT: ## %bb.11: ## %cond.load13
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm1[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[2,0],xmm1[2,3]
; SSE2-NEXT: movaps %xmm3, %xmm1
; SSE2-NEXT: LBB8_12: ## %else14
; SSE2-NEXT: pextrw $6, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB8_14
; SSE2-NEXT: ## %bb.13: ## %cond.load16
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm1[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm3[0,2]
; SSE2-NEXT: LBB8_14: ## %else17
; SSE2-NEXT: pextrw $7, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB8_16
; SSE2-NEXT: ## %bb.15: ## %cond.load19
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm1[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm2[2,0]
; SSE2-NEXT: LBB8_16: ## %else20
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_zero_v8f32_v8i1:
; SSE42: ## %bb.0:
; SSE42-NEXT: movdqa %xmm0, %xmm2
; SSE42-NEXT: pextrb $0, %xmm0, %eax
; SSE42-NEXT: pxor %xmm0, %xmm0
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: xorps %xmm1, %xmm1
; SSE42-NEXT: je LBB8_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movd {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE42-NEXT: xorps %xmm1, %xmm1
; SSE42-NEXT: LBB8_2: ## %else
; SSE42-NEXT: pextrb $2, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB8_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[2,3]
; SSE42-NEXT: LBB8_4: ## %else2
; SSE42-NEXT: pextrb $4, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB8_6
; SSE42-NEXT: ## %bb.5: ## %cond.load4
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
; SSE42-NEXT: LBB8_6: ## %else5
; SSE42-NEXT: pextrb $6, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB8_8
; SSE42-NEXT: ## %bb.7: ## %cond.load7
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
; SSE42-NEXT: LBB8_8: ## %else8
; SSE42-NEXT: pextrb $8, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB8_10
; SSE42-NEXT: ## %bb.9: ## %cond.load10
; SSE42-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE42-NEXT: blendps {{.*#+}} xmm1 = xmm3[0],xmm1[1,2,3]
; SSE42-NEXT: LBB8_10: ## %else11
; SSE42-NEXT: pextrb $10, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB8_12
; SSE42-NEXT: ## %bb.11: ## %cond.load13
; SSE42-NEXT: insertps {{.*#+}} xmm1 = xmm1[0],mem[0],xmm1[2,3]
; SSE42-NEXT: LBB8_12: ## %else14
; SSE42-NEXT: pextrb $12, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB8_14
; SSE42-NEXT: ## %bb.13: ## %cond.load16
; SSE42-NEXT: insertps {{.*#+}} xmm1 = xmm1[0,1],mem[0],xmm1[3]
; SSE42-NEXT: LBB8_14: ## %else17
; SSE42-NEXT: pextrb $14, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB8_16
; SSE42-NEXT: ## %bb.15: ## %cond.load19
; SSE42-NEXT: insertps {{.*#+}} xmm1 = xmm1[0,1,2],mem[0]
; SSE42-NEXT: LBB8_16: ## %else20
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_zero_v8f32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpmovzxwd {{.*#+}} xmm1 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
; AVX1-NEXT: vpslld $31, %xmm1, %xmm1
; AVX1-NEXT: vpsrad $31, %xmm1, %xmm1
; AVX1-NEXT: vpunpckhwd {{.*#+}} xmm0 = xmm0[4,4,5,5,6,6,7,7]
; AVX1-NEXT: vpslld $31, %xmm0, %xmm0
; AVX1-NEXT: vpsrad $31, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; AVX1-NEXT: vmaskmovps (%rdi), %ymm0, %ymm0
; AVX1-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_zero_v8f32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpmovzxwd {{.*#+}} ymm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero,xmm0[4],zero,xmm0[5],zero,xmm0[6],zero,xmm0[7],zero
; AVX2-NEXT: vpslld $31, %ymm0, %ymm0
; AVX2-NEXT: vpsrad $31, %ymm0, %ymm0
; AVX2-NEXT: vmaskmovps (%rdi), %ymm0, %ymm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_zero_v8f32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
[X86] Turn X86ISD::VSEXT into X86ISD::VZEXT if the upper bits aren't demanded. This makes X86ISD::VSEXT more similar to ISD::SIGN_EXTEND and ISD::ZERO_EXTEND. I'm hoping to replace X86ISD::VSEXT/VZEXT with target independent nodes. Making the target specific nodes similar to the target independent nodes helps minimize test diffs in that patch. llvm-svn: 346539
2018-11-10 03:05:51 +08:00
; AVX512F-NEXT: vpmovzxwq {{.*#+}} zmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero,xmm0[2],zero,zero,zero,xmm0[3],zero,zero,zero,xmm0[4],zero,zero,zero,xmm0[5],zero,zero,zero,xmm0[6],zero,zero,zero,xmm0[7],zero,zero,zero
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: vpsllq $63, %zmm0, %zmm0
[X86][AVX512] Improve lowering of AVX512 test intrinsics Added TESTM and TESTNM to the list of instructions that already zeroing unused upper bits and does not need the redundant shift left and shift right instructions afterwards. Added a pattern for TESTM and TESTNM in iselLowering, so now icmp(neq,and(X,Y), 0) goes folds into TESTM and icmp(eq,and(X,Y), 0) goes folds into TESTNM This commit is a preparation for lowering the test and testn X86 intrinsics to IR. Differential Revision: https://reviews.llvm.org/D38732 llvm-svn: 317465
2017-11-06 17:22:38 +08:00
; AVX512F-NEXT: vptestmq %zmm0, %zmm0, %k1
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: vmovups (%rdi), %zmm0 {%k1} {z}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_zero_v8f32_v8i1:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vpsllw $15, %xmm0, %xmm0
; AVX512VLBW-NEXT: vpmovw2m %xmm0, %k1
; AVX512VLBW-NEXT: vmovaps (%rdi), %ymm0 {%k1} {z}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <8 x float> @llvm.masked.load.v8f32.p0v8f32(<8 x float>* %addr, i32 32, <8 x i1> %mask, <8 x float> zeroinitializer)
AVX-512: Fixed masked load / store instruction selection for KNL. Patterns were missing for KNL target for <8 x i32>, <8 x float> masked load/store. This intrinsic comes with all legal types: <8 x float> @llvm.masked.load.v8f32(<8 x float>* %addr, i32 align, <8 x i1> %mask, <8 x float> %passThru), but still requires lowering, because VMASKMOVPS, VMASKMOVDQU32 work with 512-bit vectors only. All data operands should be widened to 512-bit vector. The mask operand should be widened to v16i1 with zeroes. Differential Revision: http://reviews.llvm.org/D15265 llvm-svn: 254909
2015-12-07 21:39:24 +08:00
ret <8 x float> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <8 x i32> @load_zero_v8i32_v8i1(<8 x i1> %mask, <8 x i32>* %addr) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_zero_v8i32_v8i1:
; SSE2: ## %bb.0:
; SSE2-NEXT: movdqa %xmm0, %xmm2
; SSE2-NEXT: movd %xmm0, %eax
; SSE2-NEXT: pxor %xmm0, %xmm0
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: xorps %xmm1, %xmm1
; SSE2-NEXT: je LBB9_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movd {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: xorps %xmm1, %xmm1
; SSE2-NEXT: LBB9_2: ## %else
; SSE2-NEXT: shrl $16, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB9_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm0[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[2,0],xmm0[2,3]
; SSE2-NEXT: movaps %xmm3, %xmm0
; SSE2-NEXT: LBB9_4: ## %else2
; SSE2-NEXT: pextrw $2, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB9_6
; SSE2-NEXT: ## %bb.5: ## %cond.load4
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm0[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,1],xmm3[0,2]
; SSE2-NEXT: LBB9_6: ## %else5
; SSE2-NEXT: pextrw $3, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB9_8
; SSE2-NEXT: ## %bb.7: ## %cond.load7
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm0[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,1],xmm3[2,0]
; SSE2-NEXT: LBB9_8: ## %else8
; SSE2-NEXT: pextrw $4, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB9_10
; SSE2-NEXT: ## %bb.9: ## %cond.load10
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm1 = xmm3[0],xmm1[1,2,3]
; SSE2-NEXT: LBB9_10: ## %else11
; SSE2-NEXT: pextrw $5, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB9_12
; SSE2-NEXT: ## %bb.11: ## %cond.load13
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm1[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[2,0],xmm1[2,3]
; SSE2-NEXT: movaps %xmm3, %xmm1
; SSE2-NEXT: LBB9_12: ## %else14
; SSE2-NEXT: pextrw $6, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB9_14
; SSE2-NEXT: ## %bb.13: ## %cond.load16
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm1[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm3[0,2]
; SSE2-NEXT: LBB9_14: ## %else17
; SSE2-NEXT: pextrw $7, %xmm2, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB9_16
; SSE2-NEXT: ## %bb.15: ## %cond.load19
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm1[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm2[2,0]
; SSE2-NEXT: LBB9_16: ## %else20
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_zero_v8i32_v8i1:
; SSE42: ## %bb.0:
; SSE42-NEXT: movdqa %xmm0, %xmm2
; SSE42-NEXT: pextrb $0, %xmm0, %eax
; SSE42-NEXT: pxor %xmm0, %xmm0
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: pxor %xmm1, %xmm1
; SSE42-NEXT: je LBB9_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movd {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE42-NEXT: pxor %xmm1, %xmm1
; SSE42-NEXT: LBB9_2: ## %else
; SSE42-NEXT: pextrb $2, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB9_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: pinsrd $1, 4(%rdi), %xmm0
; SSE42-NEXT: LBB9_4: ## %else2
; SSE42-NEXT: pextrb $4, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB9_6
; SSE42-NEXT: ## %bb.5: ## %cond.load4
; SSE42-NEXT: pinsrd $2, 8(%rdi), %xmm0
; SSE42-NEXT: LBB9_6: ## %else5
; SSE42-NEXT: pextrb $6, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB9_8
; SSE42-NEXT: ## %bb.7: ## %cond.load7
; SSE42-NEXT: pinsrd $3, 12(%rdi), %xmm0
; SSE42-NEXT: LBB9_8: ## %else8
; SSE42-NEXT: pextrb $8, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB9_10
; SSE42-NEXT: ## %bb.9: ## %cond.load10
; SSE42-NEXT: pinsrd $0, 16(%rdi), %xmm1
; SSE42-NEXT: LBB9_10: ## %else11
; SSE42-NEXT: pextrb $10, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB9_12
; SSE42-NEXT: ## %bb.11: ## %cond.load13
; SSE42-NEXT: pinsrd $1, 20(%rdi), %xmm1
; SSE42-NEXT: LBB9_12: ## %else14
; SSE42-NEXT: pextrb $12, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB9_14
; SSE42-NEXT: ## %bb.13: ## %cond.load16
; SSE42-NEXT: pinsrd $2, 24(%rdi), %xmm1
; SSE42-NEXT: LBB9_14: ## %else17
; SSE42-NEXT: pextrb $14, %xmm2, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB9_16
; SSE42-NEXT: ## %bb.15: ## %cond.load19
; SSE42-NEXT: pinsrd $3, 28(%rdi), %xmm1
; SSE42-NEXT: LBB9_16: ## %else20
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_zero_v8i32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpmovzxwd {{.*#+}} xmm1 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
; AVX1-NEXT: vpslld $31, %xmm1, %xmm1
; AVX1-NEXT: vpsrad $31, %xmm1, %xmm1
; AVX1-NEXT: vpunpckhwd {{.*#+}} xmm0 = xmm0[4,4,5,5,6,6,7,7]
; AVX1-NEXT: vpslld $31, %xmm0, %xmm0
; AVX1-NEXT: vpsrad $31, %xmm0, %xmm0
; AVX1-NEXT: vinsertf128 $1, %xmm0, %ymm1, %ymm0
; AVX1-NEXT: vmaskmovps (%rdi), %ymm0, %ymm0
; AVX1-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_zero_v8i32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpmovzxwd {{.*#+}} ymm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero,xmm0[4],zero,xmm0[5],zero,xmm0[6],zero,xmm0[7],zero
; AVX2-NEXT: vpslld $31, %ymm0, %ymm0
; AVX2-NEXT: vpsrad $31, %ymm0, %ymm0
; AVX2-NEXT: vpmaskmovd (%rdi), %ymm0, %ymm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_zero_v8i32_v8i1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
[X86] Turn X86ISD::VSEXT into X86ISD::VZEXT if the upper bits aren't demanded. This makes X86ISD::VSEXT more similar to ISD::SIGN_EXTEND and ISD::ZERO_EXTEND. I'm hoping to replace X86ISD::VSEXT/VZEXT with target independent nodes. Making the target specific nodes similar to the target independent nodes helps minimize test diffs in that patch. llvm-svn: 346539
2018-11-10 03:05:51 +08:00
; AVX512F-NEXT: vpmovzxwq {{.*#+}} zmm0 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero,xmm0[2],zero,zero,zero,xmm0[3],zero,zero,zero,xmm0[4],zero,zero,zero,xmm0[5],zero,zero,zero,xmm0[6],zero,zero,zero,xmm0[7],zero,zero,zero
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: vpsllq $63, %zmm0, %zmm0
[X86][AVX512] Improve lowering of AVX512 test intrinsics Added TESTM and TESTNM to the list of instructions that already zeroing unused upper bits and does not need the redundant shift left and shift right instructions afterwards. Added a pattern for TESTM and TESTNM in iselLowering, so now icmp(neq,and(X,Y), 0) goes folds into TESTM and icmp(eq,and(X,Y), 0) goes folds into TESTNM This commit is a preparation for lowering the test and testn X86 intrinsics to IR. Differential Revision: https://reviews.llvm.org/D38732 llvm-svn: 317465
2017-11-06 17:22:38 +08:00
; AVX512F-NEXT: vptestmq %zmm0, %zmm0, %k1
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: vmovdqu32 (%rdi), %zmm0 {%k1} {z}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_zero_v8i32_v8i1:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vpsllw $15, %xmm0, %xmm0
; AVX512VLBW-NEXT: vpmovw2m %xmm0, %k1
; AVX512VLBW-NEXT: vmovdqu32 (%rdi), %ymm0 {%k1} {z}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <8 x i32> @llvm.masked.load.v8i32.p0v8i32(<8 x i32>* %addr, i32 4, <8 x i1> %mask, <8 x i32> zeroinitializer)
AVX-512: Fixed masked load / store instruction selection for KNL. Patterns were missing for KNL target for <8 x i32>, <8 x float> masked load/store. This intrinsic comes with all legal types: <8 x float> @llvm.masked.load.v8f32(<8 x float>* %addr, i32 align, <8 x i1> %mask, <8 x float> %passThru), but still requires lowering, because VMASKMOVPS, VMASKMOVDQU32 work with 512-bit vectors only. All data operands should be widened to 512-bit vector. The mask operand should be widened to v16i1 with zeroes. Differential Revision: http://reviews.llvm.org/D15265 llvm-svn: 254909
2015-12-07 21:39:24 +08:00
ret <8 x i32> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <2 x float> @load_v2f32_v2i32(<2 x i32> %trigger, <2 x float>* %addr, <2 x float> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_v2f32_v2i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: pcmpeqd %xmm0, %xmm2
; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm2[1,0,3,2]
; SSE2-NEXT: pand %xmm2, %xmm0
; SSE2-NEXT: movd %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB10_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm1 = xmm2[0],xmm1[1,2,3]
; SSE2-NEXT: LBB10_2: ## %else
; SSE2-NEXT: pextrw $4, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB10_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[2,0],xmm1[2,3]
; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: LBB10_4: ## %else2
; SSE2-NEXT: movaps %xmm1, %xmm0
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_v2f32_v2i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: pxor %xmm2, %xmm2
[X86] Sign extend the 8-bit immediate when commuting blend instructions to match isel. Conversion from ConstantSDNode to MachineInstr sign extends immediates from their APInt representation to int64_t. This commit makes sure we do the same for commuting. The tests changes show how this improves CSE. This issue was made worse by the MachineCSE using commuteInstruction to undo a commute. So we virtually guarantee the sign extend from isel would be lost. The improved CSE also occurred with r354363, but that was reverted. I'm working to undo the revert, but wanted to get this fix in while it was easy to see the results. llvm-svn: 354724
2019-02-23 16:34:10 +08:00
; SSE42-NEXT: pblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; SSE42-NEXT: pcmpeqq %xmm2, %xmm0
; SSE42-NEXT: pextrb $0, %xmm0, %eax
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB10_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
[X86] Sign extend the 8-bit immediate when commuting blend instructions to match isel. Conversion from ConstantSDNode to MachineInstr sign extends immediates from their APInt representation to int64_t. This commit makes sure we do the same for commuting. The tests changes show how this improves CSE. This issue was made worse by the MachineCSE using commuteInstruction to undo a commute. So we virtually guarantee the sign extend from isel would be lost. The improved CSE also occurred with r354363, but that was reverted. I'm working to undo the revert, but wanted to get this fix in while it was easy to see the results. llvm-svn: 354724
2019-02-23 16:34:10 +08:00
; SSE42-NEXT: movd {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE42-NEXT: pblendw {{.*#+}} xmm1 = xmm2[0,1],xmm1[2,3,4,5,6,7]
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE42-NEXT: LBB10_2: ## %else
; SSE42-NEXT: pextrb $8, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB10_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: insertps {{.*#+}} xmm1 = xmm1[0],mem[0],xmm1[2,3]
; SSE42-NEXT: LBB10_4: ## %else2
; SSE42-NEXT: movaps %xmm1, %xmm0
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_v2f32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; AVX1-NEXT: vpcmpeqq %xmm2, %xmm0, %xmm0
[x86] use a single shufps when it can save instructions This is a tiny patch with a big pile of test changes. This partially fixes PR27885: https://llvm.org/bugs/show_bug.cgi?id=27885 My motivating case looks like this: - vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,2] - vpshufd {{.*#+}} xmm0 = xmm0[0,2,2,3] - vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm1[4,5,6,7] + vshufps {{.*#+}} xmm0 = xmm0[0,2],xmm1[0,2] And this happens several times in the diffs. For chips with domain-crossing penalties, the instruction count and size reduction should usually overcome any potential domain-crossing penalty due to using an FP op in a sequence of int ops. For chips such as recent Intel big cores and Atom, there is no domain-crossing penalty for shufps, so using shufps is a pure win. So the test case diffs all appear to be improvements except one test in vector-shuffle-combining.ll where we miss an opportunity to use a shift to generate zero elements and one test in combine-sra.ll where multiple uses prevent the expected shuffle combining. Differential Revision: https://reviews.llvm.org/D27692 llvm-svn: 289837
2016-12-16 02:03:38 +08:00
; AVX1-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,2],zero,zero
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vmaskmovps (%rdi), %xmm0, %xmm2
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX1-NEXT: vblendvps %xmm0, %xmm2, %xmm1, %xmm0
; AVX1-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_v2f32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX2-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm2[1],xmm0[2],xmm2[3]
; AVX2-NEXT: vpcmpeqq %xmm2, %xmm0, %xmm0
[x86] use a single shufps when it can save instructions This is a tiny patch with a big pile of test changes. This partially fixes PR27885: https://llvm.org/bugs/show_bug.cgi?id=27885 My motivating case looks like this: - vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,2] - vpshufd {{.*#+}} xmm0 = xmm0[0,2,2,3] - vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm1[4,5,6,7] + vshufps {{.*#+}} xmm0 = xmm0[0,2],xmm1[0,2] And this happens several times in the diffs. For chips with domain-crossing penalties, the instruction count and size reduction should usually overcome any potential domain-crossing penalty due to using an FP op in a sequence of int ops. For chips such as recent Intel big cores and Atom, there is no domain-crossing penalty for shufps, so using shufps is a pure win. So the test case diffs all appear to be improvements except one test in vector-shuffle-combining.ll where we miss an opportunity to use a shift to generate zero elements and one test in combine-sra.ll where multiple uses prevent the expected shuffle combining. Differential Revision: https://reviews.llvm.org/D27692 llvm-svn: 289837
2016-12-16 02:03:38 +08:00
; AVX2-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,2],zero,zero
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vmaskmovps (%rdi), %xmm0, %xmm2
; AVX2-NEXT: vblendvps %xmm0, %xmm2, %xmm1, %xmm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_v2f32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm1 killed $xmm1 def $zmm1
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX512F-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm2[1],xmm0[2],xmm2[3]
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmq %zmm0, %zmm0, %k0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: kshiftlw $14, %k0, %k0
; AVX512F-NEXT: kshiftrw $14, %k0, %k1
; AVX512F-NEXT: vblendmps (%rdi), %zmm1, %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 killed $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_v2f32_v2i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX512VLBW-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm2[1],xmm0[2],xmm2[3]
; AVX512VLBW-NEXT: vptestnmq %xmm0, %xmm0, %k1
; AVX512VLBW-NEXT: vblendmps (%rdi), %xmm1, %xmm0 {%k1}
; AVX512VLBW-NEXT: retq
Masked Load/Store - fixed a bug in type legalization. llvm-svn: 225441
2015-01-08 20:29:19 +08:00
%mask = icmp eq <2 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <2 x float> @llvm.masked.load.v2f32.p0v2f32(<2 x float>* %addr, i32 4, <2 x i1>%mask, <2 x float>%dst)
Masked Load/Store - fixed a bug in type legalization. llvm-svn: 225441
2015-01-08 20:29:19 +08:00
ret <2 x float> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <2 x i32> @load_v2i32_v2i32(<2 x i32> %trigger, <2 x i32>* %addr, <2 x i32> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_v2i32_v2i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: pcmpeqd %xmm0, %xmm2
; SSE2-NEXT: pshufd {{.*#+}} xmm0 = xmm2[1,0,3,2]
; SSE2-NEXT: pand %xmm2, %xmm0
; SSE2-NEXT: movd %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB11_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movl (%rdi), %eax
; SSE2-NEXT: movq %rax, %xmm2
; SSE2-NEXT: movsd {{.*#+}} xmm1 = xmm2[0],xmm1[1]
; SSE2-NEXT: LBB11_2: ## %else
; SSE2-NEXT: pextrw $4, %xmm0, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB11_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movl 4(%rdi), %eax
; SSE2-NEXT: movq %rax, %xmm0
; SSE2-NEXT: unpcklpd {{.*#+}} xmm1 = xmm1[0],xmm0[0]
; SSE2-NEXT: LBB11_4: ## %else2
; SSE2-NEXT: movapd %xmm1, %xmm0
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_v2i32_v2i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: pxor %xmm2, %xmm2
[X86] Sign extend the 8-bit immediate when commuting blend instructions to match isel. Conversion from ConstantSDNode to MachineInstr sign extends immediates from their APInt representation to int64_t. This commit makes sure we do the same for commuting. The tests changes show how this improves CSE. This issue was made worse by the MachineCSE using commuteInstruction to undo a commute. So we virtually guarantee the sign extend from isel would be lost. The improved CSE also occurred with r354363, but that was reverted. I'm working to undo the revert, but wanted to get this fix in while it was easy to see the results. llvm-svn: 354724
2019-02-23 16:34:10 +08:00
; SSE42-NEXT: pblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; SSE42-NEXT: pcmpeqq %xmm2, %xmm0
; SSE42-NEXT: pextrb $0, %xmm0, %eax
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB11_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movl (%rdi), %eax
; SSE42-NEXT: pinsrq $0, %rax, %xmm1
; SSE42-NEXT: LBB11_2: ## %else
; SSE42-NEXT: pextrb $8, %xmm0, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB11_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: movl 4(%rdi), %eax
; SSE42-NEXT: pinsrq $1, %rax, %xmm1
; SSE42-NEXT: LBB11_4: ## %else2
; SSE42-NEXT: movdqa %xmm1, %xmm0
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_v2i32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,3],xmm0[4,5],xmm2[6,7]
; AVX1-NEXT: vpcmpeqq %xmm2, %xmm0, %xmm0
[x86] use a single shufps when it can save instructions This is a tiny patch with a big pile of test changes. This partially fixes PR27885: https://llvm.org/bugs/show_bug.cgi?id=27885 My motivating case looks like this: - vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,2] - vpshufd {{.*#+}} xmm0 = xmm0[0,2,2,3] - vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm1[4,5,6,7] + vshufps {{.*#+}} xmm0 = xmm0[0,2],xmm1[0,2] And this happens several times in the diffs. For chips with domain-crossing penalties, the instruction count and size reduction should usually overcome any potential domain-crossing penalty due to using an FP op in a sequence of int ops. For chips such as recent Intel big cores and Atom, there is no domain-crossing penalty for shufps, so using shufps is a pure win. So the test case diffs all appear to be improvements except one test in vector-shuffle-combining.ll where we miss an opportunity to use a shift to generate zero elements and one test in combine-sra.ll where multiple uses prevent the expected shuffle combining. Differential Revision: https://reviews.llvm.org/D27692 llvm-svn: 289837
2016-12-16 02:03:38 +08:00
; AVX1-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,2],zero,zero
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vmaskmovps (%rdi), %xmm0, %xmm2
[X86] Teach execution domain fixing to convert between VPERMILPS and VPSHUFD. llvm-svn: 313507
2017-09-18 11:29:47 +08:00
; AVX1-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[0,2,2,3]
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vblendvps %xmm0, %xmm2, %xmm1, %xmm0
Recommit r352255 "[SelectionDAG][X86] Don't use SEXTLOAD for promoting masked loads in the type legalizer" This did not cause the buildbot failure it was previously reverted for. Original commit message: I'm not sure why we were using SEXTLOAD. EXTLOAD seems more appropriate since we don't care about the upper bits. This patch changes this and then modifies the X86 post legalization combine to emit a extending shuffle instead of a sign_extend_vector_inreg. Could maybe use an any_extend_vector_inre On AVX512 targets I think we might be able to use a masked vpmovzx and not have to expand this at all. llvm-svn: 352433
2019-01-29 05:38:47 +08:00
; AVX1-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX1-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_v2i32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX2-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm2[1],xmm0[2],xmm2[3]
; AVX2-NEXT: vpcmpeqq %xmm2, %xmm0, %xmm0
[x86] use a single shufps when it can save instructions This is a tiny patch with a big pile of test changes. This partially fixes PR27885: https://llvm.org/bugs/show_bug.cgi?id=27885 My motivating case looks like this: - vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,2] - vpshufd {{.*#+}} xmm0 = xmm0[0,2,2,3] - vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm1[4,5,6,7] + vshufps {{.*#+}} xmm0 = xmm0[0,2],xmm1[0,2] And this happens several times in the diffs. For chips with domain-crossing penalties, the instruction count and size reduction should usually overcome any potential domain-crossing penalty due to using an FP op in a sequence of int ops. For chips such as recent Intel big cores and Atom, there is no domain-crossing penalty for shufps, so using shufps is a pure win. So the test case diffs all appear to be improvements except one test in vector-shuffle-combining.ll where we miss an opportunity to use a shift to generate zero elements and one test in combine-sra.ll where multiple uses prevent the expected shuffle combining. Differential Revision: https://reviews.llvm.org/D27692 llvm-svn: 289837
2016-12-16 02:03:38 +08:00
; AVX2-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,2],zero,zero
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpmaskmovd (%rdi), %xmm0, %xmm2
[X86] Teach execution domain fixing to convert between VPERMILPS and VPSHUFD. llvm-svn: 313507
2017-09-18 11:29:47 +08:00
; AVX2-NEXT: vpermilps {{.*#+}} xmm1 = xmm1[0,2,2,3]
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vblendvps %xmm0, %xmm2, %xmm1, %xmm0
Recommit r352255 "[SelectionDAG][X86] Don't use SEXTLOAD for promoting masked loads in the type legalizer" This did not cause the buildbot failure it was previously reverted for. Original commit message: I'm not sure why we were using SEXTLOAD. EXTLOAD seems more appropriate since we don't care about the upper bits. This patch changes this and then modifies the X86 post legalization combine to emit a extending shuffle instead of a sign_extend_vector_inreg. Could maybe use an any_extend_vector_inre On AVX512 targets I think we might be able to use a masked vpmovzx and not have to expand this at all. llvm-svn: 352433
2019-01-29 05:38:47 +08:00
; AVX2-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_v2i32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX512F-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm2[1],xmm0[2],xmm2[3]
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmq %zmm0, %zmm0, %k0
; AVX512F-NEXT: vpshufd {{.*#+}} xmm0 = xmm1[0,2,2,3]
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: kshiftlw $14, %k0, %k0
; AVX512F-NEXT: kshiftrw $14, %k0, %k1
; AVX512F-NEXT: vmovdqu32 (%rdi), %zmm0 {%k1}
Recommit r352255 "[SelectionDAG][X86] Don't use SEXTLOAD for promoting masked loads in the type legalizer" This did not cause the buildbot failure it was previously reverted for. Original commit message: I'm not sure why we were using SEXTLOAD. EXTLOAD seems more appropriate since we don't care about the upper bits. This patch changes this and then modifies the X86 post legalization combine to emit a extending shuffle instead of a sign_extend_vector_inreg. Could maybe use an any_extend_vector_inre On AVX512 targets I think we might be able to use a masked vpmovzx and not have to expand this at all. llvm-svn: 352433
2019-01-29 05:38:47 +08:00
; AVX512F-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_v2i32_v2i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vpxor %xmm2, %xmm2, %xmm2
; AVX512VLBW-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm2[1],xmm0[2],xmm2[3]
; AVX512VLBW-NEXT: vptestnmq %xmm0, %xmm0, %k1
; AVX512VLBW-NEXT: vpshufd {{.*#+}} xmm0 = xmm1[0,2,2,3]
; AVX512VLBW-NEXT: vmovdqu32 (%rdi), %xmm0 {%k1}
Recommit r352255 "[SelectionDAG][X86] Don't use SEXTLOAD for promoting masked loads in the type legalizer" This did not cause the buildbot failure it was previously reverted for. Original commit message: I'm not sure why we were using SEXTLOAD. EXTLOAD seems more appropriate since we don't care about the upper bits. This patch changes this and then modifies the X86 post legalization combine to emit a extending shuffle instead of a sign_extend_vector_inreg. Could maybe use an any_extend_vector_inre On AVX512 targets I think we might be able to use a masked vpmovzx and not have to expand this at all. llvm-svn: 352433
2019-01-29 05:38:47 +08:00
; AVX512VLBW-NEXT: vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-NEXT: retq
Masked Load/Store - fixed a bug in type legalization. llvm-svn: 225441
2015-01-08 20:29:19 +08:00
%mask = icmp eq <2 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <2 x i32> @llvm.masked.load.v2i32.p0v2i32(<2 x i32>* %addr, i32 4, <2 x i1>%mask, <2 x i32>%dst)
Masked Load/Store - fixed a bug in type legalization. llvm-svn: 225441
2015-01-08 20:29:19 +08:00
ret <2 x i32> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <2 x float> @load_undef_v2f32_v2i32(<2 x i32> %trigger, <2 x float>* %addr) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_undef_v2f32_v2i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: pand {{.*}}(%rip), %xmm0
; SSE2-NEXT: pxor %xmm2, %xmm2
; SSE2-NEXT: pcmpeqd %xmm0, %xmm2
; SSE2-NEXT: pshufd {{.*#+}} xmm1 = xmm2[1,0,3,2]
; SSE2-NEXT: pand %xmm2, %xmm1
; SSE2-NEXT: movd %xmm1, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: ## implicit-def: $xmm0
; SSE2-NEXT: je LBB12_2
; SSE2-NEXT: ## %bb.1: ## %cond.load
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: LBB12_2: ## %else
; SSE2-NEXT: pextrw $4, %xmm1, %eax
; SSE2-NEXT: testb $1, %al
; SSE2-NEXT: je LBB12_4
; SSE2-NEXT: ## %bb.3: ## %cond.load1
; SSE2-NEXT: movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,0],xmm0[0,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[2,0],xmm0[2,3]
; SSE2-NEXT: movaps %xmm1, %xmm0
; SSE2-NEXT: LBB12_4: ## %else2
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_undef_v2f32_v2i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: movdqa %xmm0, %xmm1
[X86] Sign extend the 8-bit immediate when commuting blend instructions to match isel. Conversion from ConstantSDNode to MachineInstr sign extends immediates from their APInt representation to int64_t. This commit makes sure we do the same for commuting. The tests changes show how this improves CSE. This issue was made worse by the MachineCSE using commuteInstruction to undo a commute. So we virtually guarantee the sign extend from isel would be lost. The improved CSE also occurred with r354363, but that was reverted. I'm working to undo the revert, but wanted to get this fix in while it was easy to see the results. llvm-svn: 354724
2019-02-23 16:34:10 +08:00
; SSE42-NEXT: pxor %xmm0, %xmm0
; SSE42-NEXT: pblendw {{.*#+}} xmm1 = xmm1[0,1],xmm0[2,3],xmm1[4,5],xmm0[6,7]
; SSE42-NEXT: pcmpeqq %xmm0, %xmm1
; SSE42-NEXT: pextrb $0, %xmm1, %eax
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: ## implicit-def: $xmm0
; SSE42-NEXT: je LBB12_2
; SSE42-NEXT: ## %bb.1: ## %cond.load
; SSE42-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE42-NEXT: LBB12_2: ## %else
; SSE42-NEXT: pextrb $8, %xmm1, %eax
; SSE42-NEXT: testb $1, %al
; SSE42-NEXT: je LBB12_4
; SSE42-NEXT: ## %bb.3: ## %cond.load1
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[2,3]
; SSE42-NEXT: LBB12_4: ## %else2
; SSE42-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX1-LABEL: load_undef_v2f32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX1-NEXT: vpblendw {{.*#+}} xmm0 = xmm0[0,1],xmm1[2,3],xmm0[4,5],xmm1[6,7]
; AVX1-NEXT: vpcmpeqq %xmm1, %xmm0, %xmm0
[x86] use a single shufps when it can save instructions This is a tiny patch with a big pile of test changes. This partially fixes PR27885: https://llvm.org/bugs/show_bug.cgi?id=27885 My motivating case looks like this: - vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,2] - vpshufd {{.*#+}} xmm0 = xmm0[0,2,2,3] - vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm1[4,5,6,7] + vshufps {{.*#+}} xmm0 = xmm0[0,2],xmm1[0,2] And this happens several times in the diffs. For chips with domain-crossing penalties, the instruction count and size reduction should usually overcome any potential domain-crossing penalty due to using an FP op in a sequence of int ops. For chips such as recent Intel big cores and Atom, there is no domain-crossing penalty for shufps, so using shufps is a pure win. So the test case diffs all appear to be improvements except one test in vector-shuffle-combining.ll where we miss an opportunity to use a shift to generate zero elements and one test in combine-sra.ll where multiple uses prevent the expected shuffle combining. Differential Revision: https://reviews.llvm.org/D27692 llvm-svn: 289837
2016-12-16 02:03:38 +08:00
; AVX1-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,2],zero,zero
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX1-NEXT: vmaskmovps (%rdi), %xmm0, %xmm0
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX1-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX2-LABEL: load_undef_v2f32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX2-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2],xmm1[3]
; AVX2-NEXT: vpcmpeqq %xmm1, %xmm0, %xmm0
[x86] use a single shufps when it can save instructions This is a tiny patch with a big pile of test changes. This partially fixes PR27885: https://llvm.org/bugs/show_bug.cgi?id=27885 My motivating case looks like this: - vpshufd {{.*#+}} xmm1 = xmm1[0,1,0,2] - vpshufd {{.*#+}} xmm0 = xmm0[0,2,2,3] - vpblendw {{.*#+}} xmm0 = xmm0[0,1,2,3],xmm1[4,5,6,7] + vshufps {{.*#+}} xmm0 = xmm0[0,2],xmm1[0,2] And this happens several times in the diffs. For chips with domain-crossing penalties, the instruction count and size reduction should usually overcome any potential domain-crossing penalty due to using an FP op in a sequence of int ops. For chips such as recent Intel big cores and Atom, there is no domain-crossing penalty for shufps, so using shufps is a pure win. So the test case diffs all appear to be improvements except one test in vector-shuffle-combining.ll where we miss an opportunity to use a shift to generate zero elements and one test in combine-sra.ll where multiple uses prevent the expected shuffle combining. Differential Revision: https://reviews.llvm.org/D27692 llvm-svn: 289837
2016-12-16 02:03:38 +08:00
; AVX2-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,2],zero,zero
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
; AVX2-NEXT: vmaskmovps (%rdi), %xmm0, %xmm0
; AVX2-NEXT: retq
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_undef_v2f32_v2i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512F-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2],xmm1[3]
[X86] Use vptestm/vptestnm for comparisons with zero to avoid creating a zero vector. We can use the same input for both operands to get a free compare with zero. We already use this trick in a couple places where we explicitly create PTESTM with the same input twice. This generalizes it. I'm hoping to remove the ISD opcodes and move this to isel patterns like we do for scalar cmp/test. llvm-svn: 323605
2018-01-28 04:19:09 +08:00
; AVX512F-NEXT: vptestnmq %zmm0, %zmm0, %k0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: kshiftlw $14, %k0, %k0
; AVX512F-NEXT: kshiftrw $14, %k0, %k1
; AVX512F-NEXT: vmovups (%rdi), %zmm0 {%k1} {z}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 killed $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_undef_v2f32_v2i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: vpxor %xmm1, %xmm1, %xmm1
; AVX512VLBW-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm1[1],xmm0[2],xmm1[3]
; AVX512VLBW-NEXT: vptestnmq %xmm0, %xmm0, %k1
; AVX512VLBW-NEXT: vmovups (%rdi), %xmm0 {%k1} {z}
; AVX512VLBW-NEXT: retq
Masked Load/Store - fixed a bug in type legalization. llvm-svn: 225441
2015-01-08 20:29:19 +08:00
%mask = icmp eq <2 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <2 x float> @llvm.masked.load.v2f32.p0v2f32(<2 x float>* %addr, i32 4, <2 x i1>%mask, <2 x float>undef)
Masked Load/Store - fixed a bug in type legalization. llvm-svn: 225441
2015-01-08 20:29:19 +08:00
ret <2 x float> %res
}
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
define <4 x float> @load_all_v4f32_v4i32(<4 x i32> %trigger, <4 x float>* %addr) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: load_all_v4f32_v4i32:
; SSE: ## %bb.0:
; SSE-NEXT: movups (%rdi), %xmm0
; SSE-NEXT: retq
;
; AVX1OR2-LABEL: load_all_v4f32_v4i32:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vmovups (%rdi), %xmm0
; AVX1OR2-NEXT: retq
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
;
[X86] Update masked load/store test names llvm-svn: 346887
2018-11-15 04:25:50 +08:00
; AVX512F-LABEL: load_all_v4f32_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: movw $15, %ax
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovups (%rdi), %zmm0 {%k1} {z}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 killed $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: load_all_v4f32_v4i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: kxnorw %k0, %k0, %k1
; AVX512VLBW-NEXT: vmovups (%rdi), %xmm0 {%k1} {z}
; AVX512VLBW-NEXT: retq
Masked Load/Store optimization for scalar code When we have to convert the masked.load, masked.store to scalar code, we generate a chain of conditional basic blocks. I added optimization for constant mask vector. Differential Revision: http://reviews.llvm.org/D13855 llvm-svn: 250893
2015-10-21 19:50:54 +08:00
%mask = icmp eq <4 x i32> %trigger, zeroinitializer
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x float> @llvm.masked.load.v4f32.p0v4f32(<4 x float>* %addr, i32 4, <4 x i1><i1 true, i1 true, i1 true, i1 true>, <4 x float>undef)
Masked Load/Store optimization for scalar code When we have to convert the masked.load, masked.store to scalar code, we generate a chain of conditional basic blocks. I added optimization for constant mask vector. Differential Revision: http://reviews.llvm.org/D13855 llvm-svn: 250893
2015-10-21 19:50:54 +08:00
ret <4 x float> %res
}
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
;;; Loads with Constant Masks - these should be optimized to use something other than a variable blend.
; 128-bit FP vectors are supported with AVX.
define <4 x float> @mload_constmask_v4f32(<4 x float>* %addr, <4 x float> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: mload_constmask_v4f32:
; SSE2: ## %bb.0:
; SSE2-NEXT: movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
; SSE2-NEXT: movaps %xmm0, %xmm1
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm2[0,2]
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm1[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,1],xmm2[2,0]
; SSE2-NEXT: retq
;
; SSE42-LABEL: mload_constmask_v4f32:
; SSE42: ## %bb.0:
; SSE42-NEXT: movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
; SSE42-NEXT: blendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0,1,2],mem[0]
; SSE42-NEXT: retq
;
; AVX1OR2-LABEL: mload_constmask_v4f32:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vblendps {{.*#+}} xmm0 = mem[0],xmm0[1],mem[2,3]
; AVX1OR2-NEXT: retq
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
;
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX512F-LABEL: mload_constmask_v4f32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 def $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: movw $13, %ax
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovups (%rdi), %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 killed $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
AVX1 : Enable vector masked_load/store to AVX1. Use AVX1 FP instructions (vmaskmovps/pd) in place of the AVX2 int instructions (vpmaskmovd/q). Differential Revision: http://reviews.llvm.org/D16528 llvm-svn: 258675
2016-01-25 18:17:11 +08:00
; AVX512F-NEXT: retq
regenerate checks and note some near-term improvements For the moment, this file takes way too long to run (see inline comments), but that should be a temporary problem. The fact that the compile time is so slow for a target that doesn't support maskmov may be a bug worth investigating too. llvm-svn: 258629
2016-01-24 01:52:56 +08:00
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v4f32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $13, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovups (%rdi), %xmm0 {%k1}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x float> @llvm.masked.load.v4f32.p0v4f32(<4 x float>* %addr, i32 4, <4 x i1> <i1 1, i1 0, i1 1, i1 1>, <4 x float> %dst)
Masked Load/Store optimization for scalar code When we have to convert the masked.load, masked.store to scalar code, we generate a chain of conditional basic blocks. I added optimization for constant mask vector. Differential Revision: http://reviews.llvm.org/D13855 llvm-svn: 250893
2015-10-21 19:50:54 +08:00
ret <4 x float> %res
}
[X86][AVX] Add v2i64/v2f64 load tests Ensure these use insertions, not masked load ops llvm-svn: 322076
2018-01-09 21:35:18 +08:00
define <2 x double> @mload_constmask_v2f64(<2 x double>* %addr, <2 x double> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: mload_constmask_v2f64:
; SSE: ## %bb.0:
; SSE-NEXT: movhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
; SSE-NEXT: retq
;
[X86][AVX] Add v2i64/v2f64 load tests Ensure these use insertions, not masked load ops llvm-svn: 322076
2018-01-09 21:35:18 +08:00
; AVX-LABEL: mload_constmask_v2f64:
; AVX: ## %bb.0:
; AVX-NEXT: vmovhpd {{.*#+}} xmm0 = xmm0[0],mem[0]
; AVX-NEXT: retq
%res = call <2 x double> @llvm.masked.load.v2f64.p0v2f64(<2 x double>* %addr, i32 4, <2 x i1> <i1 0, i1 1>, <2 x double> %dst)
ret <2 x double> %res
}
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; 128-bit integer vectors are supported with AVX2.
define <4 x i32> @mload_constmask_v4i32(<4 x i32>* %addr, <4 x i32> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: mload_constmask_v4i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,0],xmm0[0,0]
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: movaps %xmm1, %xmm2
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[2,0],xmm0[0,2]
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm2[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[2,0],xmm0[2,0]
; SSE2-NEXT: movaps %xmm1, %xmm0
; SSE2-NEXT: retq
;
; SSE42-LABEL: mload_constmask_v4i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: pinsrd $1, 4(%rdi), %xmm0
; SSE42-NEXT: pinsrd $2, 8(%rdi), %xmm0
; SSE42-NEXT: pinsrd $3, 12(%rdi), %xmm0
; SSE42-NEXT: retq
;
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX1-LABEL: mload_constmask_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX1-NEXT: vmovaps {{.*#+}} xmm1 = [0,4294967295,4294967295,4294967295]
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
; AVX1-NEXT: vmaskmovps (%rdi), %xmm1, %xmm1
[X86][SSE] Add custom execution domain fixing for BLENDPD/BLENDPS/PBLENDD/PBLENDW (PR34873) Add support for custom execution domain fixing and implement support for BLENDPD/BLENDPS/PBLENDD/PBLENDW. Differential Revision: https://reviews.llvm.org/D42042 llvm-svn: 322524
2018-01-16 06:18:45 +08:00
; AVX1-NEXT: vblendps {{.*#+}} xmm0 = xmm0[0],xmm1[1,2,3]
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX1-NEXT: retq
;
; AVX2-LABEL: mload_constmask_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX2-NEXT: vmovdqa {{.*#+}} xmm1 = [0,4294967295,4294967295,4294967295]
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
; AVX2-NEXT: vpmaskmovd (%rdi), %xmm1, %xmm1
; AVX2-NEXT: vpblendd {{.*#+}} xmm0 = xmm0[0],xmm1[1,2,3]
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX2-NEXT: retq
;
; AVX512F-LABEL: mload_constmask_v4i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 def $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: movw $14, %ax
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovdqu32 (%rdi), %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $xmm0 killed $xmm0 killed $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: vzeroupper
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v4i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $14, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovdqu32 (%rdi), %xmm0 {%k1}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x i32> @llvm.masked.load.v4i32.p0v4i32(<4 x i32>* %addr, i32 4, <4 x i1> <i1 0, i1 1, i1 1, i1 1>, <4 x i32> %dst)
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
ret <4 x i32> %res
}
[X86][AVX] Add v2i64/v2f64 load tests Ensure these use insertions, not masked load ops llvm-svn: 322076
2018-01-09 21:35:18 +08:00
define <2 x i64> @mload_constmask_v2i64(<2 x i64>* %addr, <2 x i64> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: mload_constmask_v2i64:
; SSE2: ## %bb.0:
; SSE2-NEXT: movsd {{.*#+}} xmm1 = mem[0],zero
; SSE2-NEXT: movlhps {{.*#+}} xmm0 = xmm0[0],xmm1[0]
; SSE2-NEXT: retq
;
; SSE42-LABEL: mload_constmask_v2i64:
; SSE42: ## %bb.0:
; SSE42-NEXT: pinsrq $1, 8(%rdi), %xmm0
; SSE42-NEXT: retq
;
[X86][AVX] Add v2i64/v2f64 load tests Ensure these use insertions, not masked load ops llvm-svn: 322076
2018-01-09 21:35:18 +08:00
; AVX-LABEL: mload_constmask_v2i64:
; AVX: ## %bb.0:
; AVX-NEXT: vpinsrq $1, 8(%rdi), %xmm0, %xmm0
; AVX-NEXT: retq
%res = call <2 x i64> @llvm.masked.load.v2i64.p0v2i64(<2 x i64>* %addr, i32 4, <2 x i1> <i1 0, i1 1>, <2 x i64> %dst)
ret <2 x i64> %res
}
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; 256-bit FP vectors are supported with AVX.
define <8 x float> @mload_constmask_v8f32(<8 x float>* %addr, <8 x float> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: mload_constmask_v8f32:
; SSE2: ## %bb.0:
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm3[0,0]
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm0[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[2,0],xmm3[0,2]
; SSE2-NEXT: movaps %xmm2, %xmm0
; SSE2-NEXT: retq
;
; SSE42-LABEL: mload_constmask_v8f32:
; SSE42: ## %bb.0:
; SSE42-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE42-NEXT: blendps {{.*#+}} xmm0 = xmm2[0],xmm0[1,2,3]
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0],mem[0],xmm0[2,3]
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
; SSE42-NEXT: retq
;
; AVX1OR2-LABEL: mload_constmask_v8f32:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vmovaps {{.*#+}} ymm1 = [4294967295,4294967295,4294967295,0,0,0,0,0]
; AVX1OR2-NEXT: vmaskmovps (%rdi), %ymm1, %ymm1
; AVX1OR2-NEXT: vblendps {{.*#+}} ymm0 = ymm1[0,1,2],ymm0[3,4,5,6,7]
; AVX1OR2-NEXT: retq
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
;
; AVX512F-LABEL: mload_constmask_v8f32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 def $zmm0
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX512F-NEXT: movw $7, %ax
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovups (%rdi), %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v8f32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $7, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovups (%rdi), %ymm0 {%k1}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <8 x float> @llvm.masked.load.v8f32.p0v8f32(<8 x float>* %addr, i32 4, <8 x i1> <i1 1, i1 1, i1 1, i1 0, i1 0, i1 0, i1 0, i1 0>, <8 x float> %dst)
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
ret <8 x float> %res
}
define <4 x double> @mload_constmask_v4f64(<4 x double>* %addr, <4 x double> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: mload_constmask_v4f64:
; SSE: ## %bb.0:
; SSE-NEXT: movups (%rdi), %xmm0
; SSE-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE-NEXT: retq
;
; AVX1OR2-LABEL: mload_constmask_v4f64:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vmovapd {{.*#+}} ymm1 = [18446744073709551615,18446744073709551615,18446744073709551615,0]
; AVX1OR2-NEXT: vmaskmovpd (%rdi), %ymm1, %ymm1
; AVX1OR2-NEXT: vblendpd {{.*#+}} ymm0 = ymm1[0,1,2],ymm0[3]
; AVX1OR2-NEXT: retq
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
;
; AVX512F-LABEL: mload_constmask_v4f64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 def $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: movb $7, %al
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovupd (%rdi), %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v4f64:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $7, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovupd (%rdi), %ymm0 {%k1}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x double> @llvm.masked.load.v4f64.p0v4f64(<4 x double>* %addr, i32 4, <4 x i1> <i1 1, i1 1, i1 1, i1 0>, <4 x double> %dst)
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
ret <4 x double> %res
}
; 256-bit integer vectors are supported with AVX2.
define <8 x i32> @mload_constmask_v8i32(<8 x i32>* %addr, <8 x i32> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: mload_constmask_v8i32:
; SSE2: ## %bb.0:
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm2 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[0,0],xmm3[0,0]
; SSE2-NEXT: movss {{.*#+}} xmm3 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm3 = xmm3[0,0],xmm0[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm2 = xmm2[2,0],xmm3[0,2]
; SSE2-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,0],xmm1[2,0]
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,1],xmm0[2,0]
; SSE2-NEXT: movaps %xmm2, %xmm0
; SSE2-NEXT: retq
;
; SSE42-LABEL: mload_constmask_v8i32:
; SSE42: ## %bb.0:
; SSE42-NEXT: pinsrd $0, (%rdi), %xmm0
; SSE42-NEXT: pinsrd $1, 4(%rdi), %xmm0
; SSE42-NEXT: pinsrd $2, 8(%rdi), %xmm0
; SSE42-NEXT: pinsrd $3, 28(%rdi), %xmm1
; SSE42-NEXT: retq
;
; AVX1OR2-LABEL: mload_constmask_v8i32:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vblendps {{.*#+}} ymm0 = mem[0,1,2],ymm0[3,4,5,6],mem[7]
; AVX1OR2-NEXT: retq
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
;
; AVX512F-LABEL: mload_constmask_v8i32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 def $zmm0
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX512F-NEXT: movw $135, %ax
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovdqu32 (%rdi), %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v8i32:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $-121, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovdqu32 (%rdi), %ymm0 {%k1}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <8 x i32> @llvm.masked.load.v8i32.p0v8i32(<8 x i32>* %addr, i32 4, <8 x i1> <i1 1, i1 1, i1 1, i1 0, i1 0, i1 0, i1 0, i1 1>, <8 x i32> %dst)
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
ret <8 x i32> %res
}
define <4 x i64> @mload_constmask_v4i64(<4 x i64>* %addr, <4 x i64> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: mload_constmask_v4i64:
; SSE2: ## %bb.0:
; SSE2-NEXT: movlpd {{.*#+}} xmm0 = mem[0],xmm0[1]
; SSE2-NEXT: movsd {{.*#+}} xmm2 = mem[0],zero
; SSE2-NEXT: movlhps {{.*#+}} xmm1 = xmm1[0],xmm2[0]
; SSE2-NEXT: retq
;
; SSE42-LABEL: mload_constmask_v4i64:
; SSE42: ## %bb.0:
; SSE42-NEXT: pinsrq $0, (%rdi), %xmm0
; SSE42-NEXT: pinsrq $1, 24(%rdi), %xmm1
; SSE42-NEXT: retq
;
; AVX1OR2-LABEL: mload_constmask_v4i64:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vblendps {{.*#+}} ymm0 = mem[0,1],ymm0[2,3,4,5],mem[6,7]
; AVX1OR2-NEXT: retq
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
;
; AVX512F-LABEL: mload_constmask_v4i64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 def $zmm0
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: movb $9, %al
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovdqu64 (%rdi), %zmm0 {%k1}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v4i64:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $9, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovdqu64 (%rdi), %ymm0 {%k1}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x i64> @llvm.masked.load.v4i64.p0v4i64(<4 x i64>* %addr, i32 4, <4 x i1> <i1 1, i1 0, i1 0, i1 1>, <4 x i64> %dst)
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
ret <4 x i64> %res
}
; 512-bit FP vectors are supported with AVX512.
define <8 x double> @mload_constmask_v8f64(<8 x double>* %addr, <8 x double> %dst) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: mload_constmask_v8f64:
; SSE: ## %bb.0:
; SSE-NEXT: movups (%rdi), %xmm0
; SSE-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE-NEXT: movhpd {{.*#+}} xmm3 = xmm3[0],mem[0]
; SSE-NEXT: retq
;
; AVX1OR2-LABEL: mload_constmask_v8f64:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vblendps {{.*#+}} ymm0 = mem[0,1,2,3,4,5],ymm0[6,7]
; AVX1OR2-NEXT: vblendps {{.*#+}} ymm1 = ymm1[0,1,2,3,4,5],mem[6,7]
; AVX1OR2-NEXT: retq
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
;
[AVX-512] Fix accidental uses of AH/BH/CH/DH after copies to/from mask registers We've had several bugs(PR32256, PR32241) recently that resulted from usages of AH/BH/CH/DH either before or after a copy to/from a mask register. This ultimately occurs because we create COPY_TO_REGCLASS with VK1 and GR8. Then in CopyToFromAsymmetricReg in X86InstrInfo we find a 32-bit super register for the GR8 to emit the KMOV with. But as these tests are demonstrating, its possible for the GR8 register to be a high register and we end up doing an accidental extra or insert from bits 15:8. I think the best way forward is to stop making copies directly between mask registers and GR8/GR16. Instead I think we should restrict to only copies between mask registers and GR32/GR64 and use EXTRACT_SUBREG/INSERT_SUBREG to handle the conversion from GR32 to GR16/8 or vice versa. Unfortunately, this complicates fastisel a bit more now to create the subreg extracts where we used to create GR8 copies. We can probably make a helper function to bring down the repitition. This does result in KMOVD being used for copies when BWI is available because we don't know the original mask register size. This caused a lot of deltas on tests because we have to split the checks for KMOVD vs KMOVW based on BWI. Differential Revision: https://reviews.llvm.org/D30968 llvm-svn: 298928
2017-03-29 00:35:29 +08:00
; AVX512F-LABEL: mload_constmask_v8f64:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
[AVX-512] Fix accidental uses of AH/BH/CH/DH after copies to/from mask registers We've had several bugs(PR32256, PR32241) recently that resulted from usages of AH/BH/CH/DH either before or after a copy to/from a mask register. This ultimately occurs because we create COPY_TO_REGCLASS with VK1 and GR8. Then in CopyToFromAsymmetricReg in X86InstrInfo we find a 32-bit super register for the GR8 to emit the KMOV with. But as these tests are demonstrating, its possible for the GR8 register to be a high register and we end up doing an accidental extra or insert from bits 15:8. I think the best way forward is to stop making copies directly between mask registers and GR8/GR16. Instead I think we should restrict to only copies between mask registers and GR32/GR64 and use EXTRACT_SUBREG/INSERT_SUBREG to handle the conversion from GR32 to GR16/8 or vice versa. Unfortunately, this complicates fastisel a bit more now to create the subreg extracts where we used to create GR8 copies. We can probably make a helper function to bring down the repitition. This does result in KMOVD being used for copies when BWI is available because we don't know the original mask register size. This caused a lot of deltas on tests because we have to split the checks for KMOVD vs KMOVW based on BWI. Differential Revision: https://reviews.llvm.org/D30968 llvm-svn: 298928
2017-03-29 00:35:29 +08:00
; AVX512F-NEXT: movb $-121, %al
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovupd (%rdi), %zmm0 {%k1}
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v8f64:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $-121, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovupd (%rdi), %zmm0 {%k1}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <8 x double> @llvm.masked.load.v8f64.p0v8f64(<8 x double>* %addr, i32 4, <8 x i1> <i1 1, i1 1, i1 1, i1 0, i1 0, i1 0, i1 0, i1 1>, <8 x double> %dst)
[x86] add tests for masked loads with constant masks llvm-svn: 262758
2016-03-05 07:28:07 +08:00
ret <8 x double> %res
}
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
; If the pass-through operand is undef, no blend is needed.
define <4 x double> @mload_constmask_v4f64_undef_passthrough(<4 x double>* %addr) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: mload_constmask_v4f64_undef_passthrough:
; SSE: ## %bb.0:
; SSE-NEXT: movsd {{.*#+}} xmm1 = mem[0],zero
; SSE-NEXT: movups (%rdi), %xmm0
; SSE-NEXT: retq
;
; AVX1OR2-LABEL: mload_constmask_v4f64_undef_passthrough:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vmovapd {{.*#+}} ymm0 = [18446744073709551615,18446744073709551615,18446744073709551615,0]
; AVX1OR2-NEXT: vmaskmovpd (%rdi), %ymm0, %ymm0
; AVX1OR2-NEXT: retq
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
;
; AVX512F-LABEL: mload_constmask_v4f64_undef_passthrough:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: movb $7, %al
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovupd (%rdi), %zmm0 {%k1} {z}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v4f64_undef_passthrough:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $7, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovupd (%rdi), %ymm0 {%k1} {z}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x double> @llvm.masked.load.v4f64.p0v4f64(<4 x double>* %addr, i32 4, <4 x i1> <i1 1, i1 1, i1 1, i1 0>, <4 x double> undef)
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
ret <4 x double> %res
}
define <4 x i64> @mload_constmask_v4i64_undef_passthrough(<4 x i64>* %addr) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: mload_constmask_v4i64_undef_passthrough:
; SSE: ## %bb.0:
; SSE-NEXT: movsd {{.*#+}} xmm1 = mem[0],zero
; SSE-NEXT: movq {{.*#+}} xmm0 = mem[0],zero
; SSE-NEXT: pshufd {{.*#+}} xmm0 = xmm0[0,1,0,1]
; SSE-NEXT: retq
;
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
; AVX1-LABEL: mload_constmask_v4i64_undef_passthrough:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
; AVX1-NEXT: vmovapd {{.*#+}} ymm0 = [0,18446744073709551615,18446744073709551615,0]
; AVX1-NEXT: vmaskmovpd (%rdi), %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: mload_constmask_v4i64_undef_passthrough:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
; AVX2-NEXT: vmovdqa {{.*#+}} ymm0 = [0,18446744073709551615,18446744073709551615,0]
; AVX2-NEXT: vpmaskmovq (%rdi), %ymm0, %ymm0
; AVX2-NEXT: retq
;
; AVX512F-LABEL: mload_constmask_v4i64_undef_passthrough:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512F: ## %bb.0:
[X86] Make v2i1 and v4i1 legal types without VLX Summary: There are few oddities that occur due to v1i1, v8i1, v16i1 being legal without v2i1 and v4i1 being legal when we don't have VLX. Particularly during legalization of v2i32/v4i32/v2i64/v4i64 masked gather/scatter/load/store. We end up promoting the mask argument to these during type legalization and then have to widen the promoted type to v8iX/v16iX and truncate it to get the element size back down to v8i1/v16i1 to use a 512-bit operation. Since need to fill the upper bits of the mask we have to fill with 0s at the promoted type. It would be better if we could just have the v2i1/v4i1 types as legal so they don't undergo any promotion. Then we can just widen with 0s directly in a k register. There are no real v4i1/v2i1 instructions anyway. Everything is done on a larger register anyway. This also fixes an issue that we couldn't implement a masked vextractf32x4 from zmm to xmm properly. We now have to support widening more compares to 512-bit to get a mask result out so new tablegen patterns got added. I had to hack the legalizer for widening the operand of a setcc a bit so it didn't try create a setcc returning v4i32, extract from it, then try to promote it using a sign extend to v2i1. Now we create the setcc with v4i1 if the original setcc's result type is v2i1. Then extract that and don't sign extend it at all. There's definitely room for improvement with some follow up patches. Reviewers: RKSimon, zvi, guyblank Reviewed By: RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D41560 llvm-svn: 321967
2018-01-08 02:20:37 +08:00
; AVX512F-NEXT: movb $6, %al
; AVX512F-NEXT: kmovw %eax, %k1
; AVX512F-NEXT: vmovdqu64 (%rdi), %zmm0 {%k1} {z}
Followup on Proposal to move MIR physical register namespace to '$' sigil. Discussed here: http://lists.llvm.org/pipermail/llvm-dev/2018-January/120320.html In preparation for adding support for named vregs we are changing the sigil for physical registers in MIR to '$' from '%'. This will prevent name clashes of named physical register with named vregs. llvm-svn: 323922
2018-02-01 06:04:26 +08:00
; AVX512F-NEXT: ## kill: def $ymm0 killed $ymm0 killed $zmm0
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
; AVX512F-NEXT: retq
;
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; AVX512VLBW-LABEL: mload_constmask_v4i64_undef_passthrough:
; AVX512VLBW: ## %bb.0:
; AVX512VLBW-NEXT: movb $6, %al
; AVX512VLBW-NEXT: kmovd %eax, %k1
; AVX512VLBW-NEXT: vmovdqu64 (%rdi), %ymm0 {%k1} {z}
; AVX512VLBW-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x i64> @llvm.masked.load.v4i64.p0v4i64(<4 x i64>* %addr, i32 4, <4 x i1> <i1 0, i1 1, i1 1, i1 0>, <4 x i64> undef)
[x86, AVX] optimize masked loads with constant masks Instead of a variable-blend instruction, form a blend with immediate because those are always cheaper. Differential Revision: http://reviews.llvm.org/D17899 llvm-svn: 263067
2016-03-10 06:12:08 +08:00
ret <4 x i64> %res
}
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; When only one element of the mask is set, reduce to a scalar load.
define <4 x i32> @load_one_mask_bit_set1(<4 x i32>* %addr, <4 x i32> %val) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_one_mask_bit_set1:
; SSE2: ## %bb.0:
; SSE2-NEXT: movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
; SSE2-NEXT: movss {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_one_mask_bit_set1:
; SSE42: ## %bb.0:
; SSE42-NEXT: pinsrd $0, (%rdi), %xmm0
; SSE42-NEXT: retq
;
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX-LABEL: load_one_mask_bit_set1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: ## %bb.0:
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX-NEXT: vpinsrd $0, (%rdi), %xmm0, %xmm0
; AVX-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x i32> @llvm.masked.load.v4i32.p0v4i32(<4 x i32>* %addr, i32 4, <4 x i1><i1 true, i1 false, i1 false, i1 false>, <4 x i32> %val)
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
ret <4 x i32> %res
}
; Choose a different element to show that the correct address offset is produced.
define <4 x float> @load_one_mask_bit_set2(<4 x float>* %addr, <4 x float> %val) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_one_mask_bit_set2:
; SSE2: ## %bb.0:
; SSE2-NEXT: movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
; SSE2-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,0],xmm0[3,0]
; SSE2-NEXT: shufps {{.*#+}} xmm0 = xmm0[0,1],xmm1[0,2]
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_one_mask_bit_set2:
; SSE42: ## %bb.0:
; SSE42-NEXT: insertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
; SSE42-NEXT: retq
;
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX-LABEL: load_one_mask_bit_set2:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: ## %bb.0:
[AVX512] Add VLX 128/256-bit SET0 operations that encode to 128/256-bit EVEX encoded VPXORD so all 32 registers can be used. llvm-svn: 268884
2016-05-09 05:33:53 +08:00
; AVX-NEXT: vinsertps {{.*#+}} xmm0 = xmm0[0,1],mem[0],xmm0[3]
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x float> @llvm.masked.load.v4f32.p0v4f32(<4 x float>* %addr, i32 4, <4 x i1><i1 false, i1 false, i1 true, i1 false>, <4 x float> %val)
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
ret <4 x float> %res
}
; Choose a different scalar type and a high element of a 256-bit vector because AVX doesn't support those evenly.
define <4 x i64> @load_one_mask_bit_set3(<4 x i64>* %addr, <4 x i64> %val) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE2-LABEL: load_one_mask_bit_set3:
; SSE2: ## %bb.0:
; SSE2-NEXT: movlpd {{.*#+}} xmm1 = mem[0],xmm1[1]
; SSE2-NEXT: retq
;
; SSE42-LABEL: load_one_mask_bit_set3:
; SSE42: ## %bb.0:
; SSE42-NEXT: pinsrq $0, 16(%rdi), %xmm1
; SSE42-NEXT: retq
;
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX1-LABEL: load_one_mask_bit_set3:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX1: ## %bb.0:
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX1-NEXT: vextractf128 $1, %ymm0, %xmm1
; AVX1-NEXT: vpinsrq $0, 16(%rdi), %xmm1, %xmm1
; AVX1-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: load_one_mask_bit_set3:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX2: ## %bb.0:
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX2-NEXT: vextracti128 $1, %ymm0, %xmm1
; AVX2-NEXT: vpinsrq $0, 16(%rdi), %xmm1, %xmm1
; AVX2-NEXT: vinserti128 $1, %xmm1, %ymm0, %ymm0
; AVX2-NEXT: retq
;
[AVX-512] Teach EVEX to VEX conversion pass to handle VINSERT and VEXTRACT instructions. llvm-svn: 290869
2017-01-03 13:46:18 +08:00
; AVX512-LABEL: load_one_mask_bit_set3:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512: ## %bb.0:
[AVX-512] Teach EVEX to VEX conversion pass to handle VINSERT and VEXTRACT instructions. llvm-svn: 290869
2017-01-03 13:46:18 +08:00
; AVX512-NEXT: vextracti128 $1, %ymm0, %xmm1
; AVX512-NEXT: vpinsrq $0, 16(%rdi), %xmm1, %xmm1
; AVX512-NEXT: vinserti128 $1, %xmm1, %ymm0, %ymm0
; AVX512-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x i64> @llvm.masked.load.v4i64.p0v4i64(<4 x i64>* %addr, i32 4, <4 x i1><i1 false, i1 false, i1 true, i1 false>, <4 x i64> %val)
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
ret <4 x i64> %res
}
; Choose a different scalar type and a high element of a 256-bit vector because AVX doesn't support those evenly.
define <4 x double> @load_one_mask_bit_set4(<4 x double>* %addr, <4 x double> %val) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: load_one_mask_bit_set4:
; SSE: ## %bb.0:
; SSE-NEXT: movhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
; SSE-NEXT: retq
;
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX-LABEL: load_one_mask_bit_set4:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: ## %bb.0:
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX-NEXT: vextractf128 $1, %ymm0, %xmm1
[AVX512] Add VLX 128/256-bit SET0 operations that encode to 128/256-bit EVEX encoded VPXORD so all 32 registers can be used. llvm-svn: 268884
2016-05-09 05:33:53 +08:00
; AVX-NEXT: vmovhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX-NEXT: vinsertf128 $1, %xmm1, %ymm0, %ymm0
; AVX-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <4 x double> @llvm.masked.load.v4f64.p0v4f64(<4 x double>* %addr, i32 4, <4 x i1><i1 false, i1 false, i1 false, i1 true>, <4 x double> %val)
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
ret <4 x double> %res
}
; Try a 512-bit vector to make sure AVX doesn't die and AVX512 works as expected.
define <8 x double> @load_one_mask_bit_set5(<8 x double>* %addr, <8 x double> %val) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: load_one_mask_bit_set5:
; SSE: ## %bb.0:
; SSE-NEXT: movhpd {{.*#+}} xmm3 = xmm3[0],mem[0]
; SSE-NEXT: retq
;
; AVX1OR2-LABEL: load_one_mask_bit_set5:
; AVX1OR2: ## %bb.0:
; AVX1OR2-NEXT: vextractf128 $1, %ymm1, %xmm2
; AVX1OR2-NEXT: vmovhpd {{.*#+}} xmm2 = xmm2[0],mem[0]
; AVX1OR2-NEXT: vinsertf128 $1, %xmm2, %ymm1, %ymm1
; AVX1OR2-NEXT: retq
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
;
; AVX512-LABEL: load_one_mask_bit_set5:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX512: ## %bb.0:
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
; AVX512-NEXT: vextractf32x4 $3, %zmm0, %xmm1
; AVX512-NEXT: vmovhpd {{.*#+}} xmm1 = xmm1[0],mem[0]
; AVX512-NEXT: vinsertf32x4 $3, %xmm1, %zmm0, %zmm0
; AVX512-NEXT: retq
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
%res = call <8 x double> @llvm.masked.load.v8f64.p0v8f64(<8 x double>* %addr, i32 4, <8 x i1><i1 false, i1 false, i1 false, i1 false, i1 false, i1 false, i1 false, i1 true>, <8 x double> %val)
[x86] convert masked load of exactly one element to scalar load This is the load counterpart to the store optimization that was added in: http://reviews.llvm.org/rL260145 llvm-svn: 260325
2016-02-10 07:44:35 +08:00
ret <8 x double> %res
}
[X86] Add the test case from PR38986. The assembly for this test should be optimal now after changes to the ScalarizeMaskedMemIntrin patch. llvm-svn: 343281
2018-09-28 07:25:10 +08:00
define i32 @pr38986(i1 %c, i32* %p) {
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
; SSE-LABEL: pr38986:
; SSE: ## %bb.0:
; SSE-NEXT: testb $1, %dil
; SSE-NEXT: ## implicit-def: $eax
; SSE-NEXT: je LBB30_2
; SSE-NEXT: ## %bb.1: ## %cond.load
; SSE-NEXT: movl (%rsi), %eax
; SSE-NEXT: LBB30_2: ## %else
; SSE-NEXT: retq
;
[X86] Add the test case from PR38986. The assembly for this test should be optimal now after changes to the ScalarizeMaskedMemIntrin patch. llvm-svn: 343281
2018-09-28 07:25:10 +08:00
; AVX-LABEL: pr38986:
; AVX: ## %bb.0:
; AVX-NEXT: testb $1, %dil
; AVX-NEXT: ## implicit-def: $eax
[X86] Split masked load/store test files llvm-svn: 346889
2018-11-15 04:44:59 +08:00
; AVX-NEXT: je LBB30_2
[X86] Add the test case from PR38986. The assembly for this test should be optimal now after changes to the ScalarizeMaskedMemIntrin patch. llvm-svn: 343281
2018-09-28 07:25:10 +08:00
; AVX-NEXT: ## %bb.1: ## %cond.load
; AVX-NEXT: movl (%rsi), %eax
[X86] Split masked load/store test files llvm-svn: 346889
2018-11-15 04:44:59 +08:00
; AVX-NEXT: LBB30_2: ## %else
[X86] Add the test case from PR38986. The assembly for this test should be optimal now after changes to the ScalarizeMaskedMemIntrin patch. llvm-svn: 343281
2018-09-28 07:25:10 +08:00
; AVX-NEXT: retq
%vc = insertelement <1 x i1> undef, i1 %c, i32 0
%vp = bitcast i32* %p to <1 x i32>*
%L = call <1 x i32> @llvm.masked.load.v1i32.p0v1i32 (<1 x i32>* %vp, i32 4, <1 x i1> %vc, <1 x i32> undef)
%ret = bitcast <1 x i32> %L to i32
ret i32 %ret
}
declare <1 x i32> @llvm.masked.load.v1i32.p0v1i32 (<1 x i32>*, i32, <1 x i1>, <1 x i32>)
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
declare <4 x i32> @llvm.masked.load.v4i32.p0v4i32(<4 x i32>*, i32, <4 x i1>, <4 x i32>)
declare <2 x i32> @llvm.masked.load.v2i32.p0v2i32(<2 x i32>*, i32, <2 x i1>, <2 x i32>)
declare <4 x i64> @llvm.masked.load.v4i64.p0v4i64(<4 x i64>*, i32, <4 x i1>, <4 x i64>)
[X86][AVX] Add v2i64/v2f64 load tests Ensure these use insertions, not masked load ops llvm-svn: 322076
2018-01-09 21:35:18 +08:00
declare <2 x i64> @llvm.masked.load.v2i64.p0v2i64(<2 x i64>*, i32, <2 x i1>, <2 x i64>)
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
declare <8 x float> @llvm.masked.load.v8f32.p0v8f32(<8 x float>*, i32, <8 x i1>, <8 x float>)
declare <8 x i32> @llvm.masked.load.v8i32.p0v8i32(<8 x i32>*, i32, <8 x i1>, <8 x i32>)
declare <4 x float> @llvm.masked.load.v4f32.p0v4f32(<4 x float>*, i32, <4 x i1>, <4 x float>)
declare <2 x float> @llvm.masked.load.v2f32.p0v2f32(<2 x float>*, i32, <2 x i1>, <2 x float>)
declare <8 x double> @llvm.masked.load.v8f64.p0v8f64(<8 x double>*, i32, <8 x i1>, <8 x double>)
declare <4 x double> @llvm.masked.load.v4f64.p0v4f64(<4 x double>*, i32, <4 x i1>, <4 x double>)
declare <2 x double> @llvm.masked.load.v2f64.p0v2f64(<2 x double>*, i32, <2 x i1>, <2 x double>)
[X86][SSE] Add SSE2/SSE42 masked load/store tests Now that the load/store tests are split the impact of running the tests on multiple (illegal) targets is a lot less impactful llvm-svn: 346896
2018-11-15 05:31:50 +08:00
declare <1 x double> @llvm.masked.load.v1f64.p0v1f64(<1 x double>*, i32, <1 x i1>, <1 x double>)