maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/Target/X86/X86InstrAVX512.td

7702 lines
382 KiB
TableGen
Raw Normal View History

Add a license header to the AVX512 file. llvm-svn: 229941
2015-02-20 08:36:53 +08:00
//===-- X86InstrAVX512.td - AVX512 Instruction Set ---------*- tablegen -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
// This file describes the X86 AVX512 instruction set, defining the
// instructions, and properties of the instructions which are needed for code
// generation, machine code emission, and analysis.
//
//===----------------------------------------------------------------------===//
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// Group template arguments that can be derived from the vector type (EltNum x
// EltVT). These are things like the register class for the writemask, etc.
// The idea is to pass one of these as the template argument rather than the
// individual arguments.
AVX-512: Added some comments to ERI scalar intrinsics. No functional change. llvm-svn: 223761
2014-12-09 15:06:32 +08:00
// The template is also used for scalar types, in this case numelts is 1.
[AVX512] Minor fix in lowering pattern for broadcast intrustions. No functional change. llvm-svn: 224122
2014-12-12 22:21:30 +08:00
class X86VectorVTInfo<int numelts, ValueType eltvt, RegisterClass rc,
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
string suffix = ""> {
RegisterClass RC = rc;
[AVX512] Minor fix in lowering pattern for broadcast intrustions. No functional change. llvm-svn: 224122
2014-12-12 22:21:30 +08:00
ValueType EltVT = eltvt;
[AVX512] Two new attributes in X86VectorVTInfo for subvector insert The new attributes are NumElts and the CD8TupleForm. This prepares the code to enable x8 and x2 inserts. NFC, no change in X86.td.expanded except for the new attributes. llvm-svn: 219871
2014-10-16 07:42:09 +08:00
int NumElts = numelts;
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// Corresponding mask register class.
RegisterClass KRC = !cast<RegisterClass>("VK" # NumElts);
// Corresponding write-mask register class.
RegisterClass KRCWM = !cast<RegisterClass>("VK" # NumElts # "WM");
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
// The mask VT.
ValueType KVT = !cast<ValueType>(!if (!eq (NumElts, 1), "i1",
"v" # NumElts # "i1"));
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// The GPR register class that can hold the write mask. Use GR8 for fewer
// than 8 elements. Use shift-right and equal to work around the lack of
// !lt in tablegen.
RegisterClass MRC =
!cast<RegisterClass>("GR" #
!if (!eq (!srl(NumElts, 3), 0), 8, NumElts));
// Suffix used in the instruction mnemonic.
string Suffix = suffix;
AVX-512: Added some comments to ERI scalar intrinsics. No functional change. llvm-svn: 223761
2014-12-09 15:06:32 +08:00
// VTName is a string name for vector VT. For vector types it will be
// v # NumElts # EltVT, so for vector of 8 elements of i32 it will be v8i32
// It is a little bit complex for scalar types, where NumElts = 1.
// In this case we build v4f32 or v2f64
string VTName = "v" # !if (!eq (NumElts, 1),
!if (!eq (EltVT.Size, 32), 4,
!if (!eq (EltVT.Size, 64), 2, NumElts)), NumElts) # EltVT;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// The vector VT.
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
ValueType VT = !cast<ValueType>(VTName);
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
string EltTypeName = !cast<string>(EltVT);
// Size of the element type in bits, e.g. 32 for v16i32.
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
string EltSizeName = !subst("i", "", !subst("f", "", EltTypeName));
int EltSize = EltVT.Size;
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// "i" for integer types and "f" for floating-point types
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
string TypeVariantName = !subst(EltSizeName, "", EltTypeName);
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// Size of RC in bits, e.g. 512 for VR512.
int Size = VT.Size;
// The corresponding memory operand, e.g. i512mem for VR512.
X86MemOperand MemOp = !cast<X86MemOperand>(TypeVariantName # Size # "mem");
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
X86MemOperand ScalarMemOp = !cast<X86MemOperand>(EltVT # "mem");
// Load patterns
// Note: For 128/256-bit integer VT we choose loadv2i64/loadv4i64
// due to load promotion during legalization
PatFrag LdFrag = !cast<PatFrag>("load" #
!if (!eq (TypeVariantName, "i"),
!if (!eq (Size, 128), "v2i64",
!if (!eq (Size, 256), "v4i64",
VTName)), VTName));
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
PatFrag AlignedLdFrag = !cast<PatFrag>("alignedload" #
!if (!eq (TypeVariantName, "i"),
!if (!eq (Size, 128), "v2i64",
!if (!eq (Size, 256), "v4i64",
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
!if (!eq (Size, 512),
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
!if (!eq (EltSize, 64), "v8i64", "v16i32"),
VTName))), VTName));
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
PatFrag ScalarLdFrag = !cast<PatFrag>("load" # EltVT);
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// The corresponding float type, e.g. v16f32 for v16i32
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
// Note: For EltSize < 32, FloatVT is illegal and TableGen
// fails to compile, so we choose FloatVT = VT
ValueType FloatVT = !cast<ValueType>(
!if (!eq (!srl(EltSize,5),0),
VTName,
!if (!eq(TypeVariantName, "i"),
"v" # NumElts # "f" # EltSize,
VTName)));
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
[X86][AVX512]fix dag & add intrinsics for fixupimm cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics Differential Revision: http://reviews.llvm.org/D16313 llvm-svn: 258124
2016-01-19 22:21:39 +08:00
ValueType IntVT = !cast<ValueType>(
!if (!eq (!srl(EltSize,5),0),
VTName,
!if (!eq(TypeVariantName, "f"),
"v" # NumElts # "i" # EltSize,
VTName)));
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// The string to specify embedded broadcast in assembly.
string BroadcastStr = "{1to" # NumElts # "}";
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
[AVX512] Two new attributes in X86VectorVTInfo for subvector insert The new attributes are NumElts and the CD8TupleForm. This prepares the code to enable x8 and x2 inserts. NFC, no change in X86.td.expanded except for the new attributes. llvm-svn: 219871
2014-10-16 07:42:09 +08:00
// 8-bit compressed displacement tuple/subvector format. This is only
// defined for NumElts <= 8.
CD8VForm CD8TupleForm = !if (!eq (!srl(NumElts, 4), 0),
!cast<CD8VForm>("CD8VT" # NumElts), ?);
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
SubRegIndex SubRegIdx = !if (!eq (Size, 128), sub_xmm,
!if (!eq (Size, 256), sub_ymm, ?));
Domain ExeDomain = !if (!eq (EltTypeName, "f32"), SSEPackedSingle,
!if (!eq (EltTypeName, "f64"), SSEPackedDouble,
SSEPackedInt));
[AVX512] Move DAG for all-zero node to X86VectorVTInfo No functional change. No change in X86.td.expanded except for the appearance of the new attributes. The new attributes will be used in the subsequent patch. llvm-svn: 219360
2014-10-09 07:25:31 +08:00
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
RegisterClass FRC = !if (!eq (EltTypeName, "f32"), FR32X, FR64X);
[AVX512] Move DAG for all-zero node to X86VectorVTInfo No functional change. No change in X86.td.expanded except for the appearance of the new attributes. The new attributes will be used in the subsequent patch. llvm-svn: 219360
2014-10-09 07:25:31 +08:00
// A vector type of the same width with element type i32. This is used to
// create the canonical constant zero node ImmAllZerosV.
ValueType i32VT = !cast<ValueType>("v" # !srl(Size, 5) # "i32");
dag ImmAllZerosV = (VT (bitconvert (i32VT immAllZerosV)));
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
string ZSuffix = !if (!eq (Size, 128), "Z128",
!if (!eq (Size, 256), "Z256", "Z"));
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
}
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
def v64i8_info : X86VectorVTInfo<64, i8, VR512, "b">;
def v32i16_info : X86VectorVTInfo<32, i16, VR512, "w">;
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
def v16i32_info : X86VectorVTInfo<16, i32, VR512, "d">;
def v8i64_info : X86VectorVTInfo<8, i64, VR512, "q">;
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
def v16f32_info : X86VectorVTInfo<16, f32, VR512, "ps">;
def v8f64_info : X86VectorVTInfo<8, f64, VR512, "pd">;
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
// "x" in v32i8x_info means RC = VR256X
def v32i8x_info : X86VectorVTInfo<32, i8, VR256X, "b">;
def v16i16x_info : X86VectorVTInfo<16, i16, VR256X, "w">;
def v8i32x_info : X86VectorVTInfo<8, i32, VR256X, "d">;
def v4i64x_info : X86VectorVTInfo<4, i64, VR256X, "q">;
[AVX-512] Expanded rsqrt/rcp instructions to VL subset. Refactored multiclass through AVX512_maskable llvm-svn: 220783
2014-10-29 00:37:13 +08:00
def v8f32x_info : X86VectorVTInfo<8, f32, VR256X, "ps">;
def v4f64x_info : X86VectorVTInfo<4, f64, VR256X, "pd">;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
def v16i8x_info : X86VectorVTInfo<16, i8, VR128X, "b">;
def v8i16x_info : X86VectorVTInfo<8, i16, VR128X, "w">;
def v4i32x_info : X86VectorVTInfo<4, i32, VR128X, "d">;
def v2i64x_info : X86VectorVTInfo<2, i64, VR128X, "q">;
[AVX-512] Expanded rsqrt/rcp instructions to VL subset. Refactored multiclass through AVX512_maskable llvm-svn: 220783
2014-10-29 00:37:13 +08:00
def v4f32x_info : X86VectorVTInfo<4, f32, VR128X, "ps">;
def v2f64x_info : X86VectorVTInfo<2, f64, VR128X, "pd">;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
AVX-512: Added some comments to ERI scalar intrinsics. No functional change. llvm-svn: 223761
2014-12-09 15:06:32 +08:00
// We map scalar types to the smallest (128-bit) vector type
// with the appropriate element type. This allows to use the same masking logic.
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
def i32x_info : X86VectorVTInfo<1, i32, GR32, "si">;
def i64x_info : X86VectorVTInfo<1, i64, GR64, "sq">;
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
def f32x_info : X86VectorVTInfo<1, f32, VR128X, "ss">;
def f64x_info : X86VectorVTInfo<1, f64, VR128X, "sd">;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
class AVX512VLVectorVTInfo<X86VectorVTInfo i512, X86VectorVTInfo i256,
X86VectorVTInfo i128> {
X86VectorVTInfo info512 = i512;
X86VectorVTInfo info256 = i256;
X86VectorVTInfo info128 = i128;
}
def avx512vl_i8_info : AVX512VLVectorVTInfo<v64i8_info, v32i8x_info,
v16i8x_info>;
def avx512vl_i16_info : AVX512VLVectorVTInfo<v32i16_info, v16i16x_info,
v8i16x_info>;
def avx512vl_i32_info : AVX512VLVectorVTInfo<v16i32_info, v8i32x_info,
v4i32x_info>;
def avx512vl_i64_info : AVX512VLVectorVTInfo<v8i64_info, v4i64x_info,
v2i64x_info>;
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
def avx512vl_f32_info : AVX512VLVectorVTInfo<v16f32_info, v8f32x_info,
v4f32x_info>;
def avx512vl_f64_info : AVX512VLVectorVTInfo<v8f64_info, v4f64x_info,
v2f64x_info>;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
// This multiclass generates the masking variants from the non-masking
// variant. It only provides the assembly pieces for the masking variants.
// It assumes custom ISel patterns for masking which can be provided as
// template arguments.
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
multiclass AVX512_maskable_custom<bits<8> O, Format F,
dag Outs,
dag Ins, dag MaskingIns, dag ZeroMaskingIns,
string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
list<dag> Pattern,
list<dag> MaskingPattern,
list<dag> ZeroMaskingPattern,
string MaskingConstraint = "",
InstrItinClass itin = NoItinerary,
bit IsCommutable = 0> {
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
let isCommutable = IsCommutable in
def NAME: AVX512<O, F, Outs, Ins,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
OpcodeStr#"\t{"#AttSrcAsm#", $dst|"#
[AVX-512] Remove another extra space from the Intel syntax asm strings. llvm-svn: 257304
2016-01-11 09:03:40 +08:00
"$dst, "#IntelSrcAsm#"}",
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
Pattern, itin>;
[AVX512] Generate masking instruction variants with tablegen After adding the masking variants to several instructions, I have decided to experiment with generating these from the non-masking/unconditional variant. This will hopefully reduce the amount repetition that we currently have in order to define an instruction with all its variants (for a reg/mem instruction this would be 6 instruction defs and 2 Pat<> for the intrinsic). The patch is the first cut that is currently only applied to valignd/q to make the patch small. A few notes on the approach: * In order to stitch together the dag for both the conditional and the unconditional patterns I pass the RHS of the set rather than the full pattern (set dest, RHS). * Rather than subclassing each instruction base class (e.g. AVX512AIi8), with a masking variant which wouldn't scale, I derived the masking instructions from a new base class AVX512 (this is just I<> with Requires<HasAVX512>). The instructions derive from this now, plus a new set of classes that add the format bits and everything else that instruction base class provided (i.e. AVX512AIi8 vs. AVX512AIi8Base). I hope we can go incrementally from here. I expect that: * We will need different variants of the masking class. One example is instructions requiring three vector sources. In this case we tie one of the source operands to dest rather than a new implicit source operand ($src0) * Add the zero-masking variant * Add more AVX512*Base classes as new uses are added I've looked at X86.td.expanded before and after to make sure that nothing got lost for valignd/q. llvm-svn: 215125
2014-08-08 01:53:55 +08:00
[AVX512] Add codegen test for the masking variant of valign The AddedComplexity is needed just like in avx512_perm_3src. There may be a bug in the complexity computation... llvm-svn: 215168
2014-08-08 07:18:18 +08:00
// Prefer over VMOV*rrk Pat<>
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
let AddedComplexity = 20 in
def NAME#k: AVX512<O, F, Outs, MaskingIns,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
OpcodeStr#"\t{"#AttSrcAsm#", $dst {${mask}}|"#
"$dst {${mask}}, "#IntelSrcAsm#"}",
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
MaskingPattern, itin>,
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
EVEX_K {
// In case of the 3src subclass this is overridden with a let.
string Constraints = MaskingConstraint;
}
[AVX512] Add zero-masking variant to AVX512_masking multiclass This completes one item from the todo-list of r215125 "Generate masking instruction variants with tablegen". The AddedComplexity is needed just like for the k variant. Added a codegen test based on valignq. llvm-svn: 215173
2014-08-08 07:53:38 +08:00
let AddedComplexity = 30 in // Prefer over VMOV*rrkz Pat<>
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
def NAME#kz: AVX512<O, F, Outs, ZeroMaskingIns,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
OpcodeStr#"\t{"#AttSrcAsm#", $dst {${mask}} {z}|"#
"$dst {${mask}} {z}, "#IntelSrcAsm#"}",
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
ZeroMaskingPattern,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
itin>,
[AVX512] Add zero-masking variant to AVX512_masking multiclass This completes one item from the todo-list of r215125 "Generate masking instruction variants with tablegen". The AddedComplexity is needed just like for the k variant. Added a codegen test based on valignq. llvm-svn: 215173
2014-08-08 07:53:38 +08:00
EVEX_KZ;
[AVX512] Generate masking instruction variants with tablegen After adding the masking variants to several instructions, I have decided to experiment with generating these from the non-masking/unconditional variant. This will hopefully reduce the amount repetition that we currently have in order to define an instruction with all its variants (for a reg/mem instruction this would be 6 instruction defs and 2 Pat<> for the intrinsic). The patch is the first cut that is currently only applied to valignd/q to make the patch small. A few notes on the approach: * In order to stitch together the dag for both the conditional and the unconditional patterns I pass the RHS of the set rather than the full pattern (set dest, RHS). * Rather than subclassing each instruction base class (e.g. AVX512AIi8), with a masking variant which wouldn't scale, I derived the masking instructions from a new base class AVX512 (this is just I<> with Requires<HasAVX512>). The instructions derive from this now, plus a new set of classes that add the format bits and everything else that instruction base class provided (i.e. AVX512AIi8 vs. AVX512AIi8Base). I hope we can go incrementally from here. I expect that: * We will need different variants of the masking class. One example is instructions requiring three vector sources. In this case we tie one of the source operands to dest rather than a new implicit source operand ($src0) * Add the zero-masking variant * Add more AVX512*Base classes as new uses are added I've looked at X86.td.expanded before and after to make sure that nothing got lost for valignd/q. llvm-svn: 215125
2014-08-08 01:53:55 +08:00
}
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
// Common base class of AVX512_maskable and AVX512_maskable_3src.
multiclass AVX512_maskable_common<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs,
dag Ins, dag MaskingIns, dag ZeroMaskingIns,
string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
dag RHS, dag MaskingRHS,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
SDNode Select = vselect,
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
string MaskingConstraint = "",
InstrItinClass itin = NoItinerary,
bit IsCommutable = 0> :
AVX512_maskable_custom<O, F, Outs, Ins, MaskingIns, ZeroMaskingIns, OpcodeStr,
AttSrcAsm, IntelSrcAsm,
[(set _.RC:$dst, RHS)],
[(set _.RC:$dst, MaskingRHS)],
[(set _.RC:$dst,
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
(Select _.KRCWM:$mask, RHS, _.ImmAllZerosV))],
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
MaskingConstraint, NoItinerary, IsCommutable>;
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
// This multiclass generates the unconditional/non-masking, the masking and
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
// the zero-masking variant of the vector instruction. In the masking case, the
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
// perserved vector elements come from a new dummy input operand tied to $dst.
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
multiclass AVX512_maskable<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag Ins, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
dag RHS,
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
InstrItinClass itin = NoItinerary,
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
bit IsCommutable = 0> :
AVX512_maskable_common<O, F, _, Outs, Ins,
!con((ins _.RC:$src0, _.KRCWM:$mask), Ins),
!con((ins _.KRCWM:$mask), Ins),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
(vselect _.KRCWM:$mask, RHS, _.RC:$src0), vselect,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
"$src0 = $dst", itin, IsCommutable>;
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
// This multiclass generates the unconditional/non-masking, the masking and
// the zero-masking variant of the scalar instruction.
multiclass AVX512_maskable_scalar<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag Ins, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
dag RHS,
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
InstrItinClass itin = NoItinerary,
bit IsCommutable = 0> :
AVX512_maskable_common<O, F, _, Outs, Ins,
!con((ins _.RC:$src0, _.KRCWM:$mask), Ins),
!con((ins _.KRCWM:$mask), Ins),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
[AVX512] Strengthen type checks on the X86ISD::SELECT node. Saves over 800 bytes in the DAG isel table by removing type checks for the condition operand which is always a vector or scalar of i1 matching the the number of elements in the other operands. llvm-svn: 269885
2016-05-18 14:55:59 +08:00
(X86selects _.KRCWM:$mask, RHS, _.RC:$src0),
X86selects, "$src0 = $dst", itin, IsCommutable>;
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
// Similar to AVX512_maskable but in this case one of the source operands
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
// ($src1) is already tied to $dst so we just use that for the preserved
// vector elements. NOTE that the NonTiedIns (the ins dag) should exclude
// $src1.
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
multiclass AVX512_maskable_3src<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag NonTiedIns, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
dag RHS> :
AVX512_maskable_common<O, F, _, Outs,
!con((ins _.RC:$src1), NonTiedIns),
!con((ins _.RC:$src1, _.KRCWM:$mask), NonTiedIns),
!con((ins _.RC:$src1, _.KRCWM:$mask), NonTiedIns),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
(vselect _.KRCWM:$mask, RHS, _.RC:$src1)>;
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
// Similar to AVX512_maskable_3rc but in this case the input VT for the tied
// operand differs from the output VT. This requires a bitconvert on
// the preserved vector going into the vselect.
multiclass AVX512_maskable_3src_cast<bits<8> O, Format F, X86VectorVTInfo OutVT,
X86VectorVTInfo InVT,
dag Outs, dag NonTiedIns, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
dag RHS> :
AVX512_maskable_common<O, F, OutVT, Outs,
!con((ins InVT.RC:$src1), NonTiedIns),
!con((ins InVT.RC:$src1, InVT.KRCWM:$mask), NonTiedIns),
!con((ins InVT.RC:$src1, InVT.KRCWM:$mask), NonTiedIns),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
(vselect InVT.KRCWM:$mask, RHS,
(bitconvert InVT.RC:$src1))>;
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
multiclass AVX512_maskable_3src_scalar<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag NonTiedIns, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
dag RHS> :
AVX512_maskable_common<O, F, _, Outs,
!con((ins _.RC:$src1), NonTiedIns),
!con((ins _.RC:$src1, _.KRCWM:$mask), NonTiedIns),
!con((ins _.RC:$src1, _.KRCWM:$mask), NonTiedIns),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
[AVX512] Strengthen type checks on the X86ISD::SELECT node. Saves over 800 bytes in the DAG isel table by removing type checks for the condition operand which is always a vector or scalar of i1 matching the the number of elements in the other operands. llvm-svn: 269885
2016-05-18 14:55:59 +08:00
(X86selects _.KRCWM:$mask, RHS, _.RC:$src1),
X86selects>;
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
multiclass AVX512_maskable_in_asm<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag Ins,
string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
list<dag> Pattern> :
AVX512_maskable_custom<O, F, Outs, Ins,
!con((ins _.RC:$src0, _.KRCWM:$mask), Ins),
!con((ins _.KRCWM:$mask), Ins),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
OpcodeStr, AttSrcAsm, IntelSrcAsm, Pattern, [], [],
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
"$src0 = $dst">;
[AVX512] Add asm-only support for vextract*x4 masking variants These derive from the new asm-only masking definitions. Unfortunately I wasn't able to find a ISel pattern that we could legally generate for the masking variants. The problem is that since the destination is v4* we would need VK4 register classes and v4i1 value types to express the masking. These are however not legal types/classes in AVX512f but only in VL, so things get complicated pretty quickly. We can revisit this question later if we have a more pressing need to express something like this. So the ISel patterns are empty for the masking instructions and the next patch will add Pat<>s instead to match the intrinsics calls with instructions. llvm-svn: 219361
2014-10-09 07:25:33 +08:00
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
// Instruction with mask that puts result in mask register,
// like "compare" and "vptest"
multiclass AVX512_maskable_custom_cmp<bits<8> O, Format F,
dag Outs,
dag Ins, dag MaskingIns,
string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
list<dag> Pattern,
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used. llvm-svn: 257300
2016-01-11 08:44:56 +08:00
list<dag> MaskingPattern> {
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
def NAME: AVX512<O, F, Outs, Ins,
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used. llvm-svn: 257300
2016-01-11 08:44:56 +08:00
OpcodeStr#"\t{"#AttSrcAsm#", $dst|"#
"$dst, "#IntelSrcAsm#"}",
Pattern, NoItinerary>;
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
def NAME#k: AVX512<O, F, Outs, MaskingIns,
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used. llvm-svn: 257300
2016-01-11 08:44:56 +08:00
OpcodeStr#"\t{"#AttSrcAsm#", $dst {${mask}}|"#
"$dst {${mask}}, "#IntelSrcAsm#"}",
MaskingPattern, NoItinerary>, EVEX_K;
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
}
multiclass AVX512_maskable_common_cmp<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs,
dag Ins, dag MaskingIns,
string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used. llvm-svn: 257300
2016-01-11 08:44:56 +08:00
dag RHS, dag MaskingRHS> :
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
AVX512_maskable_custom_cmp<O, F, Outs, Ins, MaskingIns, OpcodeStr,
AttSrcAsm, IntelSrcAsm,
[(set _.KRC:$dst, RHS)],
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used. llvm-svn: 257300
2016-01-11 08:44:56 +08:00
[(set _.KRC:$dst, MaskingRHS)]>;
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
multiclass AVX512_maskable_cmp<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag Ins, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used. llvm-svn: 257300
2016-01-11 08:44:56 +08:00
dag RHS> :
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
AVX512_maskable_common_cmp<O, F, _, Outs, Ins,
!con((ins _.KRCWM:$mask), Ins),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used. llvm-svn: 257300
2016-01-11 08:44:56 +08:00
(and _.KRCWM:$mask, RHS)>;
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
multiclass AVX512_maskable_cmp_alt<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag Ins, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm> :
AVX512_maskable_custom_cmp<O, F, Outs,
Ins, !con((ins _.KRCWM:$mask),Ins), OpcodeStr,
[AVX-512] Remove unused Round and Itinerary from the maskable_cmp multiclasses. They weren't used and there were extra spaces in the asm string to prepare for the concatenations of the round string that wasn't ever used. llvm-svn: 257300
2016-01-11 08:44:56 +08:00
AttSrcAsm, IntelSrcAsm, [],[]>;
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Bitcasts between 512-bit vector types. Return the original type since
// no instruction is needed for the conversion
let Predicates = [HasAVX512] in {
def : Pat<(v8f64 (bitconvert (v8i64 VR512:$src))), (v8f64 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v8f64 (bitconvert (v16i32 VR512:$src))), (v8f64 VR512:$src)>;
def : Pat<(v8f64 (bitconvert (v32i16 VR512:$src))), (v8f64 VR512:$src)>;
def : Pat<(v8f64 (bitconvert (v64i8 VR512:$src))), (v8f64 VR512:$src)>;
def : Pat<(v8f64 (bitconvert (v16f32 VR512:$src))), (v8f64 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (bitconvert (v8i64 VR512:$src))), (v16f32 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v16f32 (bitconvert (v16i32 VR512:$src))), (v16f32 VR512:$src)>;
def : Pat<(v16f32 (bitconvert (v32i16 VR512:$src))), (v16f32 VR512:$src)>;
def : Pat<(v16f32 (bitconvert (v64i8 VR512:$src))), (v16f32 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (bitconvert (v8f64 VR512:$src))), (v16f32 VR512:$src)>;
def : Pat<(v8i64 (bitconvert (v16i32 VR512:$src))), (v8i64 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v8i64 (bitconvert (v32i16 VR512:$src))), (v8i64 VR512:$src)>;
def : Pat<(v8i64 (bitconvert (v64i8 VR512:$src))), (v8i64 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i64 (bitconvert (v8f64 VR512:$src))), (v8i64 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v8i64 (bitconvert (v16f32 VR512:$src))), (v8i64 VR512:$src)>;
def : Pat<(v16i32 (bitconvert (v8i64 VR512:$src))), (v16i32 VR512:$src)>;
AVX-512: added a missing bitcast from v16f32 to v16i32 llvm-svn: 215351
2014-08-11 17:59:08 +08:00
def : Pat<(v16i32 (bitconvert (v16f32 VR512:$src))), (v16i32 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v16i32 (bitconvert (v32i16 VR512:$src))), (v16i32 VR512:$src)>;
def : Pat<(v16i32 (bitconvert (v64i8 VR512:$src))), (v16i32 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16i32 (bitconvert (v8f64 VR512:$src))), (v16i32 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v32i16 (bitconvert (v8i64 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v16i32 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v64i8 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v8f64 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v16f32 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v16f32 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v8i64 VR512:$src))), (v64i8 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v16i32 VR512:$src))), (v64i8 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v32i16 VR512:$src))), (v64i8 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v8f64 VR512:$src))), (v64i8 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v16f32 VR512:$src))), (v64i8 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v2i64 (bitconvert (v4i32 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v2i64 (bitconvert (v8i16 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v2i64 (bitconvert (v16i8 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v2i64 (bitconvert (v2f64 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v2i64 (bitconvert (v4f32 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v2i64 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v8i16 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v16i8 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v2f64 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v4f32 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v2i64 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v4i32 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v16i8 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v2f64 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v4f32 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v2i64 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v4i32 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v8i16 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v2f64 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v4f32 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v2i64 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v4i32 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v8i16 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v16i8 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v2f64 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v2i64 VR128X:$src))), (v2f64 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v4i32 VR128X:$src))), (v2f64 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v8i16 VR128X:$src))), (v2f64 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v16i8 VR128X:$src))), (v2f64 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v4f32 VR128X:$src))), (v2f64 VR128X:$src)>;
// Bitcasts between 256-bit vector types. Return the original type since
// no instruction is needed for the conversion
def : Pat<(v4f64 (bitconvert (v8f32 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v4f64 (bitconvert (v8i32 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v4f64 (bitconvert (v4i64 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v4f64 (bitconvert (v16i16 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v4f64 (bitconvert (v32i8 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v8i32 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v4i64 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v4f64 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v32i8 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v16i16 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v8f32 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v8i32 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v4f64 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v32i8 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v16i16 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v4f64 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v4i64 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v8f32 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v8i32 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v16i16 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v32i8 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v16i16 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v8f32 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v4i64 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v4f64 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v8f32 VR256X:$src))), (v16i16 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v8i32 VR256X:$src))), (v16i16 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v4i64 VR256X:$src))), (v16i16 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v4f64 VR256X:$src))), (v16i16 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v32i8 VR256X:$src))), (v16i16 VR256X:$src)>;
}
[X86] Remove extra patterns that check for BUILD_VECTOR of all 0s. These are always canonicalized to v4i32/v8i32/v16i32 except for in SSE1 only when only v4f32 is supported. llvm-svn: 268880
2016-05-09 04:10:20 +08:00
// Alias instruction that maps zero vector to pxor / xorp* for AVX-512.
// This is expanded by ExpandPostRAPseudos to an xorps / vxorps, and then
// swizzled by ExecutionDepsFix to pxor.
// We set canFoldAsLoad because this can be converted to a constant-pool
// load of an all-zeros value if folding it would be beneficial.
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let isReMaterializable = 1, isAsCheapAsAMove = 1, canFoldAsLoad = 1,
isPseudo = 1, Predicates = [HasAVX512] in {
def AVX512_512_SET0 : I<0, Pseudo, (outs VR512:$dst), (ins), "",
[X86] Remove extra patterns that check for BUILD_VECTOR of all 0s. These are always canonicalized to v4i32/v8i32/v16i32 except for in SSE1 only when only v4f32 is supported. llvm-svn: 268880
2016-05-09 04:10:20 +08:00
[(set VR512:$dst, (v16i32 immAllZerosV))]>;
Remove duplicate pattern and add predicate checks on other patterns. llvm-svn: 200455
2014-01-30 14:03:19 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Add VLX 128/256-bit SET0 operations that encode to 128/256-bit EVEX encoded VPXORD so all 32 registers can be used. llvm-svn: 268884
2016-05-09 05:33:53 +08:00
let isReMaterializable = 1, isAsCheapAsAMove = 1, canFoldAsLoad = 1,
isPseudo = 1, Predicates = [HasVLX] in {
def AVX512_128_SET0 : I<0, Pseudo, (outs VR128X:$dst), (ins), "",
[(set VR128X:$dst, (v4i32 immAllZerosV))]>;
def AVX512_256_SET0 : I<0, Pseudo, (outs VR256X:$dst), (ins), "",
[(set VR256X:$dst, (v8i32 immAllZerosV))]>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - VECTOR INSERT
//
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
multiclass vinsert_for_size<int Opcode, X86VectorVTInfo From, X86VectorVTInfo To,
PatFrag vinsert_insert> {
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
let hasSideEffects = 0, ExeDomain = To.ExeDomain in {
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
defm rr : AVX512_maskable<Opcode, MRMSrcReg, To, (outs To.RC:$dst),
(ins To.RC:$src1, From.RC:$src2, i32u8imm:$src3),
"vinsert" # From.EltTypeName # "x" # From.NumElts,
"$src3, $src2, $src1", "$src1, $src2, $src3",
(vinsert_insert:$src3 (To.VT To.RC:$src1),
(From.VT From.RC:$src2),
(iPTR imm))>, AVX512AIi8Base, EVEX_4V;
let mayLoad = 1 in
defm rm : AVX512_maskable<Opcode, MRMSrcMem, To, (outs To.RC:$dst),
(ins To.RC:$src1, From.MemOp:$src2, i32u8imm:$src3),
"vinsert" # From.EltTypeName # "x" # From.NumElts,
"$src3, $src2, $src1", "$src1, $src2, $src3",
(vinsert_insert:$src3 (To.VT To.RC:$src1),
(From.VT (bitconvert (From.LdFrag addr:$src2))),
(iPTR imm))>, AVX512AIi8Base, EVEX_4V,
EVEX_CD8<From.EltSize, From.CD8TupleForm>;
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
}
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
}
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
multiclass vinsert_for_size_lowering<string InstrStr, X86VectorVTInfo From,
X86VectorVTInfo To, PatFrag vinsert_insert,
SDNodeXForm INSERT_get_vinsert_imm , list<Predicate> p> {
let Predicates = p in {
def : Pat<(vinsert_insert:$ins
(To.VT To.RC:$src1), (From.VT From.RC:$src2), (iPTR imm)),
(To.VT (!cast<Instruction>(InstrStr#"rr")
To.RC:$src1, From.RC:$src2,
(INSERT_get_vinsert_imm To.RC:$ins)))>;
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
def : Pat<(vinsert_insert:$ins
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
(To.VT To.RC:$src1),
(From.VT (bitconvert (From.LdFrag addr:$src2))),
(iPTR imm)),
(To.VT (!cast<Instruction>(InstrStr#"rm")
To.RC:$src1, addr:$src2,
(INSERT_get_vinsert_imm To.RC:$ins)))>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Rename arg from Opcode32/64 to Opcode128/256 in vinsert_for_size It's the W bit that selects between 32 or 64 elt type and not the opcode. The opcode selects between the width of the insert (128 or 256). llvm-svn: 219870
2014-10-16 07:42:04 +08:00
multiclass vinsert_for_type<ValueType EltVT32, int Opcode128,
ValueType EltVT64, int Opcode256> {
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
let Predicates = [HasVLX] in
defm NAME # "32x4Z256" : vinsert_for_size<Opcode128,
X86VectorVTInfo< 4, EltVT32, VR128X>,
X86VectorVTInfo< 8, EltVT32, VR256X>,
vinsert128_insert>, EVEX_V256;
defm NAME # "32x4Z" : vinsert_for_size<Opcode128,
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
X86VectorVTInfo< 4, EltVT32, VR128X>,
X86VectorVTInfo<16, EltVT32, VR512>,
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
vinsert128_insert>, EVEX_V512;
defm NAME # "64x4Z" : vinsert_for_size<Opcode256,
X86VectorVTInfo< 4, EltVT64, VR256X>,
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
X86VectorVTInfo< 8, EltVT64, VR512>,
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
vinsert256_insert>, VEX_W, EVEX_V512;
let Predicates = [HasVLX, HasDQI] in
defm NAME # "64x2Z256" : vinsert_for_size<Opcode128,
X86VectorVTInfo< 2, EltVT64, VR128X>,
X86VectorVTInfo< 4, EltVT64, VR256X>,
vinsert128_insert>, VEX_W, EVEX_V256;
let Predicates = [HasDQI] in {
defm NAME # "64x2Z" : vinsert_for_size<Opcode128,
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
X86VectorVTInfo< 2, EltVT64, VR128X>,
X86VectorVTInfo< 8, EltVT64, VR512>,
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
vinsert128_insert>, VEX_W, EVEX_V512;
defm NAME # "32x8Z" : vinsert_for_size<Opcode256,
X86VectorVTInfo< 8, EltVT32, VR256X>,
X86VectorVTInfo<16, EltVT32, VR512>,
vinsert256_insert>, EVEX_V512;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
defm VINSERTF : vinsert_for_type<f32, 0x18, f64, 0x1a>;
defm VINSERTI : vinsert_for_type<i32, 0x38, i64, 0x3a>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX512: Implement instructions encoding, lowering and intrinsics vinserti64x4, vinserti64x2, vinserti32x8, vinserti32x4, vinsertf64x4, vinsertf64x2, vinsertf32x8, vinsertf32x4 Added tests for encoding, lowering and intrinsics. Differential Revision: http://reviews.llvm.org/D11893 llvm-svn: 248111
2015-09-20 14:52:42 +08:00
// Codegen pattern with the alternative types,
// Only add this if 64x2 and its friends are not supported natively via AVX512DQ.
defm : vinsert_for_size_lowering<"VINSERTF32x4Z256", v2f64x_info, v4f64x_info,
vinsert128_insert, INSERT_get_vinsert128_imm, [HasVLX, NoDQI]>;
defm : vinsert_for_size_lowering<"VINSERTI32x4Z256", v2i64x_info, v4i64x_info,
vinsert128_insert, INSERT_get_vinsert128_imm, [HasVLX, NoDQI]>;
defm : vinsert_for_size_lowering<"VINSERTF32x4Z", v2f64x_info, v8f64_info,
vinsert128_insert, INSERT_get_vinsert128_imm, [HasAVX512, NoDQI]>;
defm : vinsert_for_size_lowering<"VINSERTI32x4Z", v2i64x_info, v8i64_info,
vinsert128_insert, INSERT_get_vinsert128_imm, [HasAVX512, NoDQI]>;
defm : vinsert_for_size_lowering<"VINSERTF64x4Z", v8f32x_info, v16f32_info,
vinsert256_insert, INSERT_get_vinsert256_imm, [HasAVX512, NoDQI]>;
defm : vinsert_for_size_lowering<"VINSERTI64x4Z", v8i32x_info, v16i32_info,
vinsert256_insert, INSERT_get_vinsert256_imm, [HasAVX512, NoDQI]>;
// Codegen pattern with the alternative types insert VEC128 into VEC256
defm : vinsert_for_size_lowering<"VINSERTI32x4Z256", v8i16x_info, v16i16x_info,
vinsert128_insert, INSERT_get_vinsert128_imm, [HasVLX]>;
defm : vinsert_for_size_lowering<"VINSERTI32x4Z256", v16i8x_info, v32i8x_info,
vinsert128_insert, INSERT_get_vinsert128_imm, [HasVLX]>;
// Codegen pattern with the alternative types insert VEC128 into VEC512
defm : vinsert_for_size_lowering<"VINSERTI32x4Z", v8i16x_info, v32i16_info,
vinsert128_insert, INSERT_get_vinsert128_imm, [HasAVX512]>;
defm : vinsert_for_size_lowering<"VINSERTI32x4Z", v16i8x_info, v64i8_info,
vinsert128_insert, INSERT_get_vinsert128_imm, [HasAVX512]>;
// Codegen pattern with the alternative types insert VEC256 into VEC512
defm : vinsert_for_size_lowering<"VINSERTI64x4Z", v16i16x_info, v32i16_info,
vinsert256_insert, INSERT_get_vinsert256_imm, [HasAVX512]>;
defm : vinsert_for_size_lowering<"VINSERTI64x4Z", v32i8x_info, v64i8_info,
vinsert256_insert, INSERT_get_vinsert256_imm, [HasAVX512]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// vinsertps - insert f32 to XMM
def VINSERTPSzrr : AVX512AIi8<0x21, MRMSrcReg, (outs VR128X:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins VR128X:$src1, VR128X:$src2, u8imm:$src3),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vinsertps\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}",
Rename X86insrtps to the proper instruction name. Summary: The INSERTPS pattern fragment was called insrtps (mising 'e'), which would make it harder to grep for the patterns related to this instruction. Renaming it to use the proper instruction name. Reviewers: nadav CC: llvm-commits Differential Revision: http://reviews.llvm.org/D3443 llvm-svn: 206779
2014-04-22 04:07:29 +08:00
[(set VR128X:$dst, (X86insertps VR128X:$src1, VR128X:$src2, imm:$src3))]>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_4V;
def VINSERTPSzrm: AVX512AIi8<0x21, MRMSrcMem, (outs VR128X:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins VR128X:$src1, f32mem:$src2, u8imm:$src3),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vinsertps\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}",
Rename X86insrtps to the proper instruction name. Summary: The INSERTPS pattern fragment was called insrtps (mising 'e'), which would make it harder to grep for the patterns related to this instruction. Renaming it to use the proper instruction name. Reviewers: nadav CC: llvm-commits Differential Revision: http://reviews.llvm.org/D3443 llvm-svn: 206779
2014-04-22 04:07:29 +08:00
[(set VR128X:$dst, (X86insertps VR128X:$src1,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v4f32 (scalar_to_vector (loadf32 addr:$src2))),
imm:$src3))]>, EVEX_4V, EVEX_CD8<32, CD8VT1>;
//===----------------------------------------------------------------------===//
// AVX-512 VECTOR EXTRACT
//---
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
multiclass vextract_for_size_first_position_lowering<X86VectorVTInfo From,
X86VectorVTInfo To> {
// A subvector extract from the first vector position is
// a subregister copy that needs no instruction.
def NAME # To.NumElts:
Pat<(To.VT (extract_subvector (From.VT From.RC:$src),(iPTR 0))),
(To.VT (EXTRACT_SUBREG (From.VT From.RC:$src), To.SubRegIdx))>;
}
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
multiclass vextract_for_size<int Opcode,
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
X86VectorVTInfo From, X86VectorVTInfo To,
PatFrag vextract_extract> :
vextract_for_size_first_position_lowering<From, To> {
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
let hasSideEffects = 0, ExeDomain = To.ExeDomain in {
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
// use AVX512_maskable_in_asm (AVX512_maskable can't be used due to
// vextract_extract), we interesting only in patterns without mask,
// intrinsics pattern match generated bellow.
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rr : AVX512_maskable_in_asm<Opcode, MRMDestReg, To, (outs To.RC:$dst),
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
(ins From.RC:$src1, i32u8imm:$idx),
"vextract" # To.EltTypeName # "x" # To.NumElts,
[AVX512] Add asm-only support for vextract*x4 masking variants These derive from the new asm-only masking definitions. Unfortunately I wasn't able to find a ISel pattern that we could legally generate for the masking variants. The problem is that since the destination is v4* we would need VK4 register classes and v4i1 value types to express the masking. These are however not legal types/classes in AVX512f but only in VL, so things get complicated pretty quickly. We can revisit this question later if we have a more pressing need to express something like this. So the ISel patterns are empty for the masking instructions and the next patch will add Pat<>s instead to match the intrinsics calls with instructions. llvm-svn: 219361
2014-10-09 07:25:33 +08:00
"$idx, $src1", "$src1, $idx",
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
[(set To.RC:$dst, (vextract_extract:$idx (From.VT From.RC:$src1),
[AVX512] Add asm-only support for vextract*x4 masking variants These derive from the new asm-only masking definitions. Unfortunately I wasn't able to find a ISel pattern that we could legally generate for the masking variants. The problem is that since the destination is v4* we would need VK4 register classes and v4i1 value types to express the masking. These are however not legal types/classes in AVX512f but only in VL, so things get complicated pretty quickly. We can revisit this question later if we have a more pressing need to express something like this. So the ISel patterns are empty for the masking instructions and the next patch will add Pat<>s instead to match the intrinsics calls with instructions. llvm-svn: 219361
2014-10-09 07:25:33 +08:00
(iPTR imm)))]>,
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
AVX512AIi8Base, EVEX;
let mayStore = 1 in {
def rm : AVX512AIi8<Opcode, MRMDestMem, (outs),
(ins To.MemOp:$dst, From.RC:$src1, i32u8imm:$src2),
"vextract" # To.EltTypeName # "x" # To.NumElts #
"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[]>, EVEX;
def rmk : AVX512AIi8<Opcode, MRMDestMem, (outs),
(ins To.MemOp:$dst, To.KRCWM:$mask,
From.RC:$src1, i32u8imm:$src2),
"vextract" # To.EltTypeName # "x" # To.NumElts #
"\t{$src2, $src1, $dst {${mask}}|"
"$dst {${mask}}, $src1, $src2}",
[]>, EVEX_K, EVEX;
}//mayStore = 1
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Intrinsics for vextract*x4 This adds the Pat<>'s for the intrinsics. These are necessary because we don't lower these intrinsics to SDNodes but match them directly. See the rational in the previous commit. llvm-svn: 219362
2014-10-09 07:25:37 +08:00
// Intrinsic call with masking.
def : Pat<(!cast<Intrinsic>("int_x86_avx512_mask_vextract" # To.EltTypeName #
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
"x" # To.NumElts # "_" # From.Size)
From.RC:$src1, (iPTR imm:$idx), To.RC:$src0, To.MRC:$mask),
(!cast<Instruction>(NAME # To.EltSize # "x" # To.NumElts #
From.ZSuffix # "rrk")
To.RC:$src0,
(COPY_TO_REGCLASS To.MRC:$mask, To.KRCWM),
From.RC:$src1, imm:$idx)>;
[AVX512] Intrinsics for vextract*x4 This adds the Pat<>'s for the intrinsics. These are necessary because we don't lower these intrinsics to SDNodes but match them directly. See the rational in the previous commit. llvm-svn: 219362
2014-10-09 07:25:37 +08:00
// Intrinsic call with zero-masking.
def : Pat<(!cast<Intrinsic>("int_x86_avx512_mask_vextract" # To.EltTypeName #
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
"x" # To.NumElts # "_" # From.Size)
From.RC:$src1, (iPTR imm:$idx), To.ImmAllZerosV, To.MRC:$mask),
(!cast<Instruction>(NAME # To.EltSize # "x" # To.NumElts #
From.ZSuffix # "rrkz")
(COPY_TO_REGCLASS To.MRC:$mask, To.KRCWM),
From.RC:$src1, imm:$idx)>;
[AVX512] Intrinsics for vextract*x4 This adds the Pat<>'s for the intrinsics. These are necessary because we don't lower these intrinsics to SDNodes but match them directly. See the rational in the previous commit. llvm-svn: 219362
2014-10-09 07:25:37 +08:00
// Intrinsic call without masking.
def : Pat<(!cast<Intrinsic>("int_x86_avx512_mask_vextract" # To.EltTypeName #
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
"x" # To.NumElts # "_" # From.Size)
From.RC:$src1, (iPTR imm:$idx), To.ImmAllZerosV, (i8 -1)),
(!cast<Instruction>(NAME # To.EltSize # "x" # To.NumElts #
From.ZSuffix # "rr")
From.RC:$src1, imm:$idx)>;
}
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
// Codegen pattern for the alternative types
multiclass vextract_for_size_lowering<string InstrStr, X86VectorVTInfo From,
X86VectorVTInfo To, PatFrag vextract_extract,
SDNodeXForm EXTRACT_get_vextract_imm, list<Predicate> p> :
vextract_for_size_first_position_lowering<From, To> {
let Predicates = p in
def : Pat<(vextract_extract:$ext (From.VT From.RC:$src1), (iPTR imm)),
(To.VT (!cast<Instruction>(InstrStr#"rr")
From.RC:$src1,
(EXTRACT_get_vextract_imm To.RC:$ext)))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
multiclass vextract_for_type<ValueType EltVT32, int Opcode128,
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
ValueType EltVT64, int Opcode256> {
defm NAME # "32x4Z" : vextract_for_size<Opcode128,
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
X86VectorVTInfo<16, EltVT32, VR512>,
X86VectorVTInfo< 4, EltVT32, VR128X>,
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
vextract128_extract>,
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
EVEX_V512, EVEX_CD8<32, CD8VT4>;
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
defm NAME # "64x4Z" : vextract_for_size<Opcode256,
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
X86VectorVTInfo< 8, EltVT64, VR512>,
X86VectorVTInfo< 4, EltVT64, VR256X>,
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
vextract256_extract>,
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
VEX_W, EVEX_V512, EVEX_CD8<64, CD8VT4>;
let Predicates = [HasVLX] in
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
defm NAME # "32x4Z256" : vextract_for_size<Opcode128,
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
X86VectorVTInfo< 8, EltVT32, VR256X>,
X86VectorVTInfo< 4, EltVT32, VR128X>,
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
vextract128_extract>,
AVX512: Implemented encoding and intrinsics for vextracti64x4 ,vextracti64x2, vextracti32x8, vextracti32x4, vextractf64x4, vextractf64x2, vextractf32x8, vextractf32x4 Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11802 llvm-svn: 247276
2015-09-10 20:54:54 +08:00
EVEX_V256, EVEX_CD8<32, CD8VT4>;
let Predicates = [HasVLX, HasDQI] in
defm NAME # "64x2Z256" : vextract_for_size<Opcode128,
X86VectorVTInfo< 4, EltVT64, VR256X>,
X86VectorVTInfo< 2, EltVT64, VR128X>,
vextract128_extract>,
VEX_W, EVEX_V256, EVEX_CD8<64, CD8VT2>;
let Predicates = [HasDQI] in {
defm NAME # "64x2Z" : vextract_for_size<Opcode128,
X86VectorVTInfo< 8, EltVT64, VR512>,
X86VectorVTInfo< 2, EltVT64, VR128X>,
vextract128_extract>,
VEX_W, EVEX_V512, EVEX_CD8<64, CD8VT2>;
defm NAME # "32x8Z" : vextract_for_size<Opcode256,
X86VectorVTInfo<16, EltVT32, VR512>,
X86VectorVTInfo< 8, EltVT32, VR256X>,
vextract256_extract>,
EVEX_V512, EVEX_CD8<32, CD8VT8>;
}
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
defm VEXTRACTF : vextract_for_type<f32, 0x19, f64, 0x1b>;
defm VEXTRACTI : vextract_for_type<i32, 0x39, i64, 0x3b>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
// extract_subvector codegen patterns with the alternative types.
// Only add this if 64x2 and its friends are not supported natively via AVX512DQ.
defm : vextract_for_size_lowering<"VEXTRACTF32x4Z", v8f64_info, v2f64x_info,
vextract128_extract, EXTRACT_get_vextract128_imm, [HasAVX512, NoDQI]>;
defm : vextract_for_size_lowering<"VEXTRACTI32x4Z", v8i64_info, v2i64x_info,
vextract128_extract, EXTRACT_get_vextract128_imm, [HasAVX512, NoDQI]>;
defm : vextract_for_size_lowering<"VEXTRACTF64x4Z", v16f32_info, v8f32x_info,
AVX-512: Use correct extract vector length. Bug https://llvm.org/bugs/show_bug.cgi?id=25318 Differential Revision: http://reviews.llvm.org/D14062 llvm-svn: 251285
2015-10-26 20:26:34 +08:00
vextract256_extract, EXTRACT_get_vextract256_imm, [HasAVX512, NoDQI]>;
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
defm : vextract_for_size_lowering<"VEXTRACTI64x4Z", v16i32_info, v8i32x_info,
vextract256_extract, EXTRACT_get_vextract256_imm, [HasAVX512, NoDQI]>;
defm : vextract_for_size_lowering<"VEXTRACTF32x4Z256", v4f64x_info, v2f64x_info,
vextract128_extract, EXTRACT_get_vextract128_imm, [HasVLX, NoDQI]>;
defm : vextract_for_size_lowering<"VEXTRACTI32x4Z256", v4i64x_info, v2i64x_info,
vextract128_extract, EXTRACT_get_vextract128_imm, [HasVLX, NoDQI]>;
// Codegen pattern with the alternative types extract VEC128 from VEC512
defm : vextract_for_size_lowering<"VEXTRACTI32x4Z", v32i16_info, v8i16x_info,
vextract128_extract, EXTRACT_get_vextract128_imm, [HasAVX512]>;
defm : vextract_for_size_lowering<"VEXTRACTI32x4Z", v64i8_info, v16i8x_info,
vextract128_extract, EXTRACT_get_vextract128_imm, [HasAVX512]>;
// Codegen pattern with the alternative types extract VEC256 from VEC512
defm : vextract_for_size_lowering<"VEXTRACTI64x4Z", v32i16_info, v16i16x_info,
vextract256_extract, EXTRACT_get_vextract256_imm, [HasAVX512]>;
defm : vextract_for_size_lowering<"VEXTRACTI64x4Z", v64i8_info, v32i8x_info,
vextract256_extract, EXTRACT_get_vextract256_imm, [HasAVX512]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// A 128-bit subvector insert to the first 512-bit vector position
// is a subregister copy that needs no instruction.
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
def : Pat<(v8i64 (insert_subvector undef, (v2i64 VR128X:$src), (iPTR 0))),
(INSERT_SUBREG (v8i64 (IMPLICIT_DEF)), VR128X:$src, sub_xmm)>;
def : Pat<(v8f64 (insert_subvector undef, (v2f64 VR128X:$src), (iPTR 0))),
(INSERT_SUBREG (v8f64 (IMPLICIT_DEF)), VR128X:$src, sub_xmm)>;
def : Pat<(v16i32 (insert_subvector undef, (v4i32 VR128X:$src), (iPTR 0))),
(INSERT_SUBREG (v16i32 (IMPLICIT_DEF)), VR128X:$src, sub_xmm)>;
def : Pat<(v16f32 (insert_subvector undef, (v4f32 VR128X:$src), (iPTR 0))),
(INSERT_SUBREG (v16f32 (IMPLICIT_DEF)), VR128X:$src, sub_xmm)>;
def : Pat<(v32i16 (insert_subvector undef, (v8i16 VR128X:$src), (iPTR 0))),
(INSERT_SUBREG (v32i16 (IMPLICIT_DEF)), VR128X:$src, sub_xmm)>;
def : Pat<(v64i8 (insert_subvector undef, (v16i8 VR128X:$src), (iPTR 0))),
(INSERT_SUBREG (v64i8 (IMPLICIT_DEF)), VR128X:$src, sub_xmm)>;
def : Pat<(v8i64 (insert_subvector undef, (v4i64 VR256X:$src), (iPTR 0))),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(INSERT_SUBREG (v8i64 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
def : Pat<(v8f64 (insert_subvector undef, (v4f64 VR256X:$src), (iPTR 0))),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(INSERT_SUBREG (v8f64 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
def : Pat<(v16i32 (insert_subvector undef, (v8i32 VR256X:$src), (iPTR 0))),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(INSERT_SUBREG (v16i32 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
def : Pat<(v16f32 (insert_subvector undef, (v8f32 VR256X:$src), (iPTR 0))),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(INSERT_SUBREG (v16f32 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
def : Pat<(v32i16 (insert_subvector undef, (v16i16 VR256X:$src), (iPTR 0))),
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
(INSERT_SUBREG (v32i16 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
def : Pat<(v64i8 (insert_subvector undef, (v32i8 VR256X:$src), (iPTR 0))),
AVX512: Lowering i8/i16 vector CTLZ using the dword LZCNT vector instruction Differential Revision: http://reviews.llvm.org/D13632 llvm-svn: 250649
2015-10-18 17:56:39 +08:00
(INSERT_SUBREG (v64i8 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// vextractps - extract 32 bits from XMM
def VEXTRACTPSzrr : AVX512AIi8<0x17, MRMDestReg, (outs GR32:$dst),
[X86] Use u8imm in several places that used i32i8imm that don't require an i32 type. llvm-svn: 227031
2015-01-25 10:21:13 +08:00
(ins VR128X:$src1, u8imm:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vextractps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set GR32:$dst, (extractelt (bc_v4i32 (v4f32 VR128X:$src1)), imm:$src2))]>,
EVEX;
def VEXTRACTPSzmr : AVX512AIi8<0x17, MRMDestMem, (outs),
[X86] Use u8imm in several places that used i32i8imm that don't require an i32 type. llvm-svn: 227031
2015-01-25 10:21:13 +08:00
(ins f32mem:$dst, VR128X:$src1, u8imm:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vextractps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(store (extractelt (bc_v4i32 (v4f32 VR128X:$src1)), imm:$src2),
AVX-512: Optimized BUILD_VECTOR pattern; fixed encoding of VEXTRACTPS instruction. llvm-svn: 201134
2014-02-11 15:25:59 +08:00
addr:$dst)]>, EVEX, EVEX_CD8<32, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===---------------------------------------------------------------------===//
// AVX-512 BROADCAST
//---
Change AVX512 braodcastsd/ss patterns interaction with spilling . New implementation take a scalar register and generate a vector without COPY_TO_REGCLASS (turn it into a VR128 register ) .The issue is that during register allocation we may spill a scalar value using 128-bit loads and stores, wasting cache bandwidth. Differential Revision: http://reviews.llvm.org/D19579 llvm-svn: 268190
2016-05-01 16:40:00 +08:00
// broadcast with a scalar argument.
multiclass avx512_broadcast_scalar<bits<8> opc, string OpcodeStr,
X86VectorVTInfo DestInfo, X86VectorVTInfo SrcInfo> {
let isCodeGenOnly = 1 in {
def r_s : I< opc, MRMSrcReg, (outs DestInfo.RC:$dst),
(ins SrcInfo.FRC:$src), OpcodeStr#"\t{$src, $dst|$dst, $src}",
[(set DestInfo.RC:$dst, (DestInfo.VT (X86VBroadcast SrcInfo.FRC:$src)))]>,
Requires<[HasAVX512]>, T8PD, EVEX;
let Constraints = "$src0 = $dst" in
def rk_s : I< opc, MRMSrcReg, (outs DestInfo.RC:$dst),
(ins DestInfo.RC:$src0, DestInfo.KRCWM:$mask, SrcInfo.FRC:$src),
OpcodeStr#"\t{$src, $dst {${mask}} |$dst {${mask}}, $src}",
[(set DestInfo.RC:$dst,
(vselect DestInfo.KRCWM:$mask,
(DestInfo.VT (X86VBroadcast SrcInfo.FRC:$src)),
DestInfo.RC:$src0))]>,
Requires<[HasAVX512]>, T8PD, EVEX, EVEX_K;
def rkz_s : I< opc, MRMSrcReg, (outs DestInfo.RC:$dst),
(ins DestInfo.KRCWM:$mask, SrcInfo.FRC:$src),
OpcodeStr#"\t{$src, $dst {${mask}} {z}|$dst {${mask}} {z}, $src}",
[(set DestInfo.RC:$dst,
(vselect DestInfo.KRCWM:$mask,
(DestInfo.VT (X86VBroadcast SrcInfo.FRC:$src)),
DestInfo.ImmAllZerosV))]>,
Requires<[HasAVX512]>, T8PD, EVEX, EVEX_KZ;
} // let isCodeGenOnly = 1 in
}
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
AVX512: Implemented encoding and intrinsics for VPBROADCASTB/W/D/Q instructions. Differential Revision: http://reviews.llvm.org/D13884 llvm-svn: 250819
2015-10-20 19:56:42 +08:00
multiclass avx512_broadcast_rm<bits<8> opc, string OpcodeStr,
X86VectorVTInfo DestInfo, X86VectorVTInfo SrcInfo> {
defm r : AVX512_maskable<opc, MRMSrcReg, DestInfo, (outs DestInfo.RC:$dst),
(ins SrcInfo.RC:$src), OpcodeStr, "$src", "$src",
(DestInfo.VT (X86VBroadcast (SrcInfo.VT SrcInfo.RC:$src)))>,
T8PD, EVEX;
let mayLoad = 1 in
defm m : AVX512_maskable<opc, MRMSrcMem, DestInfo, (outs DestInfo.RC:$dst),
(ins SrcInfo.ScalarMemOp:$src), OpcodeStr, "$src", "$src",
(DestInfo.VT (X86VBroadcast
(SrcInfo.ScalarLdFrag addr:$src)))>,
T8PD, EVEX, EVEX_CD8<SrcInfo.EltSize, CD8VT1>;
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
}
AVX512: Implemented encoding and intrinsics for VPBROADCASTB/W/D/Q instructions. Differential Revision: http://reviews.llvm.org/D13884 llvm-svn: 250819
2015-10-20 19:56:42 +08:00
multiclass avx512_fp_broadcast_vl<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _> {
defm Z : avx512_broadcast_rm<opc, OpcodeStr, _.info512, _.info128>,
Change AVX512 braodcastsd/ss patterns interaction with spilling . New implementation take a scalar register and generate a vector without COPY_TO_REGCLASS (turn it into a VR128 register ) .The issue is that during register allocation we may spill a scalar value using 128-bit loads and stores, wasting cache bandwidth. Differential Revision: http://reviews.llvm.org/D19579 llvm-svn: 268190
2016-05-01 16:40:00 +08:00
avx512_broadcast_scalar<opc, OpcodeStr, _.info512, _.info128>,
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
EVEX_V512;
let Predicates = [HasVLX] in {
AVX512: Implemented encoding and intrinsics for VPBROADCASTB/W/D/Q instructions. Differential Revision: http://reviews.llvm.org/D13884 llvm-svn: 250819
2015-10-20 19:56:42 +08:00
defm Z256 : avx512_broadcast_rm<opc, OpcodeStr, _.info256, _.info128>,
Change AVX512 braodcastsd/ss patterns interaction with spilling . New implementation take a scalar register and generate a vector without COPY_TO_REGCLASS (turn it into a VR128 register ) .The issue is that during register allocation we may spill a scalar value using 128-bit loads and stores, wasting cache bandwidth. Differential Revision: http://reviews.llvm.org/D19579 llvm-svn: 268190
2016-05-01 16:40:00 +08:00
avx512_broadcast_scalar<opc, OpcodeStr, _.info256, _.info128>,
AVX512: Implemented encoding and intrinsics for VPBROADCASTB/W/D/Q instructions. Differential Revision: http://reviews.llvm.org/D13884 llvm-svn: 250819
2015-10-20 19:56:42 +08:00
EVEX_V256;
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let ExeDomain = SSEPackedSingle in {
AVX512: Implemented encoding and intrinsics for VPBROADCASTB/W/D/Q instructions. Differential Revision: http://reviews.llvm.org/D13884 llvm-svn: 250819
2015-10-20 19:56:42 +08:00
defm VBROADCASTSS : avx512_fp_broadcast_vl<0x18, "vbroadcastss",
avx512vl_f32_info>;
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
let Predicates = [HasVLX] in {
Change AVX512 braodcastsd/ss patterns interaction with spilling . New implementation take a scalar register and generate a vector without COPY_TO_REGCLASS (turn it into a VR128 register ) .The issue is that during register allocation we may spill a scalar value using 128-bit loads and stores, wasting cache bandwidth. Differential Revision: http://reviews.llvm.org/D19579 llvm-svn: 268190
2016-05-01 16:40:00 +08:00
defm VBROADCASTSSZ128 :
avx512_broadcast_rm<0x18, "vbroadcastss", v4f32x_info, v4f32x_info>,
avx512_broadcast_scalar<0x18, "vbroadcastss", v4f32x_info, v4f32x_info>,
EVEX_V128;
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
let ExeDomain = SSEPackedDouble in {
AVX512: Implemented encoding and intrinsics for VPBROADCASTB/W/D/Q instructions. Differential Revision: http://reviews.llvm.org/D13884 llvm-svn: 250819
2015-10-20 19:56:42 +08:00
defm VBROADCASTSD : avx512_fp_broadcast_vl<0x19, "vbroadcastsd",
avx512vl_f64_info>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[X86][AVX512] Add patterns that match the AVX512 floating point vbroadcast intrinsics. Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193421
2013-10-26 01:47:18 +08:00
def : Pat<(int_x86_avx512_vbroadcast_ss_512 addr:$src),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSSZm addr:$src)>;
[X86][AVX512] Add patterns that match the AVX512 floating point vbroadcast intrinsics. Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193421
2013-10-26 01:47:18 +08:00
def : Pat<(int_x86_avx512_vbroadcast_sd_512 addr:$src),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSDZm addr:$src)>;
[X86][AVX512] Add patterns that match the AVX512 floating point vbroadcast intrinsics. Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193421
2013-10-26 01:47:18 +08:00
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
multiclass avx512_int_broadcast_reg<bits<8> opc, X86VectorVTInfo _,
RegisterClass SrcRC> {
AVX512: VPBROADCASTB/W/D/Q from GPR intrinsics implementation. Differential Revision: http://reviews.llvm.org/D16813 llvm-svn: 260024
2016-02-07 16:30:50 +08:00
defm r : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins SrcRC:$src),
"vpbroadcast"##_.Suffix, "$src", "$src",
(_.VT (X86VBroadcast SrcRC:$src))>, T8PD, EVEX;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
multiclass avx512_int_broadcast_reg_vl<bits<8> opc, AVX512VLVectorVTInfo _,
RegisterClass SrcRC, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_int_broadcast_reg<opc, _.info512, SrcRC>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_int_broadcast_reg<opc, _.info256, SrcRC>, EVEX_V256;
defm Z128 : avx512_int_broadcast_reg<opc, _.info128, SrcRC>, EVEX_V128;
}
}
AVX512: VPBROADCASTB/W/D/Q from GPR intrinsics implementation. Differential Revision: http://reviews.llvm.org/D16813 llvm-svn: 260024
2016-02-07 16:30:50 +08:00
let isCodeGenOnly = 1 in {
defm VPBROADCASTBr : avx512_int_broadcast_reg_vl<0x7A, avx512vl_i8_info, GR8,
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
HasBWI>;
AVX512: VPBROADCASTB/W/D/Q from GPR intrinsics implementation. Differential Revision: http://reviews.llvm.org/D16813 llvm-svn: 260024
2016-02-07 16:30:50 +08:00
defm VPBROADCASTWr : avx512_int_broadcast_reg_vl<0x7B, avx512vl_i16_info, GR16,
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
HasBWI>;
AVX512: VPBROADCASTB/W/D/Q from GPR intrinsics implementation. Differential Revision: http://reviews.llvm.org/D16813 llvm-svn: 260024
2016-02-07 16:30:50 +08:00
}
let isAsmParserOnly = 1 in {
defm VPBROADCASTBr_Alt : avx512_int_broadcast_reg_vl<0x7A, avx512vl_i8_info,
GR32, HasBWI>;
defm VPBROADCASTWr_Alt : avx512_int_broadcast_reg_vl<0x7B, avx512vl_i16_info,
GR32, HasBWI>;
}
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
defm VPBROADCASTDr : avx512_int_broadcast_reg_vl<0x7C, avx512vl_i32_info, GR32,
HasAVX512>;
defm VPBROADCASTQr : avx512_int_broadcast_reg_vl<0x7C, avx512vl_i64_info, GR64,
HasAVX512>, VEX_W;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat <(v16i32 (X86vzext VK16WM:$mask)),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTDrZrkz VK16WM:$mask, (i32 (MOV32ri 0x1)))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat <(v8i64 (X86vzext VK8WM:$mask)),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTQrZrkz VK8WM:$mask, (i64 (MOV64ri 0x1)))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX512: Implemented encoding and intrinsics for VPBROADCASTB/W/D/Q instructions. Differential Revision: http://reviews.llvm.org/D13884 llvm-svn: 250819
2015-10-20 19:56:42 +08:00
// Provide aliases for broadcast from the same register class that
// automatically does the extract.
multiclass avx512_int_broadcast_rm_lowering<X86VectorVTInfo DestInfo,
X86VectorVTInfo SrcInfo> {
def : Pat<(DestInfo.VT (X86VBroadcast (SrcInfo.VT SrcInfo.RC:$src))),
(!cast<Instruction>(NAME#DestInfo.ZSuffix#"r")
(EXTRACT_SUBREG (SrcInfo.VT SrcInfo.RC:$src), sub_xmm))>;
}
multiclass avx512_int_broadcast_rm_vl<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _, Predicate prd> {
let Predicates = [prd] in {
defm Z : avx512_broadcast_rm<opc, OpcodeStr, _.info512, _.info128>,
avx512_int_broadcast_rm_lowering<_.info512, _.info256>,
EVEX_V512;
// Defined separately to avoid redefinition.
defm Z_Alt : avx512_int_broadcast_rm_lowering<_.info512, _.info512>;
}
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_broadcast_rm<opc, OpcodeStr, _.info256, _.info128>,
avx512_int_broadcast_rm_lowering<_.info256, _.info256>,
EVEX_V256;
defm Z128 : avx512_broadcast_rm<opc, OpcodeStr, _.info128, _.info128>,
EVEX_V128;
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX512: Implemented encoding and intrinsics for VPBROADCASTB/W/D/Q instructions. Differential Revision: http://reviews.llvm.org/D13884 llvm-svn: 250819
2015-10-20 19:56:42 +08:00
defm VPBROADCASTB : avx512_int_broadcast_rm_vl<0x78, "vpbroadcastb",
avx512vl_i8_info, HasBWI>;
defm VPBROADCASTW : avx512_int_broadcast_rm_vl<0x79, "vpbroadcastw",
avx512vl_i16_info, HasBWI>;
defm VPBROADCASTD : avx512_int_broadcast_rm_vl<0x58, "vpbroadcastd",
avx512vl_i32_info, HasAVX512>;
defm VPBROADCASTQ : avx512_int_broadcast_rm_vl<0x59, "vpbroadcastq",
avx512vl_i64_info, HasAVX512>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added VBROADCASTF64X4, VBROADCASTF64X2, VBROADCASTI32X8, and other instructions from this set Added encoding tests. llvm-svn: 237557
2015-05-18 14:42:57 +08:00
multiclass avx512_subvec_broadcast_rm<bits<8> opc, string OpcodeStr,
X86VectorVTInfo _Dst, X86VectorVTInfo _Src> {
[X86][AVX512] change broadcast to use maskable pattern Differential Revision: http://reviews.llvm.org/D15786 llvm-svn: 256469
2015-12-27 20:14:34 +08:00
let mayLoad = 1 in
defm rm : AVX512_maskable<opc, MRMSrcMem, _Dst, (outs _Dst.RC:$dst),
(ins _Src.MemOp:$src), OpcodeStr, "$src", "$src",
(_Dst.VT (X86SubVBroadcast
(_Src.VT (bitconvert (_Src.LdFrag addr:$src)))))>,
AVX5128IBase, EVEX;
[X86] AVX512: Add vbroadcasti* For now I used a separate template for these sub-vector/tuple broadcasts rather than sharing the mem variants with avx512_int_broadcast_rm. <rdar://problem/17402869> llvm-svn: 211828
2014-06-27 08:43:38 +08:00
}
AVX-512: Added VBROADCASTF64X4, VBROADCASTF64X2, VBROADCASTI32X8, and other instructions from this set Added encoding tests. llvm-svn: 237557
2015-05-18 14:42:57 +08:00
defm VBROADCASTI32X4 : avx512_subvec_broadcast_rm<0x5a, "vbroadcasti32x4",
v16i32_info, v4i32x_info>,
[X86] AVX512: Add vbroadcasti* For now I used a separate template for these sub-vector/tuple broadcasts rather than sharing the mem variants with avx512_int_broadcast_rm. <rdar://problem/17402869> llvm-svn: 211828
2014-06-27 08:43:38 +08:00
EVEX_V512, EVEX_CD8<32, CD8VT4>;
AVX-512: Added VBROADCASTF64X4, VBROADCASTF64X2, VBROADCASTI32X8, and other instructions from this set Added encoding tests. llvm-svn: 237557
2015-05-18 14:42:57 +08:00
defm VBROADCASTF32X4 : avx512_subvec_broadcast_rm<0x1a, "vbroadcastf32x4",
v16f32_info, v4f32x_info>,
EVEX_V512, EVEX_CD8<32, CD8VT4>;
defm VBROADCASTI64X4 : avx512_subvec_broadcast_rm<0x5b, "vbroadcasti64x4",
v8i64_info, v4i64x_info>, VEX_W,
EVEX_V512, EVEX_CD8<64, CD8VT4>;
defm VBROADCASTF64X4 : avx512_subvec_broadcast_rm<0x1b, "vbroadcastf64x4",
v8f64_info, v4f64x_info>, VEX_W,
[X86] AVX512: Add vbroadcasti* For now I used a separate template for these sub-vector/tuple broadcasts rather than sharing the mem variants with avx512_int_broadcast_rm. <rdar://problem/17402869> llvm-svn: 211828
2014-06-27 08:43:38 +08:00
EVEX_V512, EVEX_CD8<64, CD8VT4>;
AVX-512: Added VBROADCASTF64X4, VBROADCASTF64X2, VBROADCASTI32X8, and other instructions from this set Added encoding tests. llvm-svn: 237557
2015-05-18 14:42:57 +08:00
let Predicates = [HasVLX] in {
defm VBROADCASTI32X4Z256 : avx512_subvec_broadcast_rm<0x5a, "vbroadcasti32x4",
v8i32x_info, v4i32x_info>,
EVEX_V256, EVEX_CD8<32, CD8VT4>;
defm VBROADCASTF32X4Z256 : avx512_subvec_broadcast_rm<0x1a, "vbroadcastf32x4",
v8f32x_info, v4f32x_info>,
EVEX_V256, EVEX_CD8<32, CD8VT4>;
}
let Predicates = [HasVLX, HasDQI] in {
defm VBROADCASTI64X2Z128 : avx512_subvec_broadcast_rm<0x5a, "vbroadcasti64x2",
v4i64x_info, v2i64x_info>, VEX_W,
EVEX_V256, EVEX_CD8<64, CD8VT2>;
defm VBROADCASTF64X2Z128 : avx512_subvec_broadcast_rm<0x1a, "vbroadcastf64x2",
v4f64x_info, v2f64x_info>, VEX_W,
EVEX_V256, EVEX_CD8<64, CD8VT2>;
}
let Predicates = [HasDQI] in {
defm VBROADCASTI64X2 : avx512_subvec_broadcast_rm<0x5a, "vbroadcasti64x2",
v8i64_info, v2i64x_info>, VEX_W,
EVEX_V512, EVEX_CD8<64, CD8VT2>;
defm VBROADCASTI32X8 : avx512_subvec_broadcast_rm<0x5b, "vbroadcasti32x8",
v16i32_info, v8i32x_info>,
EVEX_V512, EVEX_CD8<32, CD8VT8>;
defm VBROADCASTF64X2 : avx512_subvec_broadcast_rm<0x1a, "vbroadcastf64x2",
v8f64_info, v2f64x_info>, VEX_W,
EVEX_V512, EVEX_CD8<64, CD8VT2>;
defm VBROADCASTF32X8 : avx512_subvec_broadcast_rm<0x1b, "vbroadcastf32x8",
v16f32_info, v8f32x_info>,
EVEX_V512, EVEX_CD8<32, CD8VT8>;
}
AVX512: Implemented encoding and intrinsics for VBROADCASTI32x2 and VBROADCASTF32x2 instructions. Differential Revision: http://reviews.llvm.org/D14216 llvm-svn: 251781
2015-11-02 15:39:36 +08:00
multiclass avx512_broadcast_32x2<bits<8> opc, string OpcodeStr,
X86VectorVTInfo _Dst, X86VectorVTInfo _Src,
SDNode OpNode = X86SubVBroadcast> {
defm r : AVX512_maskable<opc, MRMSrcReg, _Dst, (outs _Dst.RC:$dst),
(ins _Src.RC:$src), OpcodeStr, "$src", "$src",
(_Dst.VT (OpNode (_Src.VT _Src.RC:$src)))>,
T8PD, EVEX;
let mayLoad = 1 in
defm m : AVX512_maskable<opc, MRMSrcMem, _Dst, (outs _Dst.RC:$dst),
(ins _Src.ScalarMemOp:$src), OpcodeStr, "$src", "$src",
(_Dst.VT (OpNode
(_Src.VT (scalar_to_vector(loadi64 addr:$src)))))>,
T8PD, EVEX, EVEX_CD8<_Src.EltSize, CD8VT2>;
}
multiclass avx512_common_broadcast_32x2<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _> {
let Predicates = [HasDQI] in
defm Z : avx512_broadcast_32x2<opc, OpcodeStr, _.info512, _.info128>,
EVEX_V512;
let Predicates = [HasDQI, HasVLX] in
defm Z256 : avx512_broadcast_32x2<opc, OpcodeStr, _.info256, _.info128>,
EVEX_V256;
}
multiclass avx512_common_broadcast_i32x2<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _> :
avx512_common_broadcast_32x2<opc, OpcodeStr, _> {
let Predicates = [HasDQI, HasVLX] in
defm Z128 : avx512_broadcast_32x2<opc, OpcodeStr, _.info128, _.info128,
X86SubV32x2Broadcast>, EVEX_V128;
}
defm VPBROADCASTI32X2 : avx512_common_broadcast_i32x2<0x59, "vbroadcasti32x2",
avx512vl_i32_info>;
defm VPBROADCASTF32X2 : avx512_common_broadcast_32x2<0x19, "vbroadcastf32x2",
avx512vl_f32_info>;
[AVX512] Bring back vector-shuffle lowering support through broadcasts Ffter commit at rev219046 512-bit broadcasts lowering become non-optimal. Most of tests on broadcasting and embedded broadcasting were changed and they doesn’t produce efficient code. Example below is from commit changes (it’s the first test from test/CodeGen/X86/avx512-vbroadcast.ll): define <16 x i32> @_inreg16xi32(i32 %a) { ; CHECK-LABEL: _inreg16xi32: ; CHECK: ## BB#0: -; CHECK-NEXT: vpbroadcastd %edi, %zmm0 +; CHECK-NEXT: vmovd %edi, %xmm0 +; CHECK-NEXT: vpbroadcastd %xmm0, %ymm0 +; CHECK-NEXT: vinserti64x4 $1, %ymm0, %zmm0, %zmm0 ; CHECK-NEXT: retq %b = insertelement <16 x i32> undef, i32 %a, i32 0 %c = shufflevector <16 x i32> %b, <16 x i32> undef, <16 x i32> zeroinitializer ret <16 x i32> %c } Here, 256-bit broadcast was generated instead of 512-bit one. In this patch 1) I added vector-shuffle lowering through broadcasts 2) Removed asserts and branches likes because this is incorrect - assert(Subtarget->hasDQI() && "We can only lower v8i64 with AVX-512-DQI"); 3) Fixed lowering tests llvm-svn: 220774
2014-10-28 20:28:51 +08:00
def : Pat<(v16f32 (X86VBroadcast (v16f32 VR512:$src))),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSSZr (EXTRACT_SUBREG (v16f32 VR512:$src), sub_xmm))>;
AVX-512: Added patterns for scalar-to-vector broadcast llvm-svn: 237558
2015-05-18 15:06:23 +08:00
def : Pat<(v16f32 (X86VBroadcast (v8f32 VR256X:$src))),
(VBROADCASTSSZr (EXTRACT_SUBREG (v8f32 VR256X:$src), sub_xmm))>;
[AVX512] Bring back vector-shuffle lowering support through broadcasts Ffter commit at rev219046 512-bit broadcasts lowering become non-optimal. Most of tests on broadcasting and embedded broadcasting were changed and they doesn’t produce efficient code. Example below is from commit changes (it’s the first test from test/CodeGen/X86/avx512-vbroadcast.ll): define <16 x i32> @_inreg16xi32(i32 %a) { ; CHECK-LABEL: _inreg16xi32: ; CHECK: ## BB#0: -; CHECK-NEXT: vpbroadcastd %edi, %zmm0 +; CHECK-NEXT: vmovd %edi, %xmm0 +; CHECK-NEXT: vpbroadcastd %xmm0, %ymm0 +; CHECK-NEXT: vinserti64x4 $1, %ymm0, %zmm0, %zmm0 ; CHECK-NEXT: retq %b = insertelement <16 x i32> undef, i32 %a, i32 0 %c = shufflevector <16 x i32> %b, <16 x i32> undef, <16 x i32> zeroinitializer ret <16 x i32> %c } Here, 256-bit broadcast was generated instead of 512-bit one. In this patch 1) I added vector-shuffle lowering through broadcasts 2) Removed asserts and branches likes because this is incorrect - assert(Subtarget->hasDQI() && "We can only lower v8i64 with AVX-512-DQI"); 3) Fixed lowering tests llvm-svn: 220774
2014-10-28 20:28:51 +08:00
def : Pat<(v8f64 (X86VBroadcast (v8f64 VR512:$src))),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSDZr (EXTRACT_SUBREG (v8f64 VR512:$src), sub_xmm))>;
AVX-512: Added patterns for scalar-to-vector broadcast llvm-svn: 237558
2015-05-18 15:06:23 +08:00
def : Pat<(v8f64 (X86VBroadcast (v4f64 VR256X:$src))),
(VBROADCASTSDZr (EXTRACT_SUBREG (v4f64 VR256X:$src), sub_xmm))>;
[AVX512] Bring back vector-shuffle lowering support through broadcasts Ffter commit at rev219046 512-bit broadcasts lowering become non-optimal. Most of tests on broadcasting and embedded broadcasting were changed and they doesn’t produce efficient code. Example below is from commit changes (it’s the first test from test/CodeGen/X86/avx512-vbroadcast.ll): define <16 x i32> @_inreg16xi32(i32 %a) { ; CHECK-LABEL: _inreg16xi32: ; CHECK: ## BB#0: -; CHECK-NEXT: vpbroadcastd %edi, %zmm0 +; CHECK-NEXT: vmovd %edi, %xmm0 +; CHECK-NEXT: vpbroadcastd %xmm0, %ymm0 +; CHECK-NEXT: vinserti64x4 $1, %ymm0, %zmm0, %zmm0 ; CHECK-NEXT: retq %b = insertelement <16 x i32> undef, i32 %a, i32 0 %c = shufflevector <16 x i32> %b, <16 x i32> undef, <16 x i32> zeroinitializer ret <16 x i32> %c } Here, 256-bit broadcast was generated instead of 512-bit one. In this patch 1) I added vector-shuffle lowering through broadcasts 2) Removed asserts and branches likes because this is incorrect - assert(Subtarget->hasDQI() && "We can only lower v8i64 with AVX-512-DQI"); 3) Fixed lowering tests llvm-svn: 220774
2014-10-28 20:28:51 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 BROADCAST MASK TO VECTOR REGISTER
//---
[X86][AVX512CD] add mask broadcast intrinsics Differential Revision: http://reviews.llvm.org/D14573 llvm-svn: 253450
2015-11-18 17:42:45 +08:00
multiclass avx512_mask_broadcastm<bits<8> opc, string OpcodeStr,
X86VectorVTInfo _, RegisterClass KRC> {
def rr : AVX512XS8I<opc, MRMSrcReg, (outs _.RC:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[X86][AVX512CD] add mask broadcast intrinsics Differential Revision: http://reviews.llvm.org/D14573 llvm-svn: 253450
2015-11-18 17:42:45 +08:00
[(set _.RC:$dst, (_.VT (X86VBroadcastm KRC:$src)))]>, EVEX;
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
}
[X86][AVX512CD] add mask broadcast intrinsics Differential Revision: http://reviews.llvm.org/D14573 llvm-svn: 253450
2015-11-18 17:42:45 +08:00
multiclass avx512_mask_broadcast<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo, RegisterClass KRC> {
let Predicates = [HasCDI] in
defm Z : avx512_mask_broadcastm<opc, OpcodeStr, VTInfo.info512, KRC>, EVEX_V512;
let Predicates = [HasCDI, HasVLX] in {
defm Z256 : avx512_mask_broadcastm<opc, OpcodeStr, VTInfo.info256, KRC>, EVEX_V256;
defm Z128 : avx512_mask_broadcastm<opc, OpcodeStr, VTInfo.info128, KRC>, EVEX_V128;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
defm VPBROADCASTMW2D : avx512_mask_broadcast<0x3A, "vpbroadcastmw2d",
[X86][AVX512CD] add mask broadcast intrinsics Differential Revision: http://reviews.llvm.org/D14573 llvm-svn: 253450
2015-11-18 17:42:45 +08:00
avx512vl_i32_info, VK16>;
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
defm VPBROADCASTMB2Q : avx512_mask_broadcast<0x2A, "vpbroadcastmb2q",
[X86][AVX512CD] add mask broadcast intrinsics Differential Revision: http://reviews.llvm.org/D14573 llvm-svn: 253450
2015-11-18 17:42:45 +08:00
avx512vl_i64_info, VK8>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
// -- VPERMI2 - 3 source operands form --
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
multiclass avx512_perm_i<bits<8> opc, string OpcodeStr,
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
X86VectorVTInfo _, X86VectorVTInfo IdxVT> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Constraints = "$src1 = $dst" in {
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
defm rr: AVX512_maskable_3src_cast<opc, MRMSrcReg, _, IdxVT, (outs _.RC:$dst),
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
(ins _.RC:$src2, _.RC:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
(_.VT (X86VPermi2X IdxVT.RC:$src1, _.RC:$src2, _.RC:$src3))>, EVEX_4V,
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
AVX5128IBase;
let mayLoad = 1 in
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
defm rm: AVX512_maskable_3src_cast<opc, MRMSrcMem, _, IdxVT, (outs _.RC:$dst),
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
(ins _.RC:$src2, _.MemOp:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
(_.VT (X86VPermi2X IdxVT.RC:$src1, _.RC:$src2,
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
(_.VT (bitconvert (_.LdFrag addr:$src3)))))>,
EVEX_4V, AVX5128IBase;
AVX-512: Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 239999
2015-06-18 15:29:40 +08:00
}
}
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
multiclass avx512_perm_i_mb<bits<8> opc, string OpcodeStr,
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
X86VectorVTInfo _, X86VectorVTInfo IdxVT> {
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
let mayLoad = 1, Constraints = "$src1 = $dst" in
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
defm rmb: AVX512_maskable_3src_cast<opc, MRMSrcMem, _, IdxVT, (outs _.RC:$dst),
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
(ins _.RC:$src2, _.ScalarMemOp:$src3),
OpcodeStr, !strconcat("${src3}", _.BroadcastStr,", $src2"),
!strconcat("$src2, ${src3}", _.BroadcastStr ),
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
(_.VT (X86VPermi2X IdxVT.RC:$src1,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
_.RC:$src2,(_.VT (X86VBroadcast (_.ScalarLdFrag addr:$src3)))))>,
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
AVX5128IBase, EVEX_4V, EVEX_B;
AVX-512: Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 239999
2015-06-18 15:29:40 +08:00
}
reverted 239999 due to test failures llvm-svn: 240001
2015-06-18 16:06:49 +08:00
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
multiclass avx512_perm_i_sizes<bits<8> opc, string OpcodeStr,
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
AVX512VLVectorVTInfo VTInfo,
AVX512VLVectorVTInfo ShuffleMask> {
defm NAME: avx512_perm_i<opc, OpcodeStr, VTInfo.info512,
ShuffleMask.info512>,
avx512_perm_i_mb<opc, OpcodeStr, VTInfo.info512,
ShuffleMask.info512>, EVEX_V512;
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
let Predicates = [HasVLX] in {
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
defm NAME#128: avx512_perm_i<opc, OpcodeStr, VTInfo.info128,
ShuffleMask.info128>,
avx512_perm_i_mb<opc, OpcodeStr, VTInfo.info128,
ShuffleMask.info128>, EVEX_V128;
defm NAME#256: avx512_perm_i<opc, OpcodeStr, VTInfo.info256,
ShuffleMask.info256>,
avx512_perm_i_mb<opc, OpcodeStr, VTInfo.info256,
ShuffleMask.info256>, EVEX_V256;
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
}
}
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
[AVX512] Adding VPERMT2B and VPERMI2B instruction . Differential Revision: http://reviews.llvm.org/D16297 llvm-svn: 258161
2016-01-20 02:47:02 +08:00
multiclass avx512_perm_i_sizes_bw<bits<8> opc, string OpcodeStr,
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
AVX512VLVectorVTInfo VTInfo,
[AVX512] Adding VPERMT2B and VPERMI2B instruction . Differential Revision: http://reviews.llvm.org/D16297 llvm-svn: 258161
2016-01-20 02:47:02 +08:00
AVX512VLVectorVTInfo Idx,
Predicate Prd> {
let Predicates = [Prd] in
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
defm NAME: avx512_perm_i<opc, OpcodeStr, VTInfo.info512,
Idx.info512>, EVEX_V512;
[AVX512] Adding VPERMT2B and VPERMI2B instruction . Differential Revision: http://reviews.llvm.org/D16297 llvm-svn: 258161
2016-01-20 02:47:02 +08:00
let Predicates = [Prd, HasVLX] in {
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
defm NAME#128: avx512_perm_i<opc, OpcodeStr, VTInfo.info128,
Idx.info128>, EVEX_V128;
defm NAME#256: avx512_perm_i<opc, OpcodeStr, VTInfo.info256,
Idx.info256>, EVEX_V256;
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
}
}
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
defm VPERMI2D : avx512_perm_i_sizes<0x76, "vpermi2d",
avx512vl_i32_info, avx512vl_i32_info>, EVEX_CD8<32, CD8VF>;
defm VPERMI2Q : avx512_perm_i_sizes<0x76, "vpermi2q",
avx512vl_i64_info, avx512vl_i64_info>, VEX_W, EVEX_CD8<64, CD8VF>;
[AVX512] Adding VPERMT2B and VPERMI2B instruction . Differential Revision: http://reviews.llvm.org/D16297 llvm-svn: 258161
2016-01-20 02:47:02 +08:00
defm VPERMI2W : avx512_perm_i_sizes_bw<0x75, "vpermi2w",
avx512vl_i16_info, avx512vl_i16_info, HasBWI>,
VEX_W, EVEX_CD8<16, CD8VF>;
defm VPERMI2B : avx512_perm_i_sizes_bw<0x75, "vpermi2b",
avx512vl_i8_info, avx512vl_i8_info, HasVBMI>,
EVEX_CD8<8, CD8VF>;
[AVX512] The vpermi2 instructions require an integer vector for the index vector. This is reflected correctly in the intrinsics, but was not refelected in the isel patterns. For the floating point types, this requires adding a bitcast to the index vector when its passed through to the output. llvm-svn: 254277
2015-11-30 08:13:24 +08:00
defm VPERMI2PS : avx512_perm_i_sizes<0x77, "vpermi2ps",
avx512vl_f32_info, avx512vl_i32_info>, EVEX_CD8<32, CD8VF>;
defm VPERMI2PD : avx512_perm_i_sizes<0x77, "vpermi2pd",
avx512vl_f64_info, avx512vl_i64_info>, VEX_W, EVEX_CD8<64, CD8VF>;
// VPERMT2
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
multiclass avx512_perm_t<bits<8> opc, string OpcodeStr,
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
X86VectorVTInfo _, X86VectorVTInfo IdxVT> {
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
let Constraints = "$src1 = $dst" in {
defm rr: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins IdxVT.RC:$src2, _.RC:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
(_.VT (X86VPermt2 _.RC:$src1, IdxVT.RC:$src2, _.RC:$src3))>, EVEX_4V,
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
AVX5128IBase;
let mayLoad = 1 in
defm rm: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins IdxVT.RC:$src2, _.MemOp:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
(_.VT (X86VPermt2 _.RC:$src1, IdxVT.RC:$src2,
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
(bitconvert (_.LdFrag addr:$src3))))>,
EVEX_4V, AVX5128IBase;
AVX-512: (fixed) Added encoding of all forms of VPERMT2W/D/Q/PS/PD and VPERMI2W/D/Q/PS/PD. Intrinsics and tests for them are comming in the next patch. llvm-svn: 240003
2015-06-18 16:56:19 +08:00
}
}
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
multiclass avx512_perm_t_mb<bits<8> opc, string OpcodeStr,
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
X86VectorVTInfo _, X86VectorVTInfo IdxVT> {
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
let mayLoad = 1, Constraints = "$src1 = $dst" in
defm rmb: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins IdxVT.RC:$src2, _.ScalarMemOp:$src3),
OpcodeStr, !strconcat("${src3}", _.BroadcastStr,", $src2"),
!strconcat("$src2, ${src3}", _.BroadcastStr ),
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
(_.VT (X86VPermt2 _.RC:$src1,
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
IdxVT.RC:$src2,(_.VT (X86VBroadcast (_.ScalarLdFrag addr:$src3)))))>,
AVX5128IBase, EVEX_4V, EVEX_B;
}
multiclass avx512_perm_t_sizes<bits<8> opc, string OpcodeStr,
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
AVX512VLVectorVTInfo VTInfo,
AVX512VLVectorVTInfo ShuffleMask> {
defm NAME: avx512_perm_t<opc, OpcodeStr, VTInfo.info512,
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
ShuffleMask.info512>,
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
avx512_perm_t_mb<opc, OpcodeStr, VTInfo.info512,
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
ShuffleMask.info512>, EVEX_V512;
let Predicates = [HasVLX] in {
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
defm NAME#128: avx512_perm_t<opc, OpcodeStr, VTInfo.info128,
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
ShuffleMask.info128>,
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
avx512_perm_t_mb<opc, OpcodeStr, VTInfo.info128,
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
ShuffleMask.info128>, EVEX_V128;
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
defm NAME#256: avx512_perm_t<opc, OpcodeStr, VTInfo.info256,
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
ShuffleMask.info256>,
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
avx512_perm_t_mb<opc, OpcodeStr, VTInfo.info256,
ShuffleMask.info256>, EVEX_V256;
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
}
}
[AVX512] Adding VPERMT2B and VPERMI2B instruction . Differential Revision: http://reviews.llvm.org/D16297 llvm-svn: 258161
2016-01-20 02:47:02 +08:00
multiclass avx512_perm_t_sizes_bw<bits<8> opc, string OpcodeStr,
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
AVX512VLVectorVTInfo VTInfo,
[AVX512] Adding VPERMT2B and VPERMI2B instruction . Differential Revision: http://reviews.llvm.org/D16297 llvm-svn: 258161
2016-01-20 02:47:02 +08:00
AVX512VLVectorVTInfo Idx,
Predicate Prd> {
let Predicates = [Prd] in
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
defm NAME: avx512_perm_t<opc, OpcodeStr, VTInfo.info512,
Idx.info512>, EVEX_V512;
[AVX512] Adding VPERMT2B and VPERMI2B instruction . Differential Revision: http://reviews.llvm.org/D16297 llvm-svn: 258161
2016-01-20 02:47:02 +08:00
let Predicates = [Prd, HasVLX] in {
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
defm NAME#128: avx512_perm_t<opc, OpcodeStr, VTInfo.info128,
Idx.info128>, EVEX_V128;
defm NAME#256: avx512_perm_t<opc, OpcodeStr, VTInfo.info256,
Idx.info256>, EVEX_V256;
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
}
}
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
defm VPERMT2D : avx512_perm_t_sizes<0x7E, "vpermt2d",
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
avx512vl_i32_info, avx512vl_i32_info>, EVEX_CD8<32, CD8VF>;
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
defm VPERMT2Q : avx512_perm_t_sizes<0x7E, "vpermt2q",
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
avx512vl_i64_info, avx512vl_i64_info>, VEX_W, EVEX_CD8<64, CD8VF>;
[AVX512] Adding VPERMT2B and VPERMI2B instruction . Differential Revision: http://reviews.llvm.org/D16297 llvm-svn: 258161
2016-01-20 02:47:02 +08:00
defm VPERMT2W : avx512_perm_t_sizes_bw<0x7D, "vpermt2w",
avx512vl_i16_info, avx512vl_i16_info, HasBWI>,
VEX_W, EVEX_CD8<16, CD8VF>;
defm VPERMT2B : avx512_perm_t_sizes_bw<0x7D, "vpermt2b",
avx512vl_i8_info, avx512vl_i8_info, HasVBMI>,
EVEX_CD8<8, CD8VF>;
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
defm VPERMT2PS : avx512_perm_t_sizes<0x7F, "vpermt2ps",
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
avx512vl_f32_info, avx512vl_i32_info>, EVEX_CD8<32, CD8VF>;
[X86] Now that X86VPermt2 is used in all the avx512_perm_t_sizes just hardcode it into the patterns instead of passing as an argument. NFC llvm-svn: 254177
2015-11-27 04:21:29 +08:00
defm VPERMT2PD : avx512_perm_t_sizes<0x7F, "vpermt2pd",
AVX-512: Fixed a bug in VPERMT2* intrinsic. It was wrong order of operands (from intrinsic to DAG node). I added more strict type specification for instruction selection. Differential Revision: http://reviews.llvm.org/D14942 llvm-svn: 254059
2015-11-25 16:17:56 +08:00
avx512vl_f64_info, avx512vl_i64_info>, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: optimized a shuffle pattern to VINSERTI64x4. Added intrinsics for VPERMT2PS/PD/D/Q instructions. llvm-svn: 207513
2014-04-29 17:09:15 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - BLEND using mask
//
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
multiclass avx512_blendmask<bits<8> opc, string OpcodeStr, X86VectorVTInfo _> {
let ExeDomain = _.ExeDomain in {
def rr : AVX5128I<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr,
[AVX-512] Remove more superfluous spaces from asm strings. llvm-svn: 257301
2016-01-11 08:44:58 +08:00
"\t{$src2, $src1, ${dst}|${dst}, $src1, $src2}"),
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
[]>, EVEX_4V;
def rrk : AVX5128I<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2),
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, ${dst} {${mask}}|${dst} {${mask}}, $src1, $src2}"),
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
[(set _.RC:$dst, (X86select _.KRCWM:$mask, (_.VT _.RC:$src1),
(_.VT _.RC:$src2)))]>, EVEX_4V, EVEX_K;
def rrkz : AVX5128I<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr,
"\t{$src2, $src1, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src1, $src2}"),
[]>, EVEX_4V, EVEX_KZ;
let mayLoad = 1 in {
def rm : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, _.MemOp:$src2),
!strconcat(OpcodeStr,
[AVX-512] Remove more superfluous spaces from asm strings. llvm-svn: 257301
2016-01-11 08:44:58 +08:00
"\t{$src2, $src1, ${dst}|${dst}, $src1, $src2}"),
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
[]>, EVEX_4V, EVEX_CD8<_.EltSize, CD8VF>;
def rmk : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2),
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, ${dst} {${mask}}|${dst} {${mask}}, $src1, $src2}"),
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
[(set _.RC:$dst, (X86select _.KRCWM:$mask, (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2)))))]>,
EVEX_4V, EVEX_K, EVEX_CD8<_.EltSize, CD8VF>;
def rmkz : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2),
!strconcat(OpcodeStr,
"\t{$src2, $src1, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src1, $src2}"),
[]>, EVEX_4V, EVEX_KZ, EVEX_CD8<_.EltSize, CD8VF>;
}
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
multiclass avx512_blendmask_rmb<bits<8> opc, string OpcodeStr, X86VectorVTInfo _> {
def rmbk : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.ScalarMemOp:$src2),
!strconcat(OpcodeStr,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.RC:$dst,(X86select _.KRCWM:$mask, (_.VT _.RC:$src1),
(X86VBroadcast (_.ScalarLdFrag addr:$src2))))]>,
AVX-512: BLENDM - fixed encoding of the broadcast version Added more intrinsics and encoding tests. llvm-svn: 224760
2014-12-23 17:36:28 +08:00
EVEX_4V, EVEX_K, EVEX_B, EVEX_CD8<_.EltSize, CD8VF>;
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
def rmb : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2),
!strconcat(OpcodeStr,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst|",
"$dst, $src1, ${src2}", _.BroadcastStr, "}"),
AVX-512: BLENDM - fixed encoding of the broadcast version Added more intrinsics and encoding tests. llvm-svn: 224760
2014-12-23 17:36:28 +08:00
[]>, EVEX_4V, EVEX_B, EVEX_CD8<_.EltSize, CD8VF>;
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
}
multiclass blendmask_dq <bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
defm Z : avx512_blendmask <opc, OpcodeStr, VTInfo.info512>,
avx512_blendmask_rmb <opc, OpcodeStr, VTInfo.info512>, EVEX_V512;
let Predicates = [HasVLX] in {
defm Z256 : avx512_blendmask<opc, OpcodeStr, VTInfo.info256>,
avx512_blendmask_rmb <opc, OpcodeStr, VTInfo.info256>, EVEX_V256;
defm Z128 : avx512_blendmask<opc, OpcodeStr, VTInfo.info128>,
avx512_blendmask_rmb <opc, OpcodeStr, VTInfo.info128>, EVEX_V128;
}
}
multiclass blendmask_bw <bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
let Predicates = [HasBWI] in
defm Z : avx512_blendmask <opc, OpcodeStr, VTInfo.info512>, EVEX_V512;
let Predicates = [HasBWI, HasVLX] in {
defm Z256 : avx512_blendmask <opc, OpcodeStr, VTInfo.info256>, EVEX_V256;
defm Z128 : avx512_blendmask <opc, OpcodeStr, VTInfo.info128>, EVEX_V128;
}
}
defm VBLENDMPS : blendmask_dq <0x65, "vblendmps", avx512vl_f32_info>;
defm VBLENDMPD : blendmask_dq <0x65, "vblendmpd", avx512vl_f64_info>, VEX_W;
defm VPBLENDMD : blendmask_dq <0x64, "vpblendmd", avx512vl_i32_info>;
defm VPBLENDMQ : blendmask_dq <0x64, "vpblendmq", avx512vl_i64_info>, VEX_W;
defm VPBLENDMB : blendmask_bw <0x66, "vpblendmb", avx512vl_i8_info>;
defm VPBLENDMW : blendmask_bw <0x66, "vpblendmw", avx512vl_i16_info>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(v8f32 (vselect (v8i1 VK8WM:$mask), (v8f32 VR256X:$src1),
(v8f32 VR256X:$src2))),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(EXTRACT_SUBREG
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
(v16f32 (VBLENDMPSZrrk (COPY_TO_REGCLASS VK8WM:$mask, VK16WM),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_ymm)>;
def : Pat<(v8i32 (vselect (v8i1 VK8WM:$mask), (v8i32 VR256X:$src1),
(v8i32 VR256X:$src2))),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(EXTRACT_SUBREG
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
(v16i32 (VPBLENDMDZrrk (COPY_TO_REGCLASS VK8WM:$mask, VK16WM),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_ymm)>;
}
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
//===----------------------------------------------------------------------===//
// Compare Instructions
//===----------------------------------------------------------------------===//
// avx512_cmp_scalar - AVX512 CMPSS and CMPSD
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
multiclass avx512_cmp_scalar<X86VectorVTInfo _, SDNode OpNode, SDNode OpNodeRnd>{
defm rr_Int : AVX512_maskable_cmp<0xC2, MRMSrcReg, _,
(outs _.KRC:$dst),
(ins _.RC:$src1, _.RC:$src2, AVXCC:$cc),
"vcmp${cc}"#_.Suffix,
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
imm:$cc)>, EVEX_4V;
let mayLoad = 1 in
defm rm_Int : AVX512_maskable_cmp<0xC2, MRMSrcMem, _,
(outs _.KRC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2, AVXCC:$cc),
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
"vcmp${cc}"#_.Suffix,
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1),
(_.VT (scalar_to_vector (_.ScalarLdFrag addr:$src2))),
imm:$cc)>, EVEX_4V, EVEX_CD8<_.EltSize, CD8VT1>;
defm rrb_Int : AVX512_maskable_cmp<0xC2, MRMSrcReg, _,
(outs _.KRC:$dst),
(ins _.RC:$src1, _.RC:$src2, AVXCC:$cc),
"vcmp${cc}"#_.Suffix,
[AVX-512] Make spacing between comma and {sae} operand consistent in asm strings. llvm-svn: 257299
2016-01-11 08:44:52 +08:00
"{sae}, $src2, $src1", "$src1, $src2, {sae}",
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
(OpNodeRnd (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
imm:$cc,
(i32 FROUND_NO_EXC))>, EVEX_4V, EVEX_B;
// Accept explicit immediate argument form instead of comparison code.
let isAsmParserOnly = 1, hasSideEffects = 0 in {
defm rri_alt : AVX512_maskable_cmp_alt<0xC2, MRMSrcReg, _,
(outs VK1:$dst),
(ins _.RC:$src1, _.RC:$src2, u8imm:$cc),
"vcmp"#_.Suffix,
"$cc, $src2, $src1", "$src1, $src2, $cc">, EVEX_4V;
defm rmi_alt : AVX512_maskable_cmp_alt<0xC2, MRMSrcMem, _,
(outs _.KRC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2, u8imm:$cc),
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
"vcmp"#_.Suffix,
"$cc, $src2, $src1", "$src1, $src2, $cc">,
EVEX_4V, EVEX_CD8<_.EltSize, CD8VT1>;
defm rrb_alt : AVX512_maskable_cmp_alt<0xC2, MRMSrcReg, _,
(outs _.KRC:$dst),
(ins _.RC:$src1, _.RC:$src2, u8imm:$cc),
"vcmp"#_.Suffix,
[AVX-512] Make spacing between comma and {sae} operand consistent in asm strings. llvm-svn: 257299
2016-01-11 08:44:52 +08:00
"$cc, {sae}, $src2, $src1","$src1, $src2, {sae}, $cc">,
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
EVEX_4V, EVEX_B;
}// let isAsmParserOnly = 1, hasSideEffects = 0
let isCodeGenOnly = 1 in {
def rr : AVX512Ii8<0xC2, MRMSrcReg,
(outs _.KRC:$dst), (ins _.FRC:$src1, _.FRC:$src2, AVXCC:$cc),
!strconcat("vcmp${cc}", _.Suffix,
Simplify a multiclass. No functional change. llvm-svn: 227038
2015-01-25 16:49:19 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
[(set _.KRC:$dst, (OpNode _.FRC:$src1,
_.FRC:$src2,
imm:$cc))],
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
IIC_SSE_ALU_F32S_RR>, EVEX_4V;
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let mayLoad = 1 in
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
def rm : AVX512Ii8<0xC2, MRMSrcMem,
(outs _.KRC:$dst),
(ins _.FRC:$src1, _.ScalarMemOp:$src2, AVXCC:$cc),
!strconcat("vcmp${cc}", _.Suffix,
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set _.KRC:$dst, (OpNode _.FRC:$src1,
(_.ScalarLdFrag addr:$src2),
imm:$cc))],
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_CD8<_.EltSize, CD8VT1>;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
}
}
let Predicates = [HasAVX512] in {
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
defm VCMPSSZ : avx512_cmp_scalar<f32x_info, X86cmpms, X86cmpmsRnd>,
AVX512XSIi8Base;
defm VCMPSDZ : avx512_cmp_scalar<f64x_info, X86cmpms, X86cmpmsRnd>,
AVX512XDIi8Base, VEX_W;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
multiclass avx512_icmp_packed<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : AVX512BI<opc, MRMSrcReg,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2)))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_ALU_F32P_RR>, EVEX_4V;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
let mayLoad = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rm : AVX512BI<opc, MRMSrcMem,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.MemOp:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2)))))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
def rrk : AVX512BI<opc, MRMSrcReg,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2))))],
IIC_SSE_ALU_F32P_RR>, EVEX_4V, EVEX_K;
let mayLoad = 1 in
def rmk : AVX512BI<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert
(_.LdFrag addr:$src2))))))],
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K;
}
multiclass avx512_icmp_packed_rmb<bits<8> opc, string OpcodeStr, SDNode OpNode,
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
X86VectorVTInfo _> :
avx512_icmp_packed<opc, OpcodeStr, OpNode, _> {
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
let mayLoad = 1 in {
def rmb : AVX512BI<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.RC:$src1, _.ScalarMemOp:$src2),
!strconcat(OpcodeStr, "\t{${src2}", _.BroadcastStr, ", $src1, $dst",
"|$dst, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1),
(X86VBroadcast (_.ScalarLdFrag addr:$src2))))],
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_B;
def rmbk : AVX512BI<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1,
_.ScalarMemOp:$src2),
!strconcat(OpcodeStr,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(X86VBroadcast
(_.ScalarLdFrag addr:$src2)))))],
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K, EVEX_B;
}
}
multiclass avx512_icmp_packed_vl<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_packed<opc, OpcodeStr, OpNode, VTInfo.info512>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_packed<opc, OpcodeStr, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_icmp_packed<opc, OpcodeStr, OpNode, VTInfo.info128>,
EVEX_V128;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
multiclass avx512_icmp_packed_rmb_vl<bits<8> opc, string OpcodeStr,
SDNode OpNode, AVX512VLVectorVTInfo VTInfo,
Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_packed_rmb<opc, OpcodeStr, OpNode, VTInfo.info512>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_packed_rmb<opc, OpcodeStr, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_icmp_packed_rmb<opc, OpcodeStr, OpNode, VTInfo.info128>,
EVEX_V128;
}
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
defm VPCMPEQB : avx512_icmp_packed_vl<0x74, "vpcmpeqb", X86pcmpeqm,
avx512vl_i8_info, HasBWI>,
EVEX_CD8<8, CD8VF>;
defm VPCMPEQW : avx512_icmp_packed_vl<0x75, "vpcmpeqw", X86pcmpeqm,
avx512vl_i16_info, HasBWI>,
EVEX_CD8<16, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPEQD : avx512_icmp_packed_rmb_vl<0x76, "vpcmpeqd", X86pcmpeqm,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
avx512vl_i32_info, HasAVX512>,
EVEX_CD8<32, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPEQQ : avx512_icmp_packed_rmb_vl<0x29, "vpcmpeqq", X86pcmpeqm,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
avx512vl_i64_info, HasAVX512>,
T8PD, VEX_W, EVEX_CD8<64, CD8VF>;
defm VPCMPGTB : avx512_icmp_packed_vl<0x64, "vpcmpgtb", X86pcmpgtm,
avx512vl_i8_info, HasBWI>,
EVEX_CD8<8, CD8VF>;
defm VPCMPGTW : avx512_icmp_packed_vl<0x65, "vpcmpgtw", X86pcmpgtm,
avx512vl_i16_info, HasBWI>,
EVEX_CD8<16, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPGTD : avx512_icmp_packed_rmb_vl<0x66, "vpcmpgtd", X86pcmpgtm,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
avx512vl_i32_info, HasAVX512>,
EVEX_CD8<32, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPGTQ : avx512_icmp_packed_rmb_vl<0x37, "vpcmpgtq", X86pcmpgtm,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
avx512vl_i64_info, HasAVX512>,
T8PD, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i1 (X86pcmpgtm (v8i32 VR256X:$src1), (v8i32 VR256X:$src2))),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
(COPY_TO_REGCLASS (VPCMPGTDZrr
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm))), VK8)>;
def : Pat<(v8i1 (X86pcmpeqm (v8i32 VR256X:$src1), (v8i32 VR256X:$src2))),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
(COPY_TO_REGCLASS (VPCMPEQDZrr
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm))), VK8)>;
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
multiclass avx512_icmp_cc<bits<8> opc, string Suffix, SDNode OpNode,
X86VectorVTInfo _> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rri : AVX512AIi8<opc, MRMSrcReg,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2, AVX512ICC:$cc),
[X86] AVX512: Factor generating the AsmString into avx512_icmp_cc Adding a writemask variant would require a third asm string to be passed to the template. Generate the AsmString in the template instead. No change in X86.td.expanded. llvm-svn: 212113
2014-07-02 02:03:43 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
imm:$cc))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_ALU_F32P_RR>, EVEX_4V;
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
let mayLoad = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rmi : AVX512AIi8<opc, MRMSrcMem,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.MemOp:$src2, AVX512ICC:$cc),
[X86] AVX512: Factor generating the AsmString into avx512_icmp_cc Adding a writemask variant would require a third asm string to be passed to the template. Generate the AsmString in the template instead. No change in X86.td.expanded. llvm-svn: 212113
2014-07-02 02:03:43 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2))),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V;
def rrik : AVX512AIi8<opc, MRMSrcReg,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
AVX512ICC:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{$src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc)))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RR>, EVEX_4V, EVEX_K;
let mayLoad = 1 in
def rmik : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
AVX512ICC:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{$src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2))),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc)))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Accept explicit immediate argument form instead of comparison code.
Mark x86 _alt instructions as AsmParserOnly so they will be omitted from disassembler without string matches. llvm-svn: 198545
2014-01-05 12:55:55 +08:00
let isAsmParserOnly = 1, hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rri_alt : AVX512AIi8<opc, MRMSrcReg,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2, u8imm:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp", Suffix, "\t{$cc, $src2, $src1, $dst|",
"$dst, $src1, $src2, $cc}"),
[X86] AVX512: Factor generating the AsmString into avx512_icmp_cc Adding a writemask variant would require a third asm string to be passed to the template. Generate the AsmString in the template instead. No change in X86.td.expanded. llvm-svn: 212113
2014-07-02 02:03:43 +08:00
[], IIC_SSE_ALU_F32P_RR>, EVEX_4V;
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let mayLoad = 1 in
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
def rmi_alt : AVX512AIi8<opc, MRMSrcMem,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.MemOp:$src2, u8imm:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp", Suffix, "\t{$cc, $src2, $src1, $dst|",
"$dst, $src1, $src2, $cc}"),
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V;
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
def rrik_alt : AVX512AIi8<opc, MRMSrcReg,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
u8imm:$cc),
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
!strconcat("vpcmp", Suffix,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
"\t{$cc, $src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2, $cc}"),
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
[], IIC_SSE_ALU_F32P_RR>, EVEX_4V, EVEX_K;
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let mayLoad = 1 in
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
def rmik_alt : AVX512AIi8<opc, MRMSrcMem,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
u8imm:$cc),
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
!strconcat("vpcmp", Suffix,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
"\t{$cc, $src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2, $cc}"),
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
multiclass avx512_icmp_cc_rmb<bits<8> opc, string Suffix, SDNode OpNode,
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
X86VectorVTInfo _> :
avx512_icmp_cc<opc, Suffix, OpNode, _> {
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
def rmib : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.RC:$src1, _.ScalarMemOp:$src2,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
AVX512ICC:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst|",
"$dst, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1),
(X86VBroadcast (_.ScalarLdFrag addr:$src2)),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_B;
def rmibk : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
_.ScalarMemOp:$src2, AVX512ICC:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(X86VBroadcast (_.ScalarLdFrag addr:$src2)),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc)))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K, EVEX_B;
// Accept explicit immediate argument form instead of comparison code.
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let isAsmParserOnly = 1, hasSideEffects = 0, mayLoad = 1 in {
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
def rmib_alt : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.RC:$src1, _.ScalarMemOp:$src2,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
u8imm:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp", Suffix,
"\t{$cc, ${src2}", _.BroadcastStr, ", $src1, $dst|",
"$dst, $src1, ${src2}", _.BroadcastStr, ", $cc}"),
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_B;
def rmibk_alt : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
_.ScalarMemOp:$src2, u8imm:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp", Suffix,
"\t{$cc, ${src2}", _.BroadcastStr, ", $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, ${src2}", _.BroadcastStr, ", $cc}"),
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K, EVEX_B;
}
}
multiclass avx512_icmp_cc_vl<bits<8> opc, string Suffix, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_cc<opc, Suffix, OpNode, VTInfo.info512>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_cc<opc, Suffix, OpNode, VTInfo.info256>, EVEX_V256;
defm Z128 : avx512_icmp_cc<opc, Suffix, OpNode, VTInfo.info128>, EVEX_V128;
}
}
multiclass avx512_icmp_cc_rmb_vl<bits<8> opc, string Suffix, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_cc_rmb<opc, Suffix, OpNode, VTInfo.info512>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_cc_rmb<opc, Suffix, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_icmp_cc_rmb<opc, Suffix, OpNode, VTInfo.info128>,
EVEX_V128;
}
}
defm VPCMPB : avx512_icmp_cc_vl<0x3F, "b", X86cmpm, avx512vl_i8_info,
HasBWI>, EVEX_CD8<8, CD8VF>;
defm VPCMPUB : avx512_icmp_cc_vl<0x3E, "ub", X86cmpmu, avx512vl_i8_info,
HasBWI>, EVEX_CD8<8, CD8VF>;
defm VPCMPW : avx512_icmp_cc_vl<0x3F, "w", X86cmpm, avx512vl_i16_info,
HasBWI>, VEX_W, EVEX_CD8<16, CD8VF>;
defm VPCMPUW : avx512_icmp_cc_vl<0x3E, "uw", X86cmpmu, avx512vl_i16_info,
HasBWI>, VEX_W, EVEX_CD8<16, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPD : avx512_icmp_cc_rmb_vl<0x1F, "d", X86cmpm, avx512vl_i32_info,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
HasAVX512>, EVEX_CD8<32, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPUD : avx512_icmp_cc_rmb_vl<0x1E, "ud", X86cmpmu, avx512vl_i32_info,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
HasAVX512>, EVEX_CD8<32, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPQ : avx512_icmp_cc_rmb_vl<0x1F, "q", X86cmpm, avx512vl_i64_info,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
HasAVX512>, VEX_W, EVEX_CD8<64, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPUQ : avx512_icmp_cc_rmb_vl<0x1E, "uq", X86cmpmu, avx512vl_i64_info,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
HasAVX512>, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
multiclass avx512_vcmp_common<X86VectorVTInfo _> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
defm rri : AVX512_maskable_cmp<0xC2, MRMSrcReg, _,
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2,AVXCC:$cc),
"vcmp${cc}"#_.Suffix,
"$src2, $src1", "$src1, $src2",
(X86cmpm (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
imm:$cc)>;
let mayLoad = 1 in {
defm rmi : AVX512_maskable_cmp<0xC2, MRMSrcMem, _,
(outs _.KRC:$dst),(ins _.RC:$src1, _.MemOp:$src2, AVXCC:$cc),
"vcmp${cc}"#_.Suffix,
"$src2, $src1", "$src1, $src2",
(X86cmpm (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2))),
imm:$cc)>;
defm rmbi : AVX512_maskable_cmp<0xC2, MRMSrcMem, _,
(outs _.KRC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2, AVXCC:$cc),
"vcmp${cc}"#_.Suffix,
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(X86cmpm (_.VT _.RC:$src1),
(_.VT (X86VBroadcast(_.ScalarLdFrag addr:$src2))),
imm:$cc)>,EVEX_B;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Accept explicit immediate argument form instead of comparison code.
Mark x86 _alt instructions as AsmParserOnly so they will be omitted from disassembler without string matches. llvm-svn: 198545
2014-01-05 12:55:55 +08:00
let isAsmParserOnly = 1, hasSideEffects = 0 in {
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
defm rri_alt : AVX512_maskable_cmp_alt<0xC2, MRMSrcReg, _,
(outs _.KRC:$dst),
(ins _.RC:$src1, _.RC:$src2, u8imm:$cc),
"vcmp"#_.Suffix,
"$cc, $src2, $src1", "$src1, $src2, $cc">;
let mayLoad = 1 in {
defm rmi_alt : AVX512_maskable_cmp_alt<0xC2, MRMSrcMem, _,
(outs _.KRC:$dst),
(ins _.RC:$src1, _.MemOp:$src2, u8imm:$cc),
"vcmp"#_.Suffix,
"$cc, $src2, $src1", "$src1, $src2, $cc">;
defm rmbi_alt : AVX512_maskable_cmp_alt<0xC2, MRMSrcMem, _,
(outs _.KRC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2, u8imm:$cc),
"vcmp"#_.Suffix,
"$cc, ${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr##", $cc">,EVEX_B;
}
}
}
multiclass avx512_vcmp_sae<X86VectorVTInfo _> {
// comparison code form (VCMP[EQ/LT/LE/...]
defm rrib : AVX512_maskable_cmp<0xC2, MRMSrcReg, _,
(outs _.KRC:$dst),(ins _.RC:$src1, _.RC:$src2, AVXCC:$cc),
"vcmp${cc}"#_.Suffix,
[AVX-512] Make spacing between comma and {sae} operand consistent in asm strings. llvm-svn: 257299
2016-01-11 08:44:52 +08:00
"{sae}, $src2, $src1", "$src1, $src2, {sae}",
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
(X86cmpmRnd (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
imm:$cc,
(i32 FROUND_NO_EXC))>, EVEX_B;
let isAsmParserOnly = 1, hasSideEffects = 0 in {
defm rrib_alt : AVX512_maskable_cmp_alt<0xC2, MRMSrcReg, _,
(outs _.KRC:$dst),
(ins _.RC:$src1, _.RC:$src2, u8imm:$cc),
"vcmp"#_.Suffix,
[AVX-512] Make spacing between comma and {sae} operand consistent in asm strings. llvm-svn: 257299
2016-01-11 08:44:52 +08:00
"$cc, {sae}, $src2, $src1",
"$src1, $src2, {sae}, $cc">, EVEX_B;
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
}
}
multiclass avx512_vcmp<AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in {
defm Z : avx512_vcmp_common<_.info512>,
avx512_vcmp_sae<_.info512>, EVEX_V512;
}
let Predicates = [HasAVX512,HasVLX] in {
defm Z128 : avx512_vcmp_common<_.info128>, EVEX_V128;
defm Z256 : avx512_vcmp_common<_.info256>, EVEX_V256;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
defm VCMPPD : avx512_vcmp<avx512vl_f64_info>,
AVX512PDIi8Base, EVEX_4V, EVEX_CD8<64, CD8VF>, VEX_W;
defm VCMPPS : avx512_vcmp<avx512vl_f32_info>,
AVX512PSIi8Base, EVEX_4V, EVEX_CD8<32, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i1 (X86cmpm (v8f32 VR256X:$src1), (v8f32 VR256X:$src2), imm:$cc)),
(COPY_TO_REGCLASS (VCMPPSZrri
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
imm:$cc), VK8)>;
def : Pat<(v8i1 (X86cmpm (v8i32 VR256X:$src1), (v8i32 VR256X:$src2), imm:$cc)),
(COPY_TO_REGCLASS (VPCMPDZrri
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
imm:$cc), VK8)>;
def : Pat<(v8i1 (X86cmpmu (v8i32 VR256X:$src1), (v8i32 VR256X:$src2), imm:$cc)),
(COPY_TO_REGCLASS (VPCMPUDZrri
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
imm:$cc), VK8)>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
// ----------------------------------------------------------------
// FPClass
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
//handle fpclass instruction mask = op(reg_scalar,imm)
// op(mem_scalar,imm)
multiclass avx512_scalar_fpclass<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _, Predicate prd> {
let Predicates = [prd] in {
def rr : AVX512<opc, MRMSrcReg, (outs _.KRC:$dst),//_.KRC:$dst),
(ins _.RC:$src1, i32u8imm:$src2),
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
OpcodeStr##_.Suffix#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
[(set _.KRC:$dst,(OpNode (_.VT _.RC:$src1),
(i32 imm:$src2)))], NoItinerary>;
def rrk : AVX512<opc, MRMSrcReg, (outs _.KRC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix#
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
"\t{$src2, $src1, $dst {${mask}}|$dst {${mask}}, $src1, $src2}",
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
[(set _.KRC:$dst,(or _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(i32 imm:$src2))))], NoItinerary>, EVEX_K;
let mayLoad = 1, AddedComplexity = 20 in {
def rm : AVX512<opc, MRMSrcMem, (outs _.KRC:$dst),
(ins _.MemOp:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix##
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
[(set _.KRC:$dst,
(OpNode (_.VT (bitconvert (_.LdFrag addr:$src1))),
(i32 imm:$src2)))], NoItinerary>;
def rmk : AVX512<opc, MRMSrcMem, (outs _.KRC:$dst),
(ins _.KRCWM:$mask, _.MemOp:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix##
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
"\t{$src2, $src1, $dst {${mask}}|$dst {${mask}}, $src1, $src2}",
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
[(set _.KRC:$dst,(or _.KRCWM:$mask,
(OpNode (_.VT (bitconvert (_.LdFrag addr:$src1))),
(i32 imm:$src2))))], NoItinerary>, EVEX_K;
}
}
}
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
//handle fpclass instruction mask = fpclass(reg_vec, reg_vec, imm)
// fpclass(reg_vec, mem_vec, imm)
// fpclass(reg_vec, broadcast(eltVt), imm)
multiclass avx512_vector_fpclass<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _, string mem, string broadcast>{
def rr : AVX512<opc, MRMSrcReg, (outs _.KRC:$dst),
(ins _.RC:$src1, i32u8imm:$src2),
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
OpcodeStr##_.Suffix#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
[(set _.KRC:$dst,(OpNode (_.VT _.RC:$src1),
(i32 imm:$src2)))], NoItinerary>;
def rrk : AVX512<opc, MRMSrcReg, (outs _.KRC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix#
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
"\t{$src2, $src1, $dst {${mask}}|$dst {${mask}}, $src1, $src2}",
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
[(set _.KRC:$dst,(or _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(i32 imm:$src2))))], NoItinerary>, EVEX_K;
let mayLoad = 1 in {
def rm : AVX512<opc, MRMSrcMem, (outs _.KRC:$dst),
(ins _.MemOp:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix##mem#
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
[(set _.KRC:$dst,(OpNode
(_.VT (bitconvert (_.LdFrag addr:$src1))),
(i32 imm:$src2)))], NoItinerary>;
def rmk : AVX512<opc, MRMSrcMem, (outs _.KRC:$dst),
(ins _.KRCWM:$mask, _.MemOp:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix##mem#
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
"\t{$src2, $src1, $dst {${mask}}|$dst {${mask}}, $src1, $src2}",
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
[(set _.KRC:$dst, (or _.KRCWM:$mask, (OpNode
(_.VT (bitconvert (_.LdFrag addr:$src1))),
(i32 imm:$src2))))], NoItinerary>, EVEX_K;
def rmb : AVX512<opc, MRMSrcMem, (outs _.KRC:$dst),
(ins _.ScalarMemOp:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix##broadcast##"\t{$src2, ${src1}"##
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
_.BroadcastStr##", $dst|$dst, ${src1}"
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
##_.BroadcastStr##", $src2}",
[(set _.KRC:$dst,(OpNode
(_.VT (X86VBroadcast
(_.ScalarLdFrag addr:$src1))),
(i32 imm:$src2)))], NoItinerary>,EVEX_B;
def rmbk : AVX512<opc, MRMSrcMem, (outs _.KRC:$dst),
(ins _.KRCWM:$mask, _.ScalarMemOp:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix##broadcast##"\t{$src2, ${src1}"##
[AVX-512] Remove superfluous spaces from some asm strings. llvm-svn: 257150
2016-01-08 14:09:20 +08:00
_.BroadcastStr##", $dst {${mask}}|$dst {${mask}}, ${src1}"##
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
_.BroadcastStr##", $src2}",
[(set _.KRC:$dst,(or _.KRCWM:$mask, (OpNode
(_.VT (X86VBroadcast
(_.ScalarLdFrag addr:$src1))),
(i32 imm:$src2))))], NoItinerary>,
EVEX_B, EVEX_K;
}
}
multiclass avx512_vector_fpclass_all<string OpcodeStr,
AVX512VLVectorVTInfo _, bits<8> opc, SDNode OpNode, Predicate prd,
string broadcast>{
let Predicates = [prd] in {
defm Z : avx512_vector_fpclass<opc, OpcodeStr, OpNode, _.info512, "{z}",
broadcast>, EVEX_V512;
}
let Predicates = [prd, HasVLX] in {
defm Z128 : avx512_vector_fpclass<opc, OpcodeStr, OpNode, _.info128, "{x}",
broadcast>, EVEX_V128;
defm Z256 : avx512_vector_fpclass<opc, OpcodeStr, OpNode, _.info256, "{y}",
broadcast>, EVEX_V256;
}
}
multiclass avx512_fp_fpclass_all<string OpcodeStr, bits<8> opcVec,
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
bits<8> opcScalar, SDNode VecOpNode, SDNode ScalarOpNode, Predicate prd>{
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm PS : avx512_vector_fpclass_all<OpcodeStr, avx512vl_f32_info, opcVec,
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
VecOpNode, prd, "{l}">, EVEX_CD8<32, CD8VF>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm PD : avx512_vector_fpclass_all<OpcodeStr, avx512vl_f64_info, opcVec,
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
VecOpNode, prd, "{q}">,EVEX_CD8<64, CD8VF> , VEX_W;
defm SS : avx512_scalar_fpclass<opcScalar, OpcodeStr, ScalarOpNode,
f32x_info, prd>, EVEX_CD8<32, CD8VT1>;
defm SD : avx512_scalar_fpclass<opcScalar, OpcodeStr, ScalarOpNode,
f64x_info, prd>, EVEX_CD8<64, CD8VT1>, VEX_W;
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
}
[X86][AVX512DQ] add scalar fpclass Differential Revision: http://reviews.llvm.org/D13769 llvm-svn: 250650
2015-10-18 19:04:38 +08:00
defm VFPCLASS : avx512_fp_fpclass_all<"vfpclass", 0x66, 0x67, X86Vfpclass,
X86Vfpclasss, HasDQI>, AVX512AIi8Base,EVEX;
[X86][AVX512DQ] Add fpclass instruction Differential Revision: http://reviews.llvm.org/D12931 llvm-svn: 248115
2015-09-20 16:46:07 +08:00
AVX-512: Added all forms of FP compare instructions for KNL and SKX. Added intrinsics for the instructions. CC parameter of the intrinsics was changed from i8 to i32 according to the spec. By Igor Breger (igor.breger@intel.com) llvm-svn: 236714
2015-05-07 19:24:42 +08:00
//-----------------------------------------------------------------
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask register copy, including
// - copy between mask registers
// - load/store mask registers
// - copy from GPR to mask register and vice versa
//
multiclass avx512_mask_mov<bits<8> opc_kk, bits<8> opc_km, bits<8> opc_mk,
string OpcodeStr, RegisterClass KRC,
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
ValueType vvt, X86MemOperand x86memop> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def kk : I<opc_kk, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), []>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in
def km : I<opc_km, MRMSrcMem, (outs KRC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
[(set KRC:$dst, (vvt (load addr:$src)))]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayStore = 1 in
def mk : I<opc_mk, MRMDestMem, (outs), (ins x86memop:$dst, KRC:$src),
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[(store KRC:$src, addr:$dst)]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
multiclass avx512_mask_mov_gpr<bits<8> opc_kr, bits<8> opc_rk,
string OpcodeStr,
RegisterClass KRC, RegisterClass GRC> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def kr : I<opc_kr, MRMSrcReg, (outs KRC:$dst), (ins GRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), []>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rk : I<opc_rk, MRMSrcReg, (outs GRC:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), []>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasDQI] in
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
defm KMOVB : avx512_mask_mov<0x90, 0x90, 0x91, "kmovb", VK8, v8i1, i8mem>,
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
avx512_mask_mov_gpr<0x92, 0x93, "kmovb", VK8, GR32>,
VEX, PD;
let Predicates = [HasAVX512] in
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
defm KMOVW : avx512_mask_mov<0x90, 0x90, 0x91, "kmovw", VK16, v16i1, i16mem>,
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
avx512_mask_mov_gpr<0x92, 0x93, "kmovw", VK16, GR32>,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
VEX, PS;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasBWI] in {
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
defm KMOVD : avx512_mask_mov<0x90, 0x90, 0x91, "kmovd", VK32, v32i1,i32mem>,
VEX, PD, VEX_W;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
defm KMOVD : avx512_mask_mov_gpr<0x92, 0x93, "kmovd", VK32, GR32>,
VEX, XD;
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
defm KMOVQ : avx512_mask_mov<0x90, 0x90, 0x91, "kmovq", VK64, v64i1, i64mem>,
VEX, PS, VEX_W;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
defm KMOVQ : avx512_mask_mov_gpr<0x92, 0x93, "kmovq", VK64, GR64>,
VEX, XD, VEX_W;
}
// GR from/to mask register
let Predicates = [HasDQI] in {
def : Pat<(v8i1 (bitconvert (i8 GR8:$src))),
(KMOVBkr (SUBREG_TO_REG (i32 0), GR8:$src, sub_8bit))>;
def : Pat<(i8 (bitconvert (v8i1 VK8:$src))),
(EXTRACT_SUBREG (KMOVBrk VK8:$src), sub_8bit)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(v16i1 (bitconvert (i16 GR16:$src))),
(KMOVWkr (SUBREG_TO_REG (i32 0), GR16:$src, sub_16bit))>;
def : Pat<(i16 (bitconvert (v16i1 VK16:$src))),
(EXTRACT_SUBREG (KMOVWrk VK16:$src), sub_16bit)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
let Predicates = [HasBWI] in {
def : Pat<(v32i1 (bitconvert (i32 GR32:$src))), (KMOVDkr GR32:$src)>;
def : Pat<(i32 (bitconvert (v32i1 VK32:$src))), (KMOVDrk VK32:$src)>;
}
let Predicates = [HasBWI] in {
def : Pat<(v64i1 (bitconvert (i64 GR64:$src))), (KMOVQkr GR64:$src)>;
def : Pat<(i64 (bitconvert (v64i1 VK64:$src))), (KMOVQrk VK64:$src)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
// Load/store kreg
let Predicates = [HasDQI] in {
def : Pat<(store (i8 (bitconvert (v8i1 VK8:$src))), addr:$dst),
(KMOVBmk addr:$dst, VK8:$src)>;
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
def : Pat<(v8i1 (bitconvert (i8 (load addr:$src)))),
(KMOVBkm addr:$src)>;
AVX-512: store <4 x i1> and <2 x i1> values in memory Enabled DAG pattern lowering for SKX with DQI predicate. Differential Revision: http://reviews.llvm.org/D12550 llvm-svn: 246625
2015-09-02 17:20:58 +08:00
def : Pat<(store VK4:$src, addr:$dst),
(KMOVBmk addr:$dst, (COPY_TO_REGCLASS VK4:$src, VK8))>;
def : Pat<(store VK2:$src, addr:$dst),
(KMOVBmk addr:$dst, (COPY_TO_REGCLASS VK2:$src, VK8))>;
AVX512: Add store mask patterns. Differential Revision: http://reviews.llvm.org/D16596 llvm-svn: 258914
2016-01-27 16:43:25 +08:00
def : Pat<(store VK1:$src, addr:$dst),
(KMOVBmk addr:$dst, (COPY_TO_REGCLASS VK1:$src, VK8))>;
AVX-512: Load and Extended Load for i1 vectors Implemented load+{sign|zero}_extend for i1 vectors Fixed failures in i1 vector load. Covered loading of v2i1, v4i1, v8i1, v16i1, v32i1, v64i1 vectors for KNL and SKX. Differential Revision: http://reviews.llvm.org/D18737 llvm-svn: 265259
2016-04-03 16:41:12 +08:00
def : Pat<(v2i1 (load addr:$src)),
(COPY_TO_REGCLASS (KMOVBkm addr:$src), VK2)>;
def : Pat<(v4i1 (load addr:$src)),
(COPY_TO_REGCLASS (KMOVBkm addr:$src), VK4)>;
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
}
let Predicates = [HasAVX512, NoDQI] in {
AVX512: Add store mask patterns. Differential Revision: http://reviews.llvm.org/D16596 llvm-svn: 258914
2016-01-27 16:43:25 +08:00
def : Pat<(store VK1:$src, addr:$dst),
(MOV8mr addr:$dst,
(EXTRACT_SUBREG (KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)),
sub_8bit))>;
def : Pat<(store VK2:$src, addr:$dst),
(MOV8mr addr:$dst,
(EXTRACT_SUBREG (KMOVWrk (COPY_TO_REGCLASS VK2:$src, VK16)),
sub_8bit))>;
def : Pat<(store VK4:$src, addr:$dst),
(MOV8mr addr:$dst,
(EXTRACT_SUBREG (KMOVWrk (COPY_TO_REGCLASS VK4:$src, VK16)),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
sub_8bit))>;
AVX512: Add store mask patterns. Differential Revision: http://reviews.llvm.org/D16596 llvm-svn: 258914
2016-01-27 16:43:25 +08:00
def : Pat<(store VK8:$src, addr:$dst),
(MOV8mr addr:$dst,
(EXTRACT_SUBREG (KMOVWrk (COPY_TO_REGCLASS VK8:$src, VK16)),
sub_8bit))>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
AVX-512: Load and Extended Load for i1 vectors Implemented load+{sign|zero}_extend for i1 vectors Fixed failures in i1 vector load. Covered loading of v2i1, v4i1, v8i1, v16i1, v32i1, v64i1 vectors for KNL and SKX. Differential Revision: http://reviews.llvm.org/D18737 llvm-svn: 265259
2016-04-03 16:41:12 +08:00
def : Pat<(v8i1 (load addr:$src)),
(COPY_TO_REGCLASS (MOVZX16rm8 addr:$src), VK8)>;
def : Pat<(v2i1 (load addr:$src)),
(COPY_TO_REGCLASS (MOVZX16rm8 addr:$src), VK2)>;
def : Pat<(v4i1 (load addr:$src)),
(COPY_TO_REGCLASS (MOVZX16rm8 addr:$src), VK4)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
AVX-512: Load and Extended Load for i1 vectors Implemented load+{sign|zero}_extend for i1 vectors Fixed failures in i1 vector load. Covered loading of v2i1, v4i1, v8i1, v16i1, v32i1, v64i1 vectors for KNL and SKX. Differential Revision: http://reviews.llvm.org/D18737 llvm-svn: 265259
2016-04-03 16:41:12 +08:00
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(store (i16 (bitconvert (v16i1 VK16:$src))), addr:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(KMOVWmk addr:$dst, VK16:$src)>;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
def : Pat<(i1 (load addr:$src)),
AVX-512: Load and Extended Load for i1 vectors Implemented load+{sign|zero}_extend for i1 vectors Fixed failures in i1 vector load. Covered loading of v2i1, v4i1, v8i1, v16i1, v32i1, v64i1 vectors for KNL and SKX. Differential Revision: http://reviews.llvm.org/D18737 llvm-svn: 265259
2016-04-03 16:41:12 +08:00
(COPY_TO_REGCLASS (AND16ri (MOVZX16rm8 addr:$src), (i16 1)), VK1)>;
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
def : Pat<(v16i1 (bitconvert (i16 (load addr:$src)))),
(KMOVWkm addr:$src)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
let Predicates = [HasBWI] in {
def : Pat<(store (i32 (bitconvert (v32i1 VK32:$src))), addr:$dst),
(KMOVDmk addr:$dst, VK32:$src)>;
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
def : Pat<(v32i1 (bitconvert (i32 (load addr:$src)))),
(KMOVDkm addr:$src)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
def : Pat<(store (i64 (bitconvert (v64i1 VK64:$src))), addr:$dst),
(KMOVQmk addr:$dst, VK64:$src)>;
AVX-512: changes in intel_ocl_bi calling conventions - added mask types v8i1 and v16i1 to possible function parameters - enabled passing 512-bit vectors in standard CC - added a test for KNL intel_ocl_bi conventions llvm-svn: 229482
2015-02-17 17:20:12 +08:00
def : Pat<(v64i1 (bitconvert (i64 (load addr:$src)))),
(KMOVQkm addr:$src)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
AVX-512: Added implementation of CONCAT_VECTORS for v8i1 vectors (by Alexey Bader). Added implementation of "truncate" from integer type (i64/i32/i16/i8) to i1. llvm-svn: 197482
2013-12-17 16:33:15 +08:00
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasAVX512] in {
AVX-512: Fixed a bug in emitting compare for MVT:i1 type. Added a test. llvm-svn: 215889
2014-08-18 19:59:06 +08:00
def : Pat<(i1 (trunc (i64 GR64:$src))),
(COPY_TO_REGCLASS (KMOVWkr (AND32ri (EXTRACT_SUBREG $src, sub_32bit),
(i32 1))), VK1)>;
AVX-512: fixed some patterns for MVT::i1 llvm-svn: 197981
2013-12-24 22:24:07 +08:00
def : Pat<(i1 (trunc (i32 GR32:$src))),
AVX-512: Assembly parsing of broadcast semantic in AVX-512; imlemented by Nis Zinovy (zinovy.y.nis@intel.com) Fixed truncate i32 to i1; a test will be provided in the next commit. llvm-svn: 201757
2014-02-20 14:34:39 +08:00
(COPY_TO_REGCLASS (KMOVWkr (AND32ri $src, (i32 1))), VK1)>;
AVX-512: fixed some patterns for MVT::i1 llvm-svn: 197981
2013-12-24 22:24:07 +08:00
def : Pat<(i1 (trunc (i8 GR8:$src))),
AVX-512: Assembly parsing of broadcast semantic in AVX-512; imlemented by Nis Zinovy (zinovy.y.nis@intel.com) Fixed truncate i32 to i1; a test will be provided in the next commit. llvm-svn: 201757
2014-02-20 14:34:39 +08:00
(COPY_TO_REGCLASS
(KMOVWkr (AND32ri (SUBREG_TO_REG (i32 0), GR8:$src, sub_8bit), (i32 1))),
VK1)>;
def : Pat<(i1 (trunc (i16 GR16:$src))),
(COPY_TO_REGCLASS
(KMOVWkr (AND32ri (SUBREG_TO_REG (i32 0), $src, sub_16bit), (i32 1))),
VK1)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
def : Pat<(i32 (zext VK1:$src)),
(AND32ri (KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)), (i32 1))>;
AVX-512: Fixed a bug in extracting subvector from v64i1 By Igor Breger (igor.breger@intel.com) llvm-svn: 238322
2015-05-27 22:09:33 +08:00
def : Pat<(i32 (anyext VK1:$src)),
(KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16))>;
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
AVX-512: fixed some patterns for MVT::i1 llvm-svn: 197981
2013-12-24 22:24:07 +08:00
def : Pat<(i8 (zext VK1:$src)),
(EXTRACT_SUBREG
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
(AND32ri (KMOVWrk
(COPY_TO_REGCLASS VK1:$src, VK16)), (i32 1)), sub_8bit)>;
AVX512: Implemented encoding and intrinsics for vcmpss/sd. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12593 llvm-svn: 248121
2015-09-20 23:15:10 +08:00
def : Pat<(i8 (anyext VK1:$src)),
(EXTRACT_SUBREG
(KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)), sub_8bit)>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def : Pat<(i64 (zext VK1:$src)),
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
(AND64ri8 (SUBREG_TO_REG (i64 0),
(KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)), sub_32bit), (i64 1))>;
AVX-512: implemented zext fron i1 to i16 llvm-svn: 201502
2014-02-17 15:29:33 +08:00
def : Pat<(i16 (zext VK1:$src)),
(EXTRACT_SUBREG
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
(AND32ri (KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)), (i32 1)),
sub_16bit)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
Type legalizer for masked gather and scatter intrinsics. Full type legalizer that works with all vectors length - from 2 to 16, (i32, i64, float, double). This intrinsic, for example void @llvm.masked.scatter.v2f32(<2 x float>%data , <2 x float*>%ptrs , i32 align , <2 x i1>%mask ) requires type widening for data and type promotion for mask. Differential Revision: http://reviews.llvm.org/D13633 llvm-svn: 255629
2015-12-15 16:40:41 +08:00
def : Pat<(v16i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK16)>;
def : Pat<(v8i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK8)>;
def : Pat<(v4i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK4)>;
def : Pat<(v2i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK2)>;
def : Pat<(v32i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK32)>;
def : Pat<(v64i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK64)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
AVX512: Add store mask patterns. Differential Revision: http://reviews.llvm.org/D16596 llvm-svn: 258914
2016-01-27 16:43:25 +08:00
def : Pat<(store (i1 -1), addr:$dst), (MOV8mi addr:$dst, (i8 1))>;
def : Pat<(store (i1 1), addr:$dst), (MOV8mi addr:$dst, (i8 1))>;
def : Pat<(store (i1 0), addr:$dst), (MOV8mi addr:$dst, (i8 0))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// With AVX-512 only, 8-bit mask is promoted to 16-bit mask.
AVX-512: fixed a bug in i1 vectors lowering llvm-svn: 236947
2015-05-10 18:33:32 +08:00
let Predicates = [HasAVX512, NoDQI] in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// GR from/to 8-bit mask without native support
def : Pat<(v8i1 (bitconvert (i8 GR8:$src))),
(COPY_TO_REGCLASS
AVX512 : Change v8i1 bitconvert GR8 pattern, remove unnecessary movzbl instruction. code example , previous implementation. movzbl %dil, %eax kmovw %eax, %k0 new code kmovw %edi, %k0 Differential Revision: http://reviews.llvm.org/D16287 llvm-svn: 258045
2016-01-18 20:02:45 +08:00
(KMOVWkr (SUBREG_TO_REG (i32 0), GR8:$src, sub_8bit)), VK8)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(i8 (bitconvert (v8i1 VK8:$src))),
(EXTRACT_SUBREG
(KMOVWrk (COPY_TO_REGCLASS VK8:$src, VK16)),
sub_8bit)>;
AVX-512: fixed a bug in i1 vectors lowering llvm-svn: 236947
2015-05-10 18:33:32 +08:00
}
AVX-512: fixed algorithm of building vectors of i1 elements fixed extract-insert i1 element, load i1, zextload i1 should be with "and $1, %reg" to prevent loading garbage. added a bunch of new tests. llvm-svn: 237793
2015-05-20 22:32:03 +08:00
AVX-512: fixed a bug in i1 vectors lowering llvm-svn: 236947
2015-05-10 18:33:32 +08:00
let Predicates = [HasAVX512] in {
AVX-512: Fixed extract_vector_elt for v16i1 and v8i1 vectors. llvm-svn: 201066
2014-02-10 15:02:39 +08:00
def : Pat<(i1 (X86Vextract VK16:$src, (iPTR 0))),
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
(COPY_TO_REGCLASS VK16:$src, VK1)>;
AVX-512: Fixed extract_vector_elt for v16i1 and v8i1 vectors. llvm-svn: 201066
2014-02-10 15:02:39 +08:00
def : Pat<(i1 (X86Vextract VK8:$src, (iPTR 0))),
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
(COPY_TO_REGCLASS VK8:$src, VK1)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
let Predicates = [HasBWI] in {
def : Pat<(i1 (X86Vextract VK32:$src, (iPTR 0))),
(COPY_TO_REGCLASS VK32:$src, VK1)>;
def : Pat<(i1 (X86Vextract VK64:$src, (iPTR 0))),
(COPY_TO_REGCLASS VK64:$src, VK1)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
// Mask unary operation
// - KNOT
multiclass avx512_mask_unop<bits<8> opc, string OpcodeStr,
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
RegisterClass KRC, SDPatternOperator OpNode,
Predicate prd> {
let Predicates = [prd] in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : I<opc, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set KRC:$dst, (OpNode KRC:$src))]>;
}
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
multiclass avx512_mask_unop_all<bits<8> opc, string OpcodeStr,
SDPatternOperator OpNode> {
defm B : avx512_mask_unop<opc, !strconcat(OpcodeStr, "b"), VK8, OpNode,
HasDQI>, VEX, PD;
defm W : avx512_mask_unop<opc, !strconcat(OpcodeStr, "w"), VK16, OpNode,
HasAVX512>, VEX, PS;
defm D : avx512_mask_unop<opc, !strconcat(OpcodeStr, "d"), VK32, OpNode,
HasBWI>, VEX, PD, VEX_W;
defm Q : avx512_mask_unop<opc, !strconcat(OpcodeStr, "q"), VK64, OpNode,
HasBWI>, VEX, PS, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
defm KNOT : avx512_mask_unop_all<0x44, "knot", not>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
multiclass avx512_mask_unop_int<string IntName, string InstName> {
let Predicates = [HasAVX512] in
def : Pat<(!cast<Intrinsic>("int_x86_avx512_"##IntName##"_w")
(i16 GR16:$src)),
(COPY_TO_REGCLASS (!cast<Instruction>(InstName##"Wrr")
(v16i1 (COPY_TO_REGCLASS GR16:$src, VK16))), GR16)>;
}
defm : avx512_mask_unop_int<"knot", "KNOT">;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasDQI] in
def : Pat<(xor VK8:$src1, (v8i1 immAllOnesV)), (KNOTBrr VK8:$src1)>;
let Predicates = [HasAVX512] in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(xor VK16:$src1, (v16i1 immAllOnesV)), (KNOTWrr VK16:$src1)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasBWI] in
def : Pat<(xor VK32:$src1, (v32i1 immAllOnesV)), (KNOTDrr VK32:$src1)>;
let Predicates = [HasBWI] in
def : Pat<(xor VK64:$src1, (v64i1 immAllOnesV)), (KNOTQrr VK64:$src1)>;
// KNL does not support KMOVB, 8-bit mask is promoted to 16-bit
AVX-512: Fixed the "test" operation for i1 type Using KORTESTW for comparison i1 value with zero was wrong since the instruction tests 16 bits. KORTESTW may be used with KSHIFTL+KSHIFTR that clean the 15 upper bits. I removed (X86cmp i1, 0) pattern and zero-extend i1 to i8 and then use TESTB. There are some cases where i1 is in the mask register and the upper bits are already zeroed. Then KORTESTW is the better solution, but it is subject for optimization. Meanwhile, I'm fixing the correctness issue. llvm-svn: 228916
2015-02-12 16:40:34 +08:00
let Predicates = [HasAVX512, NoDQI] in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(xor VK8:$src1, (v8i1 immAllOnesV)),
(COPY_TO_REGCLASS (KNOTWrr (COPY_TO_REGCLASS VK8:$src1, VK16)), VK8)>;
def : Pat<(not VK8:$src),
(COPY_TO_REGCLASS
(KNOTWrr (COPY_TO_REGCLASS VK8:$src, VK16)), VK8)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
def : Pat<(xor VK4:$src1, (v4i1 immAllOnesV)),
(COPY_TO_REGCLASS (KNOTWrr (COPY_TO_REGCLASS VK4:$src1, VK16)), VK4)>;
def : Pat<(xor VK2:$src1, (v2i1 immAllOnesV)),
(COPY_TO_REGCLASS (KNOTWrr (COPY_TO_REGCLASS VK2:$src1, VK16)), VK2)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask binary operation
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
// - KAND, KANDN, KOR, KXNOR, KXOR
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_mask_binop<bits<8> opc, string OpcodeStr,
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
RegisterClass KRC, SDPatternOperator OpNode,
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
Predicate prd, bit IsCommutable> {
let Predicates = [prd], isCommutable = IsCommutable in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : I<opc, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src1, KRC:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set KRC:$dst, (OpNode KRC:$src1, KRC:$src2))]>;
}
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
multiclass avx512_mask_binop_all<bits<8> opc, string OpcodeStr,
AVX512: kadd implementation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D11973 llvm-svn: 246432
2015-08-31 19:50:23 +08:00
SDPatternOperator OpNode, bit IsCommutable,
Predicate prdW = HasAVX512> {
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
defm B : avx512_mask_binop<opc, !strconcat(OpcodeStr, "b"), VK8, OpNode,
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
HasDQI, IsCommutable>, VEX_4V, VEX_L, PD;
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
defm W : avx512_mask_binop<opc, !strconcat(OpcodeStr, "w"), VK16, OpNode,
AVX512: kadd implementation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D11973 llvm-svn: 246432
2015-08-31 19:50:23 +08:00
prdW, IsCommutable>, VEX_4V, VEX_L, PS;
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
defm D : avx512_mask_binop<opc, !strconcat(OpcodeStr, "d"), VK32, OpNode,
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
HasBWI, IsCommutable>, VEX_4V, VEX_L, VEX_W, PD;
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
defm Q : avx512_mask_binop<opc, !strconcat(OpcodeStr, "q"), VK64, OpNode,
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
HasBWI, IsCommutable>, VEX_4V, VEX_L, VEX_W, PS;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
def andn : PatFrag<(ops node:$i0, node:$i1), (and (not node:$i0), node:$i1)>;
def xnor : PatFrag<(ops node:$i0, node:$i1), (not (xor node:$i0, node:$i1))>;
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
defm KAND : avx512_mask_binop_all<0x41, "kand", and, 1>;
defm KOR : avx512_mask_binop_all<0x45, "kor", or, 1>;
defm KXNOR : avx512_mask_binop_all<0x46, "kxnor", xnor, 1>;
defm KXOR : avx512_mask_binop_all<0x47, "kxor", xor, 1>;
defm KANDN : avx512_mask_binop_all<0x42, "kandn", andn, 0>;
AVX512: kadd implementation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D11973 llvm-svn: 246432
2015-08-31 19:50:23 +08:00
defm KADD : avx512_mask_binop_all<0x4A, "kadd", add, 1, HasDQI>;
AVX-512: Result type of scalar SETCC is MVT::i1 for AVX-512. llvm-svn: 198008
2013-12-25 18:06:40 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_mask_binop_int<string IntName, string InstName> {
let Predicates = [HasAVX512] in
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
def : Pat<(!cast<Intrinsic>("int_x86_avx512_"##IntName##"_w")
(i16 GR16:$src1), (i16 GR16:$src2)),
(COPY_TO_REGCLASS (!cast<Instruction>(InstName##"Wrr")
(v16i1 (COPY_TO_REGCLASS GR16:$src1, VK16)),
(v16i1 (COPY_TO_REGCLASS GR16:$src2, VK16))), GR16)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm : avx512_mask_binop_int<"kand", "KAND">;
defm : avx512_mask_binop_int<"kandn", "KANDN">;
defm : avx512_mask_binop_int<"kor", "KOR">;
defm : avx512_mask_binop_int<"kxnor", "KXNOR">;
defm : avx512_mask_binop_int<"kxor", "KXOR">;
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_binop_pat<SDPatternOperator OpNode, Instruction Inst> {
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
// With AVX512F, 8-bit mask is promoted to 16-bit mask,
// for the DQI set, this type is legal and KxxxB instruction is used
let Predicates = [NoDQI] in
def : Pat<(OpNode VK8:$src1, VK8:$src2),
(COPY_TO_REGCLASS
(Inst (COPY_TO_REGCLASS VK8:$src1, VK16),
(COPY_TO_REGCLASS VK8:$src2, VK16)), VK8)>;
// All types smaller than 8 bits require conversion anyway
def : Pat<(OpNode VK1:$src1, VK1:$src2),
(COPY_TO_REGCLASS (Inst
(COPY_TO_REGCLASS VK1:$src1, VK16),
(COPY_TO_REGCLASS VK1:$src2, VK16)), VK1)>;
def : Pat<(OpNode VK2:$src1, VK2:$src2),
(COPY_TO_REGCLASS (Inst
(COPY_TO_REGCLASS VK2:$src1, VK16),
(COPY_TO_REGCLASS VK2:$src2, VK16)), VK1)>;
def : Pat<(OpNode VK4:$src1, VK4:$src2),
(COPY_TO_REGCLASS (Inst
(COPY_TO_REGCLASS VK4:$src1, VK16),
(COPY_TO_REGCLASS VK4:$src2, VK16)), VK1)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm : avx512_binop_pat<and, KANDWrr>;
defm : avx512_binop_pat<andn, KANDNWrr>;
defm : avx512_binop_pat<or, KORWrr>;
defm : avx512_binop_pat<xnor, KXNORWrr>;
defm : avx512_binop_pat<xor, KXORWrr>;
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
def : Pat<(xor (xor VK16:$src1, VK16:$src2), (v16i1 immAllOnesV)),
(KXNORWrr VK16:$src1, VK16:$src2)>;
def : Pat<(xor (xor VK8:$src1, VK8:$src2), (v8i1 immAllOnesV)),
AVX-512: Fixed a bug in comparison of i1 vectors. cmp eq should give kxnor instruction cmp neq should give kxor https://llvm.org/bugs/show_bug.cgi?id=23631 llvm-svn: 239460
2015-06-10 14:49:28 +08:00
(KXNORBrr VK8:$src1, VK8:$src2)>, Requires<[HasDQI]>;
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
def : Pat<(xor (xor VK32:$src1, VK32:$src2), (v32i1 immAllOnesV)),
AVX-512: Fixed a bug in comparison of i1 vectors. cmp eq should give kxnor instruction cmp neq should give kxor https://llvm.org/bugs/show_bug.cgi?id=23631 llvm-svn: 239460
2015-06-10 14:49:28 +08:00
(KXNORDrr VK32:$src1, VK32:$src2)>, Requires<[HasBWI]>;
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
def : Pat<(xor (xor VK64:$src1, VK64:$src2), (v64i1 immAllOnesV)),
AVX-512: Fixed a bug in comparison of i1 vectors. cmp eq should give kxnor instruction cmp neq should give kxor https://llvm.org/bugs/show_bug.cgi?id=23631 llvm-svn: 239460
2015-06-10 14:49:28 +08:00
(KXNORQrr VK64:$src1, VK64:$src2)>, Requires<[HasBWI]>;
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
let Predicates = [NoDQI] in
def : Pat<(xor (xor VK8:$src1, VK8:$src2), (v8i1 immAllOnesV)),
(COPY_TO_REGCLASS (KXNORWrr (COPY_TO_REGCLASS VK8:$src1, VK16),
(COPY_TO_REGCLASS VK8:$src2, VK16)), VK8)>;
def : Pat<(xor (xor VK4:$src1, VK4:$src2), (v4i1 immAllOnesV)),
(COPY_TO_REGCLASS (KXNORWrr (COPY_TO_REGCLASS VK4:$src1, VK16),
(COPY_TO_REGCLASS VK4:$src2, VK16)), VK4)>;
def : Pat<(xor (xor VK2:$src1, VK2:$src2), (v2i1 immAllOnesV)),
(COPY_TO_REGCLASS (KXNORWrr (COPY_TO_REGCLASS VK2:$src1, VK16),
(COPY_TO_REGCLASS VK2:$src2, VK16)), VK2)>;
def : Pat<(xor (xor VK1:$src1, VK1:$src2), (i1 1)),
(COPY_TO_REGCLASS (KXNORWrr (COPY_TO_REGCLASS VK1:$src1, VK16),
(COPY_TO_REGCLASS VK1:$src2, VK16)), VK1)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask unpacking
AVX512: kunpck encoding implementation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D12061 llvm-svn: 247010
2015-09-08 21:10:00 +08:00
multiclass avx512_mask_unpck<string Suffix,RegisterClass KRC, ValueType VT,
RegisterClass KRCSrc, Predicate prd> {
let Predicates = [prd] in {
[AVX512] Add hasSideEffects=0 to kunpck instructions since they lack a pattern in their instructions. llvm-svn: 256816
2016-01-05 15:44:08 +08:00
let hasSideEffects = 0 in
AVX512: kunpck encoding implementation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D12061 llvm-svn: 247010
2015-09-08 21:10:00 +08:00
def rr : I<0x4b, MRMSrcReg, (outs KRC:$dst),
(ins KRC:$src1, KRC:$src2),
"kunpck"#Suffix#"\t{$src2, $src1, $dst|$dst, $src1, $src2}", []>,
VEX_4V, VEX_L;
def : Pat<(VT (concat_vectors KRCSrc:$src1, KRCSrc:$src2)),
(!cast<Instruction>(NAME##rr)
(COPY_TO_REGCLASS KRCSrc:$src2, KRC),
(COPY_TO_REGCLASS KRCSrc:$src1, KRC))>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX512: kunpck encoding implementation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D12061 llvm-svn: 247010
2015-09-08 21:10:00 +08:00
defm KUNPCKBW : avx512_mask_unpck<"bw", VK16, v16i1, VK8, HasAVX512>, PD;
defm KUNPCKWD : avx512_mask_unpck<"wd", VK32, v32i1, VK16, HasBWI>, PS;
defm KUNPCKDQ : avx512_mask_unpck<"dq", VK64, v64i1, VK32, HasBWI>, PS, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask bit testing
multiclass avx512_mask_testop<bits<8> opc, string OpcodeStr, RegisterClass KRC,
AVX512: ktest implemantation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D11979 llvm-svn: 246439
2015-08-31 21:30:19 +08:00
SDNode OpNode, Predicate prd> {
let Predicates = [prd], Defs = [EFLAGS] in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : I<opc, MRMSrcReg, (outs), (ins KRC:$src1, KRC:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1|$src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set EFLAGS, (OpNode KRC:$src1, KRC:$src2))]>;
}
AVX512: ktest implemantation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D11979 llvm-svn: 246439
2015-08-31 21:30:19 +08:00
multiclass avx512_mask_testop_w<bits<8> opc, string OpcodeStr, SDNode OpNode,
Predicate prdW = HasAVX512> {
defm B : avx512_mask_testop<opc, OpcodeStr#"b", VK8, OpNode, HasDQI>,
VEX, PD;
defm W : avx512_mask_testop<opc, OpcodeStr#"w", VK16, OpNode, prdW>,
VEX, PS;
defm Q : avx512_mask_testop<opc, OpcodeStr#"q", VK64, OpNode, HasBWI>,
VEX, PS, VEX_W;
defm D : avx512_mask_testop<opc, OpcodeStr#"d", VK32, OpNode, HasBWI>,
VEX, PD, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm KORTEST : avx512_mask_testop_w<0x98, "kortest", X86kortest>;
AVX512: ktest implemantation Added tests for encoding. Differential Revision: http://reviews.llvm.org/D11979 llvm-svn: 246439
2015-08-31 21:30:19 +08:00
defm KTEST : avx512_mask_testop_w<0x99, "ktest", X86ktest, HasDQI>;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask shift
multiclass avx512_mask_shiftop<bits<8> opc, string OpcodeStr, RegisterClass KRC,
SDNode OpNode> {
let Predicates = [HasAVX512] in
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
def ri : Ii8<opc, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src, u8imm:$imm),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$imm, $src, $dst|$dst, $src, $imm}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set KRC:$dst, (OpNode KRC:$src, (i8 imm:$imm)))]>;
}
multiclass avx512_mask_shiftop_w<bits<8> opc1, bits<8> opc2, string OpcodeStr,
SDNode OpNode> {
defm W : avx512_mask_shiftop<opc1, !strconcat(OpcodeStr, "w"), VK16, OpNode>,
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
VEX, TAPD, VEX_W;
let Predicates = [HasDQI] in
defm B : avx512_mask_shiftop<opc1, !strconcat(OpcodeStr, "b"), VK8, OpNode>,
VEX, TAPD;
let Predicates = [HasBWI] in {
defm Q : avx512_mask_shiftop<opc2, !strconcat(OpcodeStr, "q"), VK64, OpNode>,
VEX, TAPD, VEX_W;
defm D : avx512_mask_shiftop<opc2, !strconcat(OpcodeStr, "d"), VK32, OpNode>,
VEX, TAPD;
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
defm KSHIFTL : avx512_mask_shiftop_w<0x32, 0x33, "kshiftl", X86vshli>;
defm KSHIFTR : avx512_mask_shiftop_w<0x30, 0x31, "kshiftr", X86vsrli>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask setting all 0s or 1s
multiclass avx512_mask_setop<RegisterClass KRC, ValueType VT, PatFrag Val> {
let Predicates = [HasAVX512] in
let isReMaterializable = 1, isAsCheapAsAMove = 1, isPseudo = 1 in
def #NAME# : I<0, Pseudo, (outs KRC:$dst), (ins), "",
[(set KRC:$dst, (VT Val))]>;
}
multiclass avx512_mask_setop_w<PatFrag Val> {
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
defm B : avx512_mask_setop<VK8, v8i1, Val>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm W : avx512_mask_setop<VK16, v16i1, Val>;
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
defm D : avx512_mask_setop<VK32, v32i1, Val>;
defm Q : avx512_mask_setop<VK64, v64i1, Val>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm KSET0 : avx512_mask_setop_w<immAllZerosV>;
defm KSET1 : avx512_mask_setop_w<immAllOnesV>;
// With AVX-512 only, 8-bit mask is promoted to 16-bit mask.
let Predicates = [HasAVX512] in {
def : Pat<(v8i1 immAllZerosV), (COPY_TO_REGCLASS (KSET0W), VK8)>;
def : Pat<(v8i1 immAllOnesV), (COPY_TO_REGCLASS (KSET1W), VK8)>;
AVX-512: Extend/Truncate operations for SKX, SETCC for bit-vectors llvm-svn: 235875
2015-04-27 20:57:59 +08:00
def : Pat<(v4i1 immAllOnesV), (COPY_TO_REGCLASS (KSET1W), VK4)>;
def : Pat<(v2i1 immAllOnesV), (COPY_TO_REGCLASS (KSET1W), VK2)>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def : Pat<(i1 0), (COPY_TO_REGCLASS (KSET0W), VK1)>;
AVX-512: fixed a bug in mask operations - (i1 1) pattern Filling k-reg with all-ones value was wrong, (i1 1) should switch on only one bit in mask register llvm-svn: 237536
2015-05-17 15:28:51 +08:00
def : Pat<(i1 1), (COPY_TO_REGCLASS (KSHIFTRWri (KSET1W), (i8 15)), VK1)>;
def : Pat<(i1 -1), (COPY_TO_REGCLASS (KSHIFTRWri (KSET1W), (i8 15)), VK1)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Fixed a bug in extracting subvector from v64i1 By Igor Breger (igor.breger@intel.com) llvm-svn: 238322
2015-05-27 22:09:33 +08:00
AVX512: Remove VSHRI kmask patterns from TD file. It is incorrect to use kshiftw to implement VSHRI v4i1 , bits 15-4 is undef so the upper bits of v4i1 may not be zeroed. v4i1 should be zero_extend to v16i1 ( or any natively supported vector). Differential Revision: http://reviews.llvm.org/D17763 llvm-svn: 262797
2016-03-06 15:46:03 +08:00
// Patterns for kmask insert_subvector/extract_subvector to/from index=0
multiclass operation_subvector_mask_lowering<RegisterClass subRC, ValueType subVT,
RegisterClass RC, ValueType VT> {
def : Pat<(subVT (extract_subvector (VT RC:$src), (iPTR 0))),
(subVT (COPY_TO_REGCLASS RC:$src, subRC))>;
def : Pat<(VT (insert_subvector undef, subRC:$src, (iPTR 0))),
(VT (COPY_TO_REGCLASS subRC:$src, RC))>;
}
AVX-512: Optimized INSERT_SUBVECTOR for i1 vector types ISERT_SUBVECTOR for i1 vectors may be done with shifts, when we insert into the lower part, or into the upper part, on into all-zero vector. CONCAT_VECTORS uses ISERT_SUBVECTOR. Differential Revision: http://reviews.llvm.org/D14815 llvm-svn: 253819
2015-11-22 21:57:38 +08:00
AVX512: Remove VSHRI kmask patterns from TD file. It is incorrect to use kshiftw to implement VSHRI v4i1 , bits 15-4 is undef so the upper bits of v4i1 may not be zeroed. v4i1 should be zero_extend to v16i1 ( or any natively supported vector). Differential Revision: http://reviews.llvm.org/D17763 llvm-svn: 262797
2016-03-06 15:46:03 +08:00
defm : operation_subvector_mask_lowering<VK2, v2i1, VK4, v4i1>;
defm : operation_subvector_mask_lowering<VK2, v2i1, VK8, v8i1>;
defm : operation_subvector_mask_lowering<VK2, v2i1, VK16, v16i1>;
defm : operation_subvector_mask_lowering<VK2, v2i1, VK32, v32i1>;
defm : operation_subvector_mask_lowering<VK2, v2i1, VK64, v64i1>;
AVX-512: Optimized INSERT_SUBVECTOR for i1 vector types ISERT_SUBVECTOR for i1 vectors may be done with shifts, when we insert into the lower part, or into the upper part, on into all-zero vector. CONCAT_VECTORS uses ISERT_SUBVECTOR. Differential Revision: http://reviews.llvm.org/D14815 llvm-svn: 253819
2015-11-22 21:57:38 +08:00
AVX512: Remove VSHRI kmask patterns from TD file. It is incorrect to use kshiftw to implement VSHRI v4i1 , bits 15-4 is undef so the upper bits of v4i1 may not be zeroed. v4i1 should be zero_extend to v16i1 ( or any natively supported vector). Differential Revision: http://reviews.llvm.org/D17763 llvm-svn: 262797
2016-03-06 15:46:03 +08:00
defm : operation_subvector_mask_lowering<VK4, v4i1, VK8, v8i1>;
defm : operation_subvector_mask_lowering<VK4, v4i1, VK16, v16i1>;
defm : operation_subvector_mask_lowering<VK4, v4i1, VK32, v32i1>;
defm : operation_subvector_mask_lowering<VK4, v4i1, VK64, v64i1>;
AVX-512: Optimized INSERT_SUBVECTOR for i1 vector types ISERT_SUBVECTOR for i1 vectors may be done with shifts, when we insert into the lower part, or into the upper part, on into all-zero vector. CONCAT_VECTORS uses ISERT_SUBVECTOR. Differential Revision: http://reviews.llvm.org/D14815 llvm-svn: 253819
2015-11-22 21:57:38 +08:00
AVX512: Remove VSHRI kmask patterns from TD file. It is incorrect to use kshiftw to implement VSHRI v4i1 , bits 15-4 is undef so the upper bits of v4i1 may not be zeroed. v4i1 should be zero_extend to v16i1 ( or any natively supported vector). Differential Revision: http://reviews.llvm.org/D17763 llvm-svn: 262797
2016-03-06 15:46:03 +08:00
defm : operation_subvector_mask_lowering<VK8, v8i1, VK16, v16i1>;
defm : operation_subvector_mask_lowering<VK8, v8i1, VK32, v32i1>;
defm : operation_subvector_mask_lowering<VK8, v8i1, VK64, v64i1>;
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
AVX512: Remove VSHRI kmask patterns from TD file. It is incorrect to use kshiftw to implement VSHRI v4i1 , bits 15-4 is undef so the upper bits of v4i1 may not be zeroed. v4i1 should be zero_extend to v16i1 ( or any natively supported vector). Differential Revision: http://reviews.llvm.org/D17763 llvm-svn: 262797
2016-03-06 15:46:03 +08:00
defm : operation_subvector_mask_lowering<VK16, v16i1, VK32, v32i1>;
defm : operation_subvector_mask_lowering<VK16, v16i1, VK64, v64i1>;
AVX-512: Optimized INSERT_SUBVECTOR for i1 vector types ISERT_SUBVECTOR for i1 vectors may be done with shifts, when we insert into the lower part, or into the upper part, on into all-zero vector. CONCAT_VECTORS uses ISERT_SUBVECTOR. Differential Revision: http://reviews.llvm.org/D14815 llvm-svn: 253819
2015-11-22 21:57:38 +08:00
AVX512: Remove VSHRI kmask patterns from TD file. It is incorrect to use kshiftw to implement VSHRI v4i1 , bits 15-4 is undef so the upper bits of v4i1 may not be zeroed. v4i1 should be zero_extend to v16i1 ( or any natively supported vector). Differential Revision: http://reviews.llvm.org/D17763 llvm-svn: 262797
2016-03-06 15:46:03 +08:00
defm : operation_subvector_mask_lowering<VK32, v32i1, VK64, v64i1>;
AVX-512: Optimized INSERT_SUBVECTOR for i1 vector types ISERT_SUBVECTOR for i1 vectors may be done with shifts, when we insert into the lower part, or into the upper part, on into all-zero vector. CONCAT_VECTORS uses ISERT_SUBVECTOR. Differential Revision: http://reviews.llvm.org/D14815 llvm-svn: 253819
2015-11-22 21:57:38 +08:00
AVX512: Add extract_subvector patterns v8i1->v4i1 , v4i1->v2i1. Differential Revision: http://reviews.llvm.org/D17953 llvm-svn: 262929
2016-03-08 23:21:25 +08:00
def : Pat<(v2i1 (extract_subvector (v4i1 VK4:$src), (iPTR 2))),
(v2i1 (COPY_TO_REGCLASS
(KSHIFTRWri (COPY_TO_REGCLASS VK4:$src, VK16), (i8 2)),
VK2))>;
def : Pat<(v4i1 (extract_subvector (v8i1 VK8:$src), (iPTR 4))),
(v4i1 (COPY_TO_REGCLASS
(KSHIFTRWri (COPY_TO_REGCLASS VK8:$src, VK16), (i8 4)),
VK4))>;
AVX512: Remove VSHRI kmask patterns from TD file. It is incorrect to use kshiftw to implement VSHRI v4i1 , bits 15-4 is undef so the upper bits of v4i1 may not be zeroed. v4i1 should be zero_extend to v16i1 ( or any natively supported vector). Differential Revision: http://reviews.llvm.org/D17763 llvm-svn: 262797
2016-03-06 15:46:03 +08:00
def : Pat<(v8i1 (extract_subvector (v16i1 VK16:$src), (iPTR 8))),
(v8i1 (COPY_TO_REGCLASS (KSHIFTRWri VK16:$src, (i8 8)), VK8))>;
def : Pat<(v16i1 (extract_subvector (v32i1 VK32:$src), (iPTR 16))),
(v16i1 (COPY_TO_REGCLASS (KSHIFTRDri VK32:$src, (i8 16)), VK16))>;
def : Pat<(v32i1 (extract_subvector (v64i1 VK64:$src), (iPTR 32))),
(v32i1 (COPY_TO_REGCLASS (KSHIFTRQri VK64:$src, (i8 32)), VK32))>;
[AVX512] Added intrinsics for 128- and 256-bit versions of VCMPEQ{BWDQ} Fixed lowering of this intrinsics in case when mask is v2i1 and v4i1. Now cmp intrinsics lower in the following way: (i8 (int_x86_avx512_mask_pcmpeq_q_128 (v2i64 %a), (v2i64 %b), (i8 %mask))) -> (i8 (bitcast (v8i1 (insert_subvector undef, (v2i1 (and (PCMPEQM %a, %b), (extract_subvector (v8i1 (bitcast %mask)), 0))), 0)))) llvm-svn: 218669
2014-09-30 19:41:54 +08:00
AVX-512: Fixed extract_vector_elt for v8i1 vector llvm-svn: 202624
2014-03-02 17:19:44 +08:00
def : Pat<(v8i1 (X86vshli VK8:$src, (i8 imm:$imm))),
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
(v8i1 (COPY_TO_REGCLASS
(KSHIFTLWri (COPY_TO_REGCLASS VK8:$src, VK16),
(I8Imm $imm)), VK8))>, Requires<[HasAVX512, NoDQI]>;
AVX-512: Fixed extract_vector_elt for v8i1 vector llvm-svn: 202624
2014-03-02 17:19:44 +08:00
AVX-512, SKX: Enabled masked_load/store operations for this target. Added lowering for ISD::CONCAT_VECTORS and ISD::INSERT_SUBVECTOR for i1 vectors, it is needed to pass all masked_memop.ll tests for SKX. llvm-svn: 231371
2015-03-05 23:11:35 +08:00
def : Pat<(v4i1 (X86vshli VK4:$src, (i8 imm:$imm))),
(v4i1 (COPY_TO_REGCLASS
(KSHIFTLWri (COPY_TO_REGCLASS VK4:$src, VK16),
(I8Imm $imm)), VK4))>, Requires<[HasAVX512]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Aligned and unaligned load and store
//
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
multiclass avx512_load<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
PatFrag ld_frag, PatFrag mload,
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
bit IsReMaterializable = 1,
SDPatternOperator SelectOprr = vselect> {
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
let hasSideEffects = 0 in {
def rr : AVX512PI<opc, MRMSrcReg, (outs _.RC:$dst), (ins _.RC:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), [],
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
_.ExeDomain>, EVEX;
def rrkz : AVX512PI<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr, "\t{$src, ${dst} {${mask}} {z}|",
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
"${dst} {${mask}} {z}, $src}"),
AVX512: Masked move intrinsic implementation. Implemented intrinsic for the follow instructions (reg move) : VMOVDQU8/16, VMOVDQA32/64, VMOVAPS/PD. Differential Revision: http://reviews.llvm.org/D16316 llvm-svn: 258398
2016-01-21 22:18:11 +08:00
[(set _.RC:$dst, (_.VT (vselect _.KRCWM:$mask,
(_.VT _.RC:$src),
_.ImmAllZerosV)))], _.ExeDomain>,
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
EVEX, EVEX_KZ;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
let canFoldAsLoad = 1, isReMaterializable = IsReMaterializable,
SchedRW = [WriteLoad] in
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
def rm : AVX512PI<opc, MRMSrcMem, (outs _.RC:$dst), (ins _.MemOp:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
[(set _.RC:$dst, (_.VT (bitconvert (ld_frag addr:$src))))],
_.ExeDomain>, EVEX;
let Constraints = "$src0 = $dst" in {
def rrk : AVX512PI<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.RC:$src0, _.KRCWM:$mask, _.RC:$src1),
!strconcat(OpcodeStr, "\t{$src1, ${dst} {${mask}}|",
"${dst} {${mask}}, $src1}"),
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
[(set _.RC:$dst, (_.VT (SelectOprr _.KRCWM:$mask,
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
(_.VT _.RC:$src1),
(_.VT _.RC:$src0))))], _.ExeDomain>,
EVEX, EVEX_K;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
let mayLoad = 1, SchedRW = [WriteLoad] in
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
def rmk : AVX512PI<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src0, _.KRCWM:$mask, _.MemOp:$src1),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr, "\t{$src1, ${dst} {${mask}}|",
"${dst} {${mask}}, $src1}"),
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
[(set _.RC:$dst, (_.VT
(vselect _.KRCWM:$mask,
(_.VT (bitconvert (ld_frag addr:$src1))),
(_.VT _.RC:$src0))))], _.ExeDomain>, EVEX, EVEX_K;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
}
let mayLoad = 1, SchedRW = [WriteLoad] in
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
def rmkz : AVX512PI<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.MemOp:$src),
OpcodeStr #"\t{$src, ${dst} {${mask}} {z}|"#
"${dst} {${mask}} {z}, $src}",
[(set _.RC:$dst, (_.VT (vselect _.KRCWM:$mask,
(_.VT (bitconvert (ld_frag addr:$src))), _.ImmAllZerosV)))],
_.ExeDomain>, EVEX, EVEX_KZ;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
}
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
def : Pat<(_.VT (mload addr:$ptr, _.KRCWM:$mask, undef)),
(!cast<Instruction>(NAME#_.ZSuffix##rmkz) _.KRCWM:$mask, addr:$ptr)>;
def : Pat<(_.VT (mload addr:$ptr, _.KRCWM:$mask, _.ImmAllZerosV)),
(!cast<Instruction>(NAME#_.ZSuffix##rmkz) _.KRCWM:$mask, addr:$ptr)>;
def : Pat<(_.VT (mload addr:$ptr, _.KRCWM:$mask, (_.VT _.RC:$src0))),
(!cast<Instruction>(NAME#_.ZSuffix##rmk) _.RC:$src0,
_.KRCWM:$mask, addr:$ptr)>;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
}
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
multiclass avx512_alignedload_vl<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _,
Predicate prd,
bit IsReMaterializable = 1> {
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
let Predicates = [prd] in
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
defm Z : avx512_load<opc, OpcodeStr, _.info512, _.info512.AlignedLdFrag,
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
masked_load_aligned512, IsReMaterializable>, EVEX_V512;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
let Predicates = [prd, HasVLX] in {
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
defm Z256 : avx512_load<opc, OpcodeStr, _.info256, _.info256.AlignedLdFrag,
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
masked_load_aligned256, IsReMaterializable>, EVEX_V256;
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
defm Z128 : avx512_load<opc, OpcodeStr, _.info128, _.info128.AlignedLdFrag,
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
masked_load_aligned128, IsReMaterializable>, EVEX_V128;
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
multiclass avx512_load_vl<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _,
Predicate prd,
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
bit IsReMaterializable = 1,
SDPatternOperator SelectOprr = vselect> {
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
let Predicates = [prd] in
defm Z : avx512_load<opc, OpcodeStr, _.info512, _.info512.LdFrag,
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
masked_load_unaligned, IsReMaterializable,
SelectOprr>, EVEX_V512;
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_load<opc, OpcodeStr, _.info256, _.info256.LdFrag,
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
masked_load_unaligned, IsReMaterializable,
SelectOprr>, EVEX_V256;
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
defm Z128 : avx512_load<opc, OpcodeStr, _.info128, _.info128.LdFrag,
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
masked_load_unaligned, IsReMaterializable,
SelectOprr>, EVEX_V128;
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
}
}
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
multiclass avx512_store<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
PatFrag st_frag, PatFrag mstore> {
AVX512: Implemented encoding for the follow instructions. vmovapd.s, vmovaps.s, vmovdqa32.s, vmovdqa64.s, vmovdqu16.s, vmovdqu32.s, vmovdqu64.s, vmovdqu8.s, vmovupd.s, vmovups.s Differential Revision: http://reviews.llvm.org/D14768 llvm-svn: 253546
2015-11-19 15:43:43 +08:00
[AVX512] Add hasSideEffects/mayLoad/mayStore flags to some instructions. llvm-svn: 268174
2016-05-01 09:03:56 +08:00
let hasSideEffects = 0 in {
AVX512: Implemented encoding for the follow instructions. vmovapd.s, vmovaps.s, vmovdqa32.s, vmovdqa64.s, vmovdqu16.s, vmovdqu32.s, vmovdqu64.s, vmovdqu8.s, vmovupd.s, vmovups.s Differential Revision: http://reviews.llvm.org/D14768 llvm-svn: 253546
2015-11-19 15:43:43 +08:00
def rr_REV : AVX512PI<opc, MRMDestReg, (outs _.RC:$dst), (ins _.RC:$src),
OpcodeStr # ".s\t{$src, $dst|$dst, $src}",
[], _.ExeDomain>, EVEX;
def rrk_REV : AVX512PI<opc, MRMDestReg, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src),
OpcodeStr # ".s\t{$src, ${dst} {${mask}}|"#
"${dst} {${mask}}, $src}",
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
[], _.ExeDomain>, EVEX, EVEX_K;
AVX512: Implemented encoding for the follow instructions. vmovapd.s, vmovaps.s, vmovdqa32.s, vmovdqa64.s, vmovdqu16.s, vmovdqu32.s, vmovdqu64.s, vmovdqu8.s, vmovupd.s, vmovups.s Differential Revision: http://reviews.llvm.org/D14768 llvm-svn: 253546
2015-11-19 15:43:43 +08:00
def rrkz_REV : AVX512PI<opc, MRMDestReg, (outs _.RC:$dst),
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
(ins _.KRCWM:$mask, _.RC:$src),
AVX512: Implemented encoding for the follow instructions. vmovapd.s, vmovaps.s, vmovdqa32.s, vmovdqa64.s, vmovdqu16.s, vmovdqu32.s, vmovdqu64.s, vmovdqu8.s, vmovupd.s, vmovups.s Differential Revision: http://reviews.llvm.org/D14768 llvm-svn: 253546
2015-11-19 15:43:43 +08:00
OpcodeStr # ".s\t{$src, ${dst} {${mask}} {z}|" #
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
"${dst} {${mask}} {z}, $src}",
[], _.ExeDomain>, EVEX, EVEX_KZ;
[AVX512] Add hasSideEffects/mayLoad/mayStore flags to some instructions. llvm-svn: 268174
2016-05-01 09:03:56 +08:00
}
AVX512: Implemented encoding for the follow instructions. vmovapd.s, vmovaps.s, vmovdqa32.s, vmovdqa64.s, vmovdqu16.s, vmovdqu32.s, vmovdqu64.s, vmovdqu8.s, vmovupd.s, vmovups.s Differential Revision: http://reviews.llvm.org/D14768 llvm-svn: 253546
2015-11-19 15:43:43 +08:00
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
def mr : AVX512PI<opc, MRMDestMem, (outs), (ins _.MemOp:$dst, _.RC:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
[(st_frag (_.VT _.RC:$src), addr:$dst)], _.ExeDomain>, EVEX;
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
def mrk : AVX512PI<opc, MRMDestMem, (outs),
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
(ins _.MemOp:$dst, _.KRCWM:$mask, _.RC:$src),
OpcodeStr # "\t{$src, ${dst} {${mask}}|${dst} {${mask}}, $src}",
[], _.ExeDomain>, EVEX, EVEX_K;
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
def: Pat<(mstore addr:$ptr, _.KRCWM:$mask, (_.VT _.RC:$src)),
(!cast<Instruction>(NAME#_.ZSuffix##mrk) addr:$ptr,
_.KRCWM:$mask, _.RC:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
multiclass avx512_store_vl< bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _, Predicate prd> {
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
let Predicates = [prd] in
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
defm Z : avx512_store<opc, OpcodeStr, _.info512, store,
masked_store_unaligned>, EVEX_V512;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
let Predicates = [prd, HasVLX] in {
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
defm Z256 : avx512_store<opc, OpcodeStr, _.info256, store,
masked_store_unaligned>, EVEX_V256;
defm Z128 : avx512_store<opc, OpcodeStr, _.info128, store,
masked_store_unaligned>, EVEX_V128;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
}
}
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
multiclass avx512_alignedstore_vl<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo _, Predicate prd> {
let Predicates = [prd] in
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
defm Z : avx512_store<opc, OpcodeStr, _.info512, alignedstore512,
masked_store_aligned512>, EVEX_V512;
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
let Predicates = [prd, HasVLX] in {
AVX-512: Moved patterns for masked load/store under avx_store, avx_load classes. No functional changes. llvm-svn: 231069
2015-03-03 23:03:35 +08:00
defm Z256 : avx512_store<opc, OpcodeStr, _.info256, alignedstore256,
masked_store_aligned256>, EVEX_V256;
defm Z128 : avx512_store<opc, OpcodeStr, _.info128, alignedstore,
masked_store_aligned128>, EVEX_V128;
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
}
}
defm VMOVAPS : avx512_alignedload_vl<0x28, "vmovaps", avx512vl_f32_info,
HasAVX512>,
avx512_alignedstore_vl<0x29, "vmovaps", avx512vl_f32_info,
HasAVX512>, PS, EVEX_CD8<32, CD8VF>;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
defm VMOVAPD : avx512_alignedload_vl<0x28, "vmovapd", avx512vl_f64_info,
HasAVX512>,
avx512_alignedstore_vl<0x29, "vmovapd", avx512vl_f64_info,
HasAVX512>, PD, VEX_W, EVEX_CD8<64, CD8VF>;
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
defm VMOVUPS : avx512_load_vl<0x10, "vmovups", avx512vl_f32_info, HasAVX512,
1, null_frag>,
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
avx512_store_vl<0x11, "vmovups", avx512vl_f32_info, HasAVX512>,
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
PS, EVEX_CD8<32, CD8VF>;
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
defm VMOVUPD : avx512_load_vl<0x10, "vmovupd", avx512vl_f64_info, HasAVX512, 0,
null_frag>,
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
avx512_store_vl<0x11, "vmovupd", avx512vl_f64_info, HasAVX512>,
PD, VEX_W, EVEX_CD8<64, CD8VF>;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
defm VMOVDQA32 : avx512_alignedload_vl<0x6F, "vmovdqa32", avx512vl_i32_info,
HasAVX512>,
avx512_alignedstore_vl<0x7F, "vmovdqa32", avx512vl_i32_info,
HasAVX512>, PD, EVEX_CD8<32, CD8VF>;
defm VMOVDQA64 : avx512_alignedload_vl<0x6F, "vmovdqa64", avx512vl_i64_info,
HasAVX512>,
avx512_alignedstore_vl<0x7F, "vmovdqa64", avx512vl_i64_info,
HasAVX512>, PD, VEX_W, EVEX_CD8<64, CD8VF>;
defm VMOVDQU8 : avx512_load_vl<0x6F, "vmovdqu8", avx512vl_i8_info, HasBWI>,
avx512_store_vl<0x7F, "vmovdqu8", avx512vl_i8_info,
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
HasBWI>, XD, EVEX_CD8<8, CD8VF>;
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
defm VMOVDQU16 : avx512_load_vl<0x6F, "vmovdqu16", avx512vl_i16_info, HasBWI>,
avx512_store_vl<0x7F, "vmovdqu16", avx512vl_i16_info,
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
HasBWI>, XD, VEX_W, EVEX_CD8<16, CD8VF>;
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
defm VMOVDQU32 : avx512_load_vl<0x6F, "vmovdqu32", avx512vl_i32_info, HasAVX512,
1, null_frag>,
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
avx512_store_vl<0x7F, "vmovdqu32", avx512vl_i32_info,
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
HasAVX512>, XS, EVEX_CD8<32, CD8VF>;
[X86] Null out some redundant patterns for masked vector register to register moves. These can be accomplished with both aligned and unaligned opcodes. Currently aligned is what is being used so remove the redundant patterns for the unaligned versions. But don't do this for the byte and word vector types since they don't have aligned versions. llvm-svn: 261985
2016-02-26 14:50:29 +08:00
defm VMOVDQU64 : avx512_load_vl<0x6F, "vmovdqu64", avx512vl_i64_info, HasAVX512,
1, null_frag>,
AVX-512: Simplified MOV patterns, no functional changes. llvm-svn: 230954
2015-03-02 20:46:21 +08:00
avx512_store_vl<0x7F, "vmovdqu64", avx512vl_i64_info,
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
HasAVX512>, XS, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: aligned / unaligned load and store for 512-bit integer vectors. llvm-svn: 193156
2013-10-22 17:19:28 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let AddedComplexity = 20 in {
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
def : Pat<(v8i64 (vselect VK8WM:$mask, (bc_v8i64 (v16i32 immAllZerosV)),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(v8i64 VR512:$src))),
AVX512: Masked move intrinsic implementation. Implemented intrinsic for the follow instructions (reg move) : VMOVDQU8/16, VMOVDQA32/64, VMOVAPS/PD. Differential Revision: http://reviews.llvm.org/D16316 llvm-svn: 258398
2016-01-21 22:18:11 +08:00
(VMOVDQA64Zrrkz (COPY_TO_REGCLASS (KNOTWrr (COPY_TO_REGCLASS VK8:$mask, VK16)),
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
VK8), VR512:$src)>;
def : Pat<(v16i32 (vselect VK16WM:$mask, (v16i32 immAllZerosV),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(v16i32 VR512:$src))),
AVX512: Masked move intrinsic implementation. Implemented intrinsic for the follow instructions (reg move) : VMOVDQU8/16, VMOVDQA32/64, VMOVAPS/PD. Differential Revision: http://reviews.llvm.org/D16316 llvm-svn: 258398
2016-01-21 22:18:11 +08:00
(VMOVDQA32Zrrkz (KNOTWrr VK16WM:$mask), VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
Masked Load / Store Intrinsics - the CodeGen part. I'm recommiting the codegen part of the patch. The vectorizer part will be send to review again. Masked Vector Load and Store Intrinsics. Introduced new target-independent intrinsics in order to support masked vector loads and stores. The loop vectorizer optimizes loops containing conditional memory accesses by generating these intrinsics for existing targets AVX2 and AVX-512. The vectorizer asks the target about availability of masked vector loads and stores. Added SDNodes for masked operations and lowering patterns for X86 code generator. Examples: <16 x i32> @llvm.masked.load.v16i32(i8* %addr, <16 x i32> %passthru, i32 4 /* align */, <16 x i1> %mask) declare void @llvm.masked.store.v8f64(i8* %addr, <8 x double> %value, i32 4, <8 x i1> %mask) Scalarizer for other targets (not AVX2/AVX-512) will be done in a separate patch. http://reviews.llvm.org/D6191 llvm-svn: 223348
2014-12-04 17:40:44 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Move Int Doubleword to Packed Double Int
//
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVDI2PDIZrr : AVX512BI<0x6E, MRMSrcReg, (outs VR128X:$dst), (ins GR32:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst,
(v4i32 (scalar_to_vector GR32:$src)))], IIC_SSE_MOVDQ>,
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
EVEX;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVDI2PDIZrm : AVX512BI<0x6E, MRMSrcMem, (outs VR128X:$dst), (ins i32mem:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst,
(v4i32 (scalar_to_vector (loadi32 addr:$src))))],
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
IIC_SSE_MOVDQ>, EVEX, EVEX_CD8<32, CD8VT1>;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOV64toPQIZrr : AVX512BI<0x6E, MRMSrcReg, (outs VR128X:$dst), (ins GR64:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst,
(v2i64 (scalar_to_vector GR64:$src)))],
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
IIC_SSE_MOVDQ>, EVEX, VEX_W;
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler. I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there. llvm-svn: 256483
2015-12-28 14:11:42 +08:00
let isCodeGenOnly = 1, ForceDisassemble = 1, hasSideEffects = 0, mayLoad = 1 in
def VMOV64toPQIZrm : AVX512BI<0x6E, MRMSrcMem, (outs VR128X:$dst),
(ins i64mem:$src),
"vmovq\t{$src, $dst|$dst, $src}", []>,
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
EVEX, VEX_W, EVEX_CD8<64, CD8VT1>;
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
let isCodeGenOnly = 1 in {
[AVX512] Fix some places that used FR64 instead of FR64X. llvm-svn: 256484
2015-12-28 14:11:45 +08:00
def VMOV64toSDZrr : AVX512BI<0x6E, MRMSrcReg, (outs FR64X:$dst), (ins GR64:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
[AVX512] Fix some places that used FR64 instead of FR64X. llvm-svn: 256484
2015-12-28 14:11:45 +08:00
[(set FR64X:$dst, (bitconvert GR64:$src))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_MOVDQ>, EVEX, VEX_W, Sched<[WriteMove]>;
[AVX512] Fix some places that used FR64 instead of FR64X. llvm-svn: 256484
2015-12-28 14:11:45 +08:00
def VMOVSDto64Zrr : AVX512BI<0x7E, MRMDestReg, (outs GR64:$dst), (ins FR64X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
[AVX512] Fix some places that used FR64 instead of FR64X. llvm-svn: 256484
2015-12-28 14:11:45 +08:00
[(set GR64:$dst, (bitconvert FR64X:$src))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_MOVDQ>, EVEX, VEX_W, Sched<[WriteMove]>;
[AVX512] Fix some places that used FR64 instead of FR64X. llvm-svn: 256484
2015-12-28 14:11:45 +08:00
def VMOVSDto64Zmr : AVX512BI<0x7E, MRMDestMem, (outs), (ins i64mem:$dst, FR64X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
[AVX512] Fix some places that used FR64 instead of FR64X. llvm-svn: 256484
2015-12-28 14:11:45 +08:00
[(store (i64 (bitconvert FR64X:$src)), addr:$dst)],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_MOVDQ>, EVEX, VEX_W, Sched<[WriteStore]>,
EVEX_CD8<64, CD8VT1>;
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler. I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there. llvm-svn: 256483
2015-12-28 14:11:42 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Move Int Doubleword to Single Scalar
//
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
let isCodeGenOnly = 1 in {
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVDI2SSZrr : AVX512BI<0x6E, MRMSrcReg, (outs FR32X:$dst), (ins GR32:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set FR32X:$dst, (bitconvert GR32:$src))],
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
IIC_SSE_MOVDQ>, EVEX;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVDI2SSZrm : AVX512BI<0x6E, MRMSrcMem, (outs FR32X:$dst), (ins i32mem:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set FR32X:$dst, (bitconvert (loadi32 addr:$src)))],
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
IIC_SSE_MOVDQ>, EVEX, EVEX_CD8<32, CD8VT1>;
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
// Move doubleword from xmm register to r/m32
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVPDI2DIZrr : AVX512BI<0x7E, MRMDestReg, (outs GR32:$dst), (ins VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
Start replacing vector_extract/vector_insert with extractelt/insertelt These are redundant pairs of nodes defined for INSERT_VECTOR_ELEMENT/EXTRACT_VECTOR_ELEMENT. insertelement/extractelement are slightly closer to the corresponding C++ node name, and has stricter type checking so prefer it. Update targets to only use these nodes where it is trivial to do so. AArch64, ARM, and Mips all have various type errors on simple replacement, so they will need work to fix. Example from AArch64: def : Pat<(sext_inreg (vector_extract (v16i8 V128:$Rn), VectorIndexB:$idx), i8), (i32 (SMOVvi8to32 V128:$Rn, VectorIndexB:$idx))>; Which is trying to do sext_inreg i8, i8. llvm-svn: 255359
2015-12-12 03:20:16 +08:00
[(set GR32:$dst, (extractelt (v4i32 VR128X:$src),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(iPTR 0)))], IIC_SSE_MOVD_ToGP>,
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
EVEX;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVPDI2DIZmr : AVX512BI<0x7E, MRMDestMem, (outs),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins i32mem:$dst, VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
Start replacing vector_extract/vector_insert with extractelt/insertelt These are redundant pairs of nodes defined for INSERT_VECTOR_ELEMENT/EXTRACT_VECTOR_ELEMENT. insertelement/extractelement are slightly closer to the corresponding C++ node name, and has stricter type checking so prefer it. Update targets to only use these nodes where it is trivial to do so. AArch64, ARM, and Mips all have various type errors on simple replacement, so they will need work to fix. Example from AArch64: def : Pat<(sext_inreg (vector_extract (v16i8 V128:$Rn), VectorIndexB:$idx), i8), (i32 (SMOVvi8to32 V128:$Rn, VectorIndexB:$idx))>; Which is trying to do sext_inreg i8, i8. llvm-svn: 255359
2015-12-12 03:20:16 +08:00
[(store (i32 (extractelt (v4i32 VR128X:$src),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(iPTR 0))), addr:$dst)], IIC_SSE_MOVDQ>,
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
EVEX, EVEX_CD8<32, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
// Move quadword from xmm1 register to r/m64
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//
def VMOVPQIto64Zrr : I<0x7E, MRMDestReg, (outs GR64:$dst), (ins VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set GR64:$dst, (extractelt (v2i64 VR128X:$src),
(iPTR 0)))],
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
IIC_SSE_MOVD_ToGP>, PD, EVEX, VEX_W,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Requires<[HasAVX512, In64BitMode]>;
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler. I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there. llvm-svn: 256483
2015-12-28 14:11:42 +08:00
let isCodeGenOnly = 1, ForceDisassemble = 1, hasSideEffects = 0, mayStore = 1 in
def VMOVPQIto64Zmr : I<0x7E, MRMDestMem, (outs), (ins i64mem:$dst, VR128X:$src),
"vmovq\t{$src, $dst|$dst, $src}",
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
[], IIC_SSE_MOVD_ToGP>, PD, EVEX, VEX_W,
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler. I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there. llvm-svn: 256483
2015-12-28 14:11:42 +08:00
Requires<[HasAVX512, In64BitMode]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler. I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there. llvm-svn: 256483
2015-12-28 14:11:42 +08:00
def VMOVPQI2QIZmr : I<0xD6, MRMDestMem, (outs),
(ins i64mem:$dst, VR128X:$src),
"vmovq\t{$src, $dst|$dst, $src}",
[(store (extractelt (v2i64 VR128X:$src), (iPTR 0)),
addr:$dst)], IIC_SSE_MOVDQ>,
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
EVEX, PD, VEX_W, EVEX_CD8<64, CD8VT1>,
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler. I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there. llvm-svn: 256483
2015-12-28 14:11:42 +08:00
Sched<[WriteStore]>, Requires<[HasAVX512, In64BitMode]>;
let hasSideEffects = 0 in
def VMOVPQI2QIZrr : AVX512BI<0xD6, MRMDestReg, (outs VR128X:$dst),
(ins VR128X:$src),
"vmovq.s\t{$src, $dst|$dst, $src}",[]>,
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
EVEX, VEX_W;
AVX512:Implemented encoding for the vmovq.s instruction. Differential Revision: http://reviews.llvm.org/D14810 llvm-svn: 254248
2015-11-29 15:41:26 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Move Scalar Single to Double Int
//
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
let isCodeGenOnly = 1 in {
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVSS2DIZrr : AVX512BI<0x7E, MRMDestReg, (outs GR32:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins FR32X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set GR32:$dst, (bitconvert FR32X:$src))],
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
IIC_SSE_MOVD_ToGP>, EVEX;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVSS2DIZmr : AVX512BI<0x7E, MRMDestMem, (outs),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins i32mem:$dst, FR32X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(store (i32 (bitconvert FR32X:$src)), addr:$dst)],
[AVX512] Remove VEX_LIG from vmovd/vmovq instructions. From what I can tell from the Intel docs these instructions require the L-bit to be 0. llvm-svn: 256486
2015-12-28 14:32:47 +08:00
IIC_SSE_MOVDQ>, EVEX, EVEX_CD8<32, CD8VT1>;
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Move Quadword Int to Packed Quadword Int
//
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler. I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there. llvm-svn: 256483
2015-12-28 14:11:42 +08:00
def VMOVQI2PQIZrm : AVX512XSI<0x7E, MRMSrcMem, (outs VR128X:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins i64mem:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst,
(v2i64 (scalar_to_vector (loadi64 addr:$src))))]>,
[AVX512] Bring vmovq instructions names into alignment with the AVX and SSE names. Add a missing encoding to disassembler and assembler. I believe this also fixes a case where a 64-bit memory form that is documented as being unsupported in 32-bit mode was able to be selected there. llvm-svn: 256483
2015-12-28 14:11:42 +08:00
EVEX, VEX_W, EVEX_CD8<8, CD8VT8>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 MOVSS, MOVSD
//===----------------------------------------------------------------------===//
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_move_scalar <string asm, SDNode OpNode,
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
X86VectorVTInfo _> {
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm rr_Int : AVX512_maskable_scalar<0x10, MRMSrcReg, _, (outs _.RC:$dst),
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
(ins _.RC:$src1, _.RC:$src2),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
asm, "$src2, $src1","$src1, $src2",
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
(_.VT (OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2))),
IIC_SSE_MOV_S_RR>, EVEX_4V;
let Constraints = "$src1 = $dst" , mayLoad = 1 in
defm rm_Int : AVX512_maskable_3src_scalar<0x10, MRMSrcMem, _,
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(outs _.RC:$dst),
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
(ins _.ScalarMemOp:$src),
asm,"$src","$src",
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(_.VT (OpNode (_.VT _.RC:$src1),
(_.VT (scalar_to_vector
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
(_.ScalarLdFrag addr:$src)))))>, EVEX;
let isCodeGenOnly = 1 in {
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
def rr : AVX512PI<0x10, MRMSrcReg, (outs _.RC:$dst),
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
(ins _.RC:$src1, _.FRC:$src2),
!strconcat(asm, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set _.RC:$dst, (_.VT (OpNode _.RC:$src1,
(scalar_to_vector _.FRC:$src2))))],
_.ExeDomain,IIC_SSE_MOV_S_RR>, EVEX_4V;
let mayLoad = 1 in
def rm : AVX512PI<0x10, MRMSrcMem, (outs _.FRC:$dst), (ins _.ScalarMemOp:$src),
!strconcat(asm, "\t{$src, $dst|$dst, $src}"),
[(set _.FRC:$dst, (_.ScalarLdFrag addr:$src))],
_.ExeDomain, IIC_SSE_MOV_S_RM>, EVEX;
}
AVX-512: Added intrinsic for VMOVSS store form with mask. llvm-svn: 216530
2014-08-27 15:38:43 +08:00
let mayStore = 1 in {
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
def mr: AVX512PI<0x11, MRMDestMem, (outs), (ins _.ScalarMemOp:$dst, _.FRC:$src),
!strconcat(asm, "\t{$src, $dst|$dst, $src}"),
[(store _.FRC:$src, addr:$dst)], _.ExeDomain, IIC_SSE_MOV_S_MR>,
EVEX;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
def mrk: AVX512PI<0x11, MRMDestMem, (outs),
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
(ins _.ScalarMemOp:$dst, VK1WM:$mask, _.FRC:$src),
!strconcat(asm, "\t{$src, $dst {${mask}}|$dst {${mask}}, $src}"),
[], _.ExeDomain, IIC_SSE_MOV_S_MR>, EVEX, EVEX_K;
AVX-512: Added intrinsic for VMOVSS store form with mask. llvm-svn: 216530
2014-08-27 15:38:43 +08:00
} // mayStore
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
defm VMOVSSZ : avx512_move_scalar<"vmovss", X86Movss, f32x_info>,
VEX_LIG, XS, EVEX_CD8<32, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
defm VMOVSDZ : avx512_move_scalar<"vmovsd", X86Movsd, f64x_info>,
VEX_LIG, XD, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Strengthen type checks on the X86ISD::SELECT node. Saves over 800 bytes in the DAG isel table by removing type checks for the condition operand which is always a vector or scalar of i1 matching the the number of elements in the other operands. llvm-svn: 269885
2016-05-18 14:55:59 +08:00
def : Pat<(f32 (X86selects VK1WM:$mask, (f32 FR32X:$src1), (f32 FR32X:$src2))),
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
(COPY_TO_REGCLASS (VMOVSSZrr_Intk (COPY_TO_REGCLASS FR32X:$src2, VR128X),
VK1WM:$mask, (v4f32 (IMPLICIT_DEF)),(COPY_TO_REGCLASS FR32X:$src1, VR128X)), FR32X)>;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
[AVX512] Strengthen type checks on the X86ISD::SELECT node. Saves over 800 bytes in the DAG isel table by removing type checks for the condition operand which is always a vector or scalar of i1 matching the the number of elements in the other operands. llvm-svn: 269885
2016-05-18 14:55:59 +08:00
def : Pat<(f64 (X86selects VK1WM:$mask, (f64 FR64X:$src1), (f64 FR64X:$src2))),
[X86][AVX512] add vmovss/sd missing encoding Differential Revision: http://reviews.llvm.org/D14701 llvm-svn: 254875
2015-12-06 21:26:56 +08:00
(COPY_TO_REGCLASS (VMOVSDZrr_Intk (COPY_TO_REGCLASS FR64X:$src2, VR128X),
VK1WM:$mask, (v2f64 (IMPLICIT_DEF)), (COPY_TO_REGCLASS FR64X:$src1, VR128X)), FR64X)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added intrinsic for VMOVSS store form with mask. llvm-svn: 216530
2014-08-27 15:38:43 +08:00
def : Pat<(int_x86_avx512_mask_store_ss addr:$dst, VR128X:$src, GR8:$mask),
(VMOVSSZmrk addr:$dst, (i1 (COPY_TO_REGCLASS GR8:$mask, VK1WM)),
(COPY_TO_REGCLASS VR128X:$src, FR32X))>;
[AVX512] Add hasSideEffects/mayLoad/mayStore flags to some instructions. llvm-svn: 268174
2016-05-01 09:03:56 +08:00
let hasSideEffects = 0 in
AVX512: Implemented encoding for the vmovss.s and vmovsd.s instructions. Differential Revision: http://reviews.llvm.org/D14771 llvm-svn: 253547
2015-11-19 15:58:33 +08:00
defm VMOVSSZrr_REV : AVX512_maskable_in_asm<0x11, MRMDestReg, f32x_info,
(outs VR128X:$dst), (ins VR128X:$src1, VR128X:$src2),
"vmovss.s", "$src2, $src1", "$src1, $src2", []>,
XS, EVEX_4V, VEX_LIG;
[AVX512] Add hasSideEffects/mayLoad/mayStore flags to some instructions. llvm-svn: 268174
2016-05-01 09:03:56 +08:00
let hasSideEffects = 0 in
AVX512: Implemented encoding for the vmovss.s and vmovsd.s instructions. Differential Revision: http://reviews.llvm.org/D14771 llvm-svn: 253547
2015-11-19 15:58:33 +08:00
defm VMOVSSDrr_REV : AVX512_maskable_in_asm<0x11, MRMDestReg, f64x_info,
(outs VR128X:$dst), (ins VR128X:$src1, VR128X:$src2),
"vmovsd.s", "$src2, $src1", "$src1, $src2", []>,
XD, EVEX_4V, VEX_LIG, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in {
let AddedComplexity = 15 in {
// Move scalar to XMM zero-extended, zeroing a VR128X then do a
// MOVS{S,D} to the lower bits.
def : Pat<(v4f32 (X86vzmovl (v4f32 (scalar_to_vector FR32X:$src)))),
(VMOVSSZrr (v4f32 (V_SET0)), FR32X:$src)>;
def : Pat<(v4f32 (X86vzmovl (v4f32 VR128X:$src))),
(VMOVSSZrr (v4f32 (V_SET0)), (COPY_TO_REGCLASS VR128X:$src, FR32X))>;
def : Pat<(v4i32 (X86vzmovl (v4i32 VR128X:$src))),
(VMOVSSZrr (v4i32 (V_SET0)), (COPY_TO_REGCLASS VR128X:$src, FR32X))>;
def : Pat<(v2f64 (X86vzmovl (v2f64 (scalar_to_vector FR64X:$src)))),
(VMOVSDZrr (v2f64 (V_SET0)), FR64X:$src)>;
// Move low f32 and clear high bits.
def : Pat<(v8f32 (X86vzmovl (v8f32 VR256X:$src))),
(SUBREG_TO_REG (i32 0),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(VMOVSSZrr (v4f32 (V_SET0)),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(EXTRACT_SUBREG (v8f32 VR256X:$src), sub_xmm)), sub_xmm)>;
def : Pat<(v8i32 (X86vzmovl (v8i32 VR256X:$src))),
(SUBREG_TO_REG (i32 0),
(VMOVSSZrr (v4i32 (V_SET0)),
(EXTRACT_SUBREG (v8i32 VR256X:$src), sub_xmm)), sub_xmm)>;
}
let AddedComplexity = 20 in {
// MOVSSrm zeros the high parts of the register; represent this
// with SUBREG_TO_REG. The AVX versions also write: DST[255:128] <- 0
def : Pat<(v4f32 (X86vzmovl (v4f32 (scalar_to_vector (loadf32 addr:$src))))),
(COPY_TO_REGCLASS (VMOVSSZrm addr:$src), VR128X)>;
def : Pat<(v4f32 (scalar_to_vector (loadf32 addr:$src))),
(COPY_TO_REGCLASS (VMOVSSZrm addr:$src), VR128X)>;
def : Pat<(v4f32 (X86vzmovl (loadv4f32 addr:$src))),
(COPY_TO_REGCLASS (VMOVSSZrm addr:$src), VR128X)>;
// MOVSDrm zeros the high parts of the register; represent this
// with SUBREG_TO_REG. The AVX versions also write: DST[255:128] <- 0
def : Pat<(v2f64 (X86vzmovl (v2f64 (scalar_to_vector (loadf64 addr:$src))))),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
def : Pat<(v2f64 (scalar_to_vector (loadf64 addr:$src))),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
def : Pat<(v2f64 (X86vzmovl (loadv2f64 addr:$src))),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
def : Pat<(v2f64 (X86vzmovl (bc_v2f64 (loadv4f32 addr:$src)))),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
def : Pat<(v2f64 (X86vzload addr:$src)),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
// Represent the same patterns above but in the form they appear for
// 256-bit types
def : Pat<(v8i32 (X86vzmovl (insert_subvector undef,
(v4i32 (scalar_to_vector (loadi32 addr:$src))), (iPTR 0)))),
AVX-512: fixed a bug in getLoadStoreRegOpcode() for AVX-512 target llvm-svn: 191818
2013-10-02 20:20:42 +08:00
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrm addr:$src), sub_xmm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f32 (X86vzmovl (insert_subvector undef,
(v4f32 (scalar_to_vector (loadf32 addr:$src))), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVSSZrm addr:$src), sub_xmm)>;
def : Pat<(v4f64 (X86vzmovl (insert_subvector undef,
(v2f64 (scalar_to_vector (loadf64 addr:$src))), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVSDZrm addr:$src), sub_xmm)>;
[X86][SSE] Select domain for 32/64-bit partial loads for EltsFromConsecutiveLoads Choose between MOVD/MOVSS and MOVQ/MOVSD depending on the target vector type. This has a lot fewer test changes than trying to add this to X86InstrInfo::setExecutionDomain..... llvm-svn: 259816
2016-02-05 03:27:51 +08:00
def : Pat<(v4f64 (X86vzload addr:$src)),
(SUBREG_TO_REG (i32 0), (VMOVSDZrm addr:$src), sub_xmm)>;
[X86][SSE] Add general 32-bit LOAD + VZEXT_MOVL support to EltsFromConsecutiveLoads This patch adds support for consecutive (load/undef elements) 32-bit loads, followed by trailing undef/zero elements to be combined to a single MOVD load. Differential Revision: http://reviews.llvm.org/D16729 llvm-svn: 259796
2016-02-05 00:12:56 +08:00
// Represent the same patterns above but in the form they appear for
// 512-bit types
def : Pat<(v16i32 (X86vzmovl (insert_subvector undef,
(v4i32 (scalar_to_vector (loadi32 addr:$src))), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrm addr:$src), sub_xmm)>;
def : Pat<(v16f32 (X86vzmovl (insert_subvector undef,
(v4f32 (scalar_to_vector (loadf32 addr:$src))), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVSSZrm addr:$src), sub_xmm)>;
def : Pat<(v8f64 (X86vzmovl (insert_subvector undef,
(v2f64 (scalar_to_vector (loadf64 addr:$src))), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVSDZrm addr:$src), sub_xmm)>;
[X86][SSE] Select domain for 32/64-bit partial loads for EltsFromConsecutiveLoads Choose between MOVD/MOVSS and MOVQ/MOVSD depending on the target vector type. This has a lot fewer test changes than trying to add this to X86InstrInfo::setExecutionDomain..... llvm-svn: 259816
2016-02-05 03:27:51 +08:00
def : Pat<(v8f64 (X86vzload addr:$src)),
(SUBREG_TO_REG (i32 0), (VMOVSDZrm addr:$src), sub_xmm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
def : Pat<(v8f32 (X86vzmovl (insert_subvector undef,
(v4f32 (scalar_to_vector FR32X:$src)), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (v4f32 (VMOVSSZrr (v4f32 (V_SET0)),
FR32X:$src)), sub_xmm)>;
def : Pat<(v4f64 (X86vzmovl (insert_subvector undef,
(v2f64 (scalar_to_vector FR64X:$src)), (iPTR 0)))),
(SUBREG_TO_REG (i64 0), (v2f64 (VMOVSDZrr (v2f64 (V_SET0)),
FR64X:$src)), sub_xmm)>;
def : Pat<(v4i64 (X86vzmovl (insert_subvector undef,
(v2i64 (scalar_to_vector (loadi64 addr:$src))), (iPTR 0)))),
AVX-512: fixed a bug in getLoadStoreRegOpcode() for AVX-512 target llvm-svn: 191818
2013-10-02 20:20:42 +08:00
(SUBREG_TO_REG (i64 0), (VMOVQI2PQIZrm addr:$src), sub_xmm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Move low f64 and clear high bits.
def : Pat<(v4f64 (X86vzmovl (v4f64 VR256X:$src))),
(SUBREG_TO_REG (i32 0),
(VMOVSDZrr (v2f64 (V_SET0)),
(EXTRACT_SUBREG (v4f64 VR256X:$src), sub_xmm)), sub_xmm)>;
def : Pat<(v4i64 (X86vzmovl (v4i64 VR256X:$src))),
(SUBREG_TO_REG (i32 0), (VMOVSDZrr (v2i64 (V_SET0)),
(EXTRACT_SUBREG (v4i64 VR256X:$src), sub_xmm)), sub_xmm)>;
// Extract and store.
Start replacing vector_extract/vector_insert with extractelt/insertelt These are redundant pairs of nodes defined for INSERT_VECTOR_ELEMENT/EXTRACT_VECTOR_ELEMENT. insertelement/extractelement are slightly closer to the corresponding C++ node name, and has stricter type checking so prefer it. Update targets to only use these nodes where it is trivial to do so. AArch64, ARM, and Mips all have various type errors on simple replacement, so they will need work to fix. Example from AArch64: def : Pat<(sext_inreg (vector_extract (v16i8 V128:$Rn), VectorIndexB:$idx), i8), (i32 (SMOVvi8to32 V128:$Rn, VectorIndexB:$idx))>; Which is trying to do sext_inreg i8, i8. llvm-svn: 255359
2015-12-12 03:20:16 +08:00
def : Pat<(store (f32 (extractelt (v4f32 VR128X:$src), (iPTR 0))),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
addr:$dst),
(VMOVSSZmr addr:$dst, (COPY_TO_REGCLASS (v4f32 VR128X:$src), FR32X))>;
// Shuffle with VMOVSS
def : Pat<(v4i32 (X86Movss VR128X:$src1, VR128X:$src2)),
(VMOVSSZrr (v4i32 VR128X:$src1),
(COPY_TO_REGCLASS (v4i32 VR128X:$src2), FR32X))>;
def : Pat<(v4f32 (X86Movss VR128X:$src1, VR128X:$src2)),
(VMOVSSZrr (v4f32 VR128X:$src1),
(COPY_TO_REGCLASS (v4f32 VR128X:$src2), FR32X))>;
// 256-bit variants
def : Pat<(v8i32 (X86Movss VR256X:$src1, VR256X:$src2)),
(SUBREG_TO_REG (i32 0),
(VMOVSSZrr (EXTRACT_SUBREG (v8i32 VR256X:$src1), sub_xmm),
(EXTRACT_SUBREG (v8i32 VR256X:$src2), sub_xmm)),
sub_xmm)>;
def : Pat<(v8f32 (X86Movss VR256X:$src1, VR256X:$src2)),
(SUBREG_TO_REG (i32 0),
(VMOVSSZrr (EXTRACT_SUBREG (v8f32 VR256X:$src1), sub_xmm),
(EXTRACT_SUBREG (v8f32 VR256X:$src2), sub_xmm)),
sub_xmm)>;
// Shuffle with VMOVSD
def : Pat<(v2i64 (X86Movsd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v2f64 (X86Movsd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v4f32 (X86Movsd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v4i32 (X86Movsd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
// 256-bit variants
def : Pat<(v4i64 (X86Movsd VR256X:$src1, VR256X:$src2)),
(SUBREG_TO_REG (i32 0),
(VMOVSDZrr (EXTRACT_SUBREG (v4i64 VR256X:$src1), sub_xmm),
(EXTRACT_SUBREG (v4i64 VR256X:$src2), sub_xmm)),
sub_xmm)>;
def : Pat<(v4f64 (X86Movsd VR256X:$src1, VR256X:$src2)),
(SUBREG_TO_REG (i32 0),
(VMOVSDZrr (EXTRACT_SUBREG (v4f64 VR256X:$src1), sub_xmm),
(EXTRACT_SUBREG (v4f64 VR256X:$src2), sub_xmm)),
sub_xmm)>;
def : Pat<(v2f64 (X86Movlpd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v2i64 (X86Movlpd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v4f32 (X86Movlps VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v4i32 (X86Movlps VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
}
let AddedComplexity = 15 in
def VMOVZPQILo2PQIZrr : AVX512XSI<0x7E, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set VR128X:$dst, (v2i64 (X86vzmovl
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v2i64 VR128X:$src))))],
IIC_SSE_MOVQ_RR>, EVEX, VEX_W;
AVX512: add encoding tests for vmovq/d instructions. llvm-svn: 251903
2015-11-03 15:30:17 +08:00
let AddedComplexity = 20 , isCodeGenOnly = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def VMOVZPQILo2PQIZrm : AVX512XSI<0x7E, MRMSrcMem, (outs VR128X:$dst),
(ins i128mem:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst, (v2i64 (X86vzmovl
(loadv2i64 addr:$src))))],
IIC_SSE_MOVDQ>, EVEX, VEX_W,
EVEX_CD8<8, CD8VT8>;
let Predicates = [HasAVX512] in {
// AVX 128-bit movd/movq instruction write zeros in the high 128-bit part.
let AddedComplexity = 20 in {
def : Pat<(v4i32 (X86vzmovl (v4i32 (scalar_to_vector (loadi32 addr:$src))))),
(VMOVDI2PDIZrm addr:$src)>;
AVX-512: Added X86vzmovl patterns llvm-svn: 191733
2013-10-01 16:38:02 +08:00
def : Pat<(v2i64 (X86vzmovl (v2i64 (scalar_to_vector GR64:$src)))),
(VMOV64toPQIZrr GR64:$src)>;
def : Pat<(v4i32 (X86vzmovl (v4i32 (scalar_to_vector GR32:$src)))),
(VMOVDI2PDIZrr GR32:$src)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v4i32 (X86vzmovl (bc_v4i32 (loadv4f32 addr:$src)))),
(VMOVDI2PDIZrm addr:$src)>;
def : Pat<(v4i32 (X86vzmovl (bc_v4i32 (loadv2i64 addr:$src)))),
(VMOVDI2PDIZrm addr:$src)>;
def : Pat<(v2i64 (X86vzmovl (loadv2i64 addr:$src))),
(VMOVZPQILo2PQIZrm addr:$src)>;
def : Pat<(v2f64 (X86vzmovl (v2f64 VR128X:$src))),
(VMOVZPQILo2PQIZrr VR128X:$src)>;
Add AVX512 patterns for v16i32 broadcast and v2i64 zero extend load. Patch by Aleksey Bader. llvm-svn: 196435
2013-12-05 08:11:25 +08:00
def : Pat<(v2i64 (X86vzload addr:$src)),
(VMOVZPQILo2PQIZrm addr:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added X86vzmovl patterns llvm-svn: 191733
2013-10-01 16:38:02 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Use regular 128-bit instructions to match 256-bit scalar_to_vec+zext.
def : Pat<(v8i32 (X86vzmovl (insert_subvector undef,
(v4i32 (scalar_to_vector GR32:$src)),(iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src), sub_xmm)>;
def : Pat<(v4i64 (X86vzmovl (insert_subvector undef,
(v2i64 (scalar_to_vector GR64:$src)),(iPTR 0)))),
(SUBREG_TO_REG (i64 0), (VMOV64toPQIZrr GR64:$src), sub_xmm)>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
def : Pat<(v4i64 (X86vzload addr:$src)),
(SUBREG_TO_REG (i64 0), (VMOVZPQILo2PQIZrm addr:$src), sub_xmm)>;
// Use regular 128-bit instructions to match 512-bit scalar_to_vec+zext.
def : Pat<(v8i64 (X86vzload addr:$src)),
(SUBREG_TO_REG (i64 0), (VMOVZPQILo2PQIZrm addr:$src), sub_xmm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
def : Pat<(v16i32 (X86Vinsert (v16i32 immAllZerosV), GR32:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src2), sub_xmm)>;
def : Pat<(v8i64 (X86Vinsert (bc_v8i64 (v16i32 immAllZerosV)), GR64:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOV64toPQIZrr GR64:$src2), sub_xmm)>;
def : Pat<(v16i32 (X86Vinsert undef, GR32:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src2), sub_xmm)>;
def : Pat<(v8i64 (X86Vinsert undef, GR64:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOV64toPQIZrr GR64:$src2), sub_xmm)>;
[X86] AVX512: Add vmovntdqa Along with the corresponding intrinsic and tests. llvm-svn: 210543
2014-06-11 00:39:53 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Non-temporals
//===----------------------------------------------------------------------===//
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
let SchedRW = [WriteLoad] in {
def VMOVNTDQAZrm : AVX512PI<0x2A, MRMSrcMem, (outs VR512:$dst),
(ins i512mem:$src), "vmovntdqa\t{$src, $dst|$dst, $src}",
[(set VR512:$dst, (int_x86_avx512_movntdqa addr:$src))],
SSEPackedInt>, EVEX, T8PD, EVEX_V512,
EVEX_CD8<64, CD8VF>;
let Predicates = [HasAVX512, HasVLX] in {
def VMOVNTDQAZ256rm : AVX512PI<0x2A, MRMSrcMem, (outs VR256X:$dst),
(ins i256mem:$src),
"vmovntdqa\t{$src, $dst|$dst, $src}", [],
SSEPackedInt>, EVEX, T8PD, EVEX_V256,
EVEX_CD8<64, CD8VF>;
def VMOVNTDQAZ128rm : AVX512PI<0x2A, MRMSrcMem, (outs VR128X:$dst),
(ins i128mem:$src),
"vmovntdqa\t{$src, $dst|$dst, $src}", [],
SSEPackedInt>, EVEX, T8PD, EVEX_V128,
EVEX_CD8<64, CD8VF>;
}
}
[X86] AVX512: Add vmovntdqa Along with the corresponding intrinsic and tests. llvm-svn: 210543
2014-06-11 00:39:53 +08:00
AVX512: Store (MOVNTPD, MOVNTPS, MOVNTDQ) using non-temporal hint intrinsic implementation. Differential Revision: http://reviews.llvm.org/D16350 llvm-svn: 258309
2016-01-20 21:11:47 +08:00
multiclass avx512_movnt<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
PatFrag st_frag = alignednontemporalstore,
InstrItinClass itin = IIC_SSE_MOVNT> {
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
let SchedRW = [WriteStore], mayStore = 1,
AddedComplexity = 400 in
AVX512: Store (MOVNTPD, MOVNTPS, MOVNTDQ) using non-temporal hint intrinsic implementation. Differential Revision: http://reviews.llvm.org/D16350 llvm-svn: 258309
2016-01-20 21:11:47 +08:00
def mr : AVX512PI<opc, MRMDestMem, (outs), (ins _.MemOp:$dst, _.RC:$src),
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX512: Store (MOVNTPD, MOVNTPS, MOVNTDQ) using non-temporal hint intrinsic implementation. Differential Revision: http://reviews.llvm.org/D16350 llvm-svn: 258309
2016-01-20 21:11:47 +08:00
[(st_frag (_.VT _.RC:$src), addr:$dst)],
_.ExeDomain, itin>, EVEX, EVEX_CD8<_.EltSize, CD8VF>;
[X86] AVX512: Add non-temporal stores Note that I followed the AVX2 convention here and didn't add LLVM intrinsics for stores. These can be generated with the nontemporal hint on LLVM IR stores (see new test). The GCC builtins are lowered directly into nontemporal stores. <rdar://problem/17082571> llvm-svn: 211176
2014-06-19 00:51:10 +08:00
}
AVX512: Store (MOVNTPD, MOVNTPS, MOVNTDQ) using non-temporal hint intrinsic implementation. Differential Revision: http://reviews.llvm.org/D16350 llvm-svn: 258309
2016-01-20 21:11:47 +08:00
multiclass avx512_movnt_vl<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
let Predicates = [HasAVX512] in
defm Z : avx512_movnt<opc, OpcodeStr, VTInfo.info512>, EVEX_V512;
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
AVX512: Store (MOVNTPD, MOVNTPS, MOVNTDQ) using non-temporal hint intrinsic implementation. Differential Revision: http://reviews.llvm.org/D16350 llvm-svn: 258309
2016-01-20 21:11:47 +08:00
let Predicates = [HasAVX512, HasVLX] in {
defm Z256 : avx512_movnt<opc, OpcodeStr, VTInfo.info256>, EVEX_V256;
defm Z128 : avx512_movnt<opc, OpcodeStr, VTInfo.info128>, EVEX_V128;
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
}
}
AVX512: Store (MOVNTPD, MOVNTPS, MOVNTDQ) using non-temporal hint intrinsic implementation. Differential Revision: http://reviews.llvm.org/D16350 llvm-svn: 258309
2016-01-20 21:11:47 +08:00
defm VMOVNTDQ : avx512_movnt_vl<0xE7, "vmovntdq", avx512vl_i64_info>, PD;
defm VMOVNTPD : avx512_movnt_vl<0x2B, "vmovntpd", avx512vl_f64_info>, PD, VEX_W;
defm VMOVNTPS : avx512_movnt_vl<0x2B, "vmovntps", avx512vl_f32_info>, PS;
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
[AVX512] Add non-temporal store patterns for v16i32/v32i16/v64i8. llvm-svn: 268889
2016-05-09 07:43:17 +08:00
let Predicates = [HasAVX512], AddedComplexity = 400 in {
def : Pat<(alignednontemporalstore (v16i32 VR512:$src), addr:$dst),
(VMOVNTDQZmr addr:$dst, VR512:$src)>;
def : Pat<(alignednontemporalstore (v32i16 VR512:$src), addr:$dst),
(VMOVNTDQZmr addr:$dst, VR512:$src)>;
def : Pat<(alignednontemporalstore (v64i8 VR512:$src), addr:$dst),
(VMOVNTDQZmr addr:$dst, VR512:$src)>;
}
[AVX512] Add missing patterns for non-temporal stores of 128/256-bit vXi8/vXi16/vXi32 when VLX is enabled. The equivalent AVX1/2 patterns are disabled by VLX. This caused regular stores to be emitted instead. llvm-svn: 268886
2016-05-09 07:08:45 +08:00
let Predicates = [HasVLX], AddedComplexity = 400 in {
def : Pat<(alignednontemporalstore (v8i32 VR256X:$src), addr:$dst),
(VMOVNTDQZ256mr addr:$dst, VR256X:$src)>;
def : Pat<(alignednontemporalstore (v16i16 VR256X:$src), addr:$dst),
(VMOVNTDQZ256mr addr:$dst, VR256X:$src)>;
def : Pat<(alignednontemporalstore (v32i8 VR256X:$src), addr:$dst),
(VMOVNTDQZ256mr addr:$dst, VR256X:$src)>;
def : Pat<(alignednontemporalstore (v4i32 VR128X:$src), addr:$dst),
(VMOVNTDQZ128mr addr:$dst, VR128X:$src)>;
def : Pat<(alignednontemporalstore (v8i16 VR128X:$src), addr:$dst),
(VMOVNTDQZ128mr addr:$dst, VR128X:$src)>;
def : Pat<(alignednontemporalstore (v16i8 VR128X:$src), addr:$dst),
(VMOVNTDQZ128mr addr:$dst, VR128X:$src)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Integer arithmetic
//
multiclass avx512_binop_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
X86VectorVTInfo _, OpndItins itins,
bit IsCommutable = 0> {
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, _.RC:$src2)),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
itins.rr, IsCommutable>,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
AVX512BIBase, EVEX_4V;
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
let mayLoad = 1 in
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1,
(bitconvert (_.LdFrag addr:$src2)))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
itins.rm>,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
AVX512BIBase, EVEX_4V;
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
}
multiclass avx512_binop_rmb<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _, OpndItins itins,
bit IsCommutable = 0> :
avx512_binop_rm<opc, OpcodeStr, OpNode, _, itins, IsCommutable> {
let mayLoad = 1 in
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rmb : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(_.VT (OpNode _.RC:$src1,
(X86VBroadcast
(_.ScalarLdFrag addr:$src2)))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
itins.rm>,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
AVX512BIBase, EVEX_4V, EVEX_B;
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
}
[AVX512] Extended avx512_binop_rm for AVX512VL subsets. Added avx512_binop_rm_vl multiclass for VL subset Added encoding tests llvm-svn: 219390
2014-10-09 16:38:48 +08:00
multiclass avx512_binop_rm_vl<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, OpndItins itins,
Predicate prd, bit IsCommutable = 0> {
let Predicates = [prd] in
defm Z : avx512_binop_rm<opc, OpcodeStr, OpNode, VTInfo.info512, itins,
IsCommutable>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_binop_rm<opc, OpcodeStr, OpNode, VTInfo.info256, itins,
IsCommutable>, EVEX_V256;
defm Z128 : avx512_binop_rm<opc, OpcodeStr, OpNode, VTInfo.info128, itins,
IsCommutable>, EVEX_V128;
}
}
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
multiclass avx512_binop_rmb_vl<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, OpndItins itins,
Predicate prd, bit IsCommutable = 0> {
let Predicates = [prd] in
defm Z : avx512_binop_rmb<opc, OpcodeStr, OpNode, VTInfo.info512, itins,
IsCommutable>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_binop_rmb<opc, OpcodeStr, OpNode, VTInfo.info256, itins,
IsCommutable>, EVEX_V256;
defm Z128 : avx512_binop_rmb<opc, OpcodeStr, OpNode, VTInfo.info128, itins,
IsCommutable>, EVEX_V128;
}
}
multiclass avx512_binop_rm_vl_q<bits<8> opc, string OpcodeStr, SDNode OpNode,
OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm NAME : avx512_binop_rmb_vl<opc, OpcodeStr, OpNode, avx512vl_i64_info,
itins, prd, IsCommutable>,
VEX_W, EVEX_CD8<64, CD8VF>;
}
multiclass avx512_binop_rm_vl_d<bits<8> opc, string OpcodeStr, SDNode OpNode,
OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm NAME : avx512_binop_rmb_vl<opc, OpcodeStr, OpNode, avx512vl_i32_info,
itins, prd, IsCommutable>, EVEX_CD8<32, CD8VF>;
}
multiclass avx512_binop_rm_vl_w<bits<8> opc, string OpcodeStr, SDNode OpNode,
OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm NAME : avx512_binop_rm_vl<opc, OpcodeStr, OpNode, avx512vl_i16_info,
itins, prd, IsCommutable>, EVEX_CD8<16, CD8VF>;
}
multiclass avx512_binop_rm_vl_b<bits<8> opc, string OpcodeStr, SDNode OpNode,
OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm NAME : avx512_binop_rm_vl<opc, OpcodeStr, OpNode, avx512vl_i8_info,
itins, prd, IsCommutable>, EVEX_CD8<8, CD8VF>;
}
multiclass avx512_binop_rm_vl_dq<bits<8> opc_d, bits<8> opc_q, string OpcodeStr,
SDNode OpNode, OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm Q : avx512_binop_rm_vl_q<opc_q, OpcodeStr#"q", OpNode, itins, prd,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
IsCommutable>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm D : avx512_binop_rm_vl_d<opc_d, OpcodeStr#"d", OpNode, itins, prd,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
IsCommutable>;
}
multiclass avx512_binop_rm_vl_bw<bits<8> opc_b, bits<8> opc_w, string OpcodeStr,
SDNode OpNode, OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm W : avx512_binop_rm_vl_w<opc_w, OpcodeStr#"w", OpNode, itins, prd,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
IsCommutable>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm B : avx512_binop_rm_vl_b<opc_b, OpcodeStr#"b", OpNode, itins, prd,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
IsCommutable>;
}
multiclass avx512_binop_rm_vl_all<bits<8> opc_b, bits<8> opc_w,
bits<8> opc_d, bits<8> opc_q,
string OpcodeStr, SDNode OpNode,
OpndItins itins, bit IsCommutable = 0> {
defm NAME : avx512_binop_rm_vl_dq<opc_d, opc_q, OpcodeStr, OpNode,
itins, HasAVX512, IsCommutable>,
avx512_binop_rm_vl_bw<opc_b, opc_w, OpcodeStr, OpNode,
itins, HasBWI, IsCommutable>;
}
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
multiclass avx512_binop_rm2<bits<8> opc, string OpcodeStr, OpndItins itins,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
SDNode OpNode,X86VectorVTInfo _Src,
[X86][AVX512VBMI] add encoding and intrinsics for Multishift Differential Revision: http://reviews.llvm.org/D16399 llvm-svn: 259363
2016-02-01 23:48:21 +08:00
X86VectorVTInfo _Dst, X86VectorVTInfo _Brdct,
bit IsCommutable = 0> {
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
defm rr : AVX512_maskable<opc, MRMSrcReg, _Dst, (outs _Dst.RC:$dst),
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
(ins _Src.RC:$src1, _Src.RC:$src2), OpcodeStr,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
"$src2, $src1","$src1, $src2",
(_Dst.VT (OpNode
(_Src.VT _Src.RC:$src1),
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
(_Src.VT _Src.RC:$src2))),
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
itins.rr, IsCommutable>,
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
AVX512BIBase, EVEX_4V;
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
let mayLoad = 1 in {
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
defm rm : AVX512_maskable<opc, MRMSrcMem, _Dst, (outs _Dst.RC:$dst),
(ins _Src.RC:$src1, _Src.MemOp:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(_Dst.VT (OpNode (_Src.VT _Src.RC:$src1),
(bitconvert (_Src.LdFrag addr:$src2)))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
itins.rm>,
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
AVX512BIBase, EVEX_4V;
defm rmb : AVX512_maskable<opc, MRMSrcMem, _Dst, (outs _Dst.RC:$dst),
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(ins _Src.RC:$src1, _Dst.ScalarMemOp:$src2),
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
OpcodeStr,
[X86][AVX512VBMI] add encoding and intrinsics for Multishift Differential Revision: http://reviews.llvm.org/D16399 llvm-svn: 259363
2016-02-01 23:48:21 +08:00
"${src2}"##_Brdct.BroadcastStr##", $src1",
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
"$src1, ${src2}"##_Dst.BroadcastStr,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(_Dst.VT (OpNode (_Src.VT _Src.RC:$src1), (bitconvert
[X86][AVX512VBMI] add encoding and intrinsics for Multishift Differential Revision: http://reviews.llvm.org/D16399 llvm-svn: 259363
2016-02-01 23:48:21 +08:00
(_Brdct.VT (X86VBroadcast
(_Brdct.ScalarLdFrag addr:$src2)))))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
itins.rm>,
AVX-512: intrinsics for VPADD, VPMULDQ and VPSUB by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233906
2015-04-02 18:51:40 +08:00
AVX512BIBase, EVEX_4V, EVEX_B;
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
defm VPADD : avx512_binop_rm_vl_all<0xFC, 0xFD, 0xFE, 0xD4, "vpadd", add,
SSE_INTALU_ITINS_P, 1>;
defm VPSUB : avx512_binop_rm_vl_all<0xF8, 0xF9, 0xFA, 0xFB, "vpsub", sub,
SSE_INTALU_ITINS_P, 0>;
AVX-512: added integer "add" and "sub" instructions with saturation for SKX with intrinsics and tests by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236418
2015-05-04 20:35:55 +08:00
defm VPADDS : avx512_binop_rm_vl_bw<0xEC, 0xED, "vpadds", X86adds,
SSE_INTALU_ITINS_P, HasBWI, 1>;
defm VPSUBS : avx512_binop_rm_vl_bw<0xE8, 0xE9, "vpsubs", X86subs,
SSE_INTALU_ITINS_P, HasBWI, 0>;
defm VPADDUS : avx512_binop_rm_vl_bw<0xDC, 0xDD, "vpaddus", X86addus,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>;
AVX-512: added integer "add" and "sub" instructions with saturation for SKX with intrinsics and tests by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236418
2015-05-04 20:35:55 +08:00
defm VPSUBUS : avx512_binop_rm_vl_bw<0xD8, 0xD9, "vpsubus", X86subus,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
SSE_INTALU_ITINS_P, HasBWI, 0>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMULLD : avx512_binop_rm_vl_d<0x40, "vpmulld", mul,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMULLW : avx512_binop_rm_vl_w<0xD5, "vpmullw", mul,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMULLQ : avx512_binop_rm_vl_q<0x40, "vpmullq", mul,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
SSE_INTALU_ITINS_P, HasDQI, 1>, T8PD;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMULHW : avx512_binop_rm_vl_w<0xE5, "vpmulhw", mulhs, SSE_INTALU_ITINS_P,
[x86][AVX512] add Multiply High Op include encoding and intrinsics tests. review http://reviews.llvm.org/D10896 llvm-svn: 241406
2015-07-05 20:23:20 +08:00
HasBWI, 1>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMULHUW : avx512_binop_rm_vl_w<0xE4, "vpmulhuw", mulhu, SSE_INTMUL_ITINS_P,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
HasBWI, 1>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMULHRSW : avx512_binop_rm_vl_w<0x0B, "vpmulhrsw", X86mulhrs, SSE_INTMUL_ITINS_P,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
HasBWI, 1>, T8PD;
[AVX512] add instructions: VPAVGB and VPAVGW review http://reviews.llvm.org/D10504 llvm-svn: 240012
2015-06-18 20:30:53 +08:00
defm VPAVG : avx512_binop_rm_vl_bw<0xE0, 0xE3, "vpavg", X86avg,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_binop_all<bits<8> opc, string OpcodeStr, OpndItins itins,
[X86][AVX512VBMI] add encoding and intrinsics for Multishift Differential Revision: http://reviews.llvm.org/D16399 llvm-svn: 259363
2016-02-01 23:48:21 +08:00
AVX512VLVectorVTInfo _SrcVTInfo, AVX512VLVectorVTInfo _DstVTInfo,
SDNode OpNode, Predicate prd, bit IsCommutable = 0> {
let Predicates = [prd] in
defm NAME#Z : avx512_binop_rm2<opc, OpcodeStr, itins, OpNode,
_SrcVTInfo.info512, _DstVTInfo.info512,
v8i64_info, IsCommutable>,
EVEX_V512, EVEX_CD8<64, CD8VF>, VEX_W;
let Predicates = [HasVLX, prd] in {
AVX-512: Added logical and arithmetic instructions for SKX by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 235375
2015-04-21 18:27:40 +08:00
defm NAME#Z256 : avx512_binop_rm2<opc, OpcodeStr, itins, OpNode,
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
_SrcVTInfo.info256, _DstVTInfo.info256,
[X86][AVX512VBMI] add encoding and intrinsics for Multishift Differential Revision: http://reviews.llvm.org/D16399 llvm-svn: 259363
2016-02-01 23:48:21 +08:00
v4i64x_info, IsCommutable>,
EVEX_V256, EVEX_CD8<64, CD8VF>, VEX_W;
AVX-512: Added logical and arithmetic instructions for SKX by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 235375
2015-04-21 18:27:40 +08:00
defm NAME#Z128 : avx512_binop_rm2<opc, OpcodeStr, itins, OpNode,
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
_SrcVTInfo.info128, _DstVTInfo.info128,
[X86][AVX512VBMI] add encoding and intrinsics for Multishift Differential Revision: http://reviews.llvm.org/D16399 llvm-svn: 259363
2016-02-01 23:48:21 +08:00
v2i64x_info, IsCommutable>,
AVX-512: Added logical and arithmetic instructions for SKX by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 235375
2015-04-21 18:27:40 +08:00
EVEX_V128, EVEX_CD8<64, CD8VF>, VEX_W;
}
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
}
AVX-512: Added logical and arithmetic instructions for SKX by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 235375
2015-04-21 18:27:40 +08:00
defm VPMULDQ : avx512_binop_all<0x28, "vpmuldq", SSE_INTALU_ITINS_P,
[X86][AVX512VBMI] add encoding and intrinsics for Multishift Differential Revision: http://reviews.llvm.org/D16399 llvm-svn: 259363
2016-02-01 23:48:21 +08:00
avx512vl_i32_info, avx512vl_i64_info,
X86pmuldq, HasAVX512, 1>,T8PD;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VPMULUDQ : avx512_binop_all<0xF4, "vpmuludq", SSE_INTMUL_ITINS_P,
[X86][AVX512VBMI] add encoding and intrinsics for Multishift Differential Revision: http://reviews.llvm.org/D16399 llvm-svn: 259363
2016-02-01 23:48:21 +08:00
avx512vl_i32_info, avx512vl_i64_info,
X86pmuludq, HasAVX512, 1>;
defm VPMULTISHIFTQB : avx512_binop_all<0x83, "vpmultishiftqb", SSE_INTALU_ITINS_P,
avx512vl_i8_info, avx512vl_i8_info,
X86multishift, HasVBMI, 0>, T8PD;
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
multiclass avx512_packs_rmb<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _Src, X86VectorVTInfo _Dst> {
let mayLoad = 1 in {
defm rmb : AVX512_maskable<opc, MRMSrcMem, _Dst, (outs _Dst.RC:$dst),
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(ins _Src.RC:$src1, _Src.ScalarMemOp:$src2),
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
OpcodeStr,
"${src2}"##_Src.BroadcastStr##", $src1",
"$src1, ${src2}"##_Src.BroadcastStr,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(_Dst.VT (OpNode (_Src.VT _Src.RC:$src1), (bitconvert
(_Src.VT (X86VBroadcast
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
(_Src.ScalarLdFrag addr:$src2))))))>,
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
EVEX_4V, EVEX_B, EVEX_CD8<_Src.EltSize, CD8VF>;
}
}
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
multiclass avx512_packs_rm<bits<8> opc, string OpcodeStr,
SDNode OpNode,X86VectorVTInfo _Src,
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
X86VectorVTInfo _Dst> {
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
defm rr : AVX512_maskable<opc, MRMSrcReg, _Dst, (outs _Dst.RC:$dst),
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
(ins _Src.RC:$src1, _Src.RC:$src2), OpcodeStr,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
"$src2, $src1","$src1, $src2",
(_Dst.VT (OpNode
(_Src.VT _Src.RC:$src1),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
(_Src.VT _Src.RC:$src2)))>,
EVEX_CD8<_Src.EltSize, CD8VF>, EVEX_4V;
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
let mayLoad = 1 in {
defm rm : AVX512_maskable<opc, MRMSrcMem, _Dst, (outs _Dst.RC:$dst),
(ins _Src.RC:$src1, _Src.MemOp:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(_Dst.VT (OpNode (_Src.VT _Src.RC:$src1),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
(bitconvert (_Src.LdFrag addr:$src2))))>,
EVEX_4V, EVEX_CD8<_Src.EltSize, CD8VF>;
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
}
}
multiclass avx512_packs_all_i32_i16<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
[AVX512] Add HasVLX to the 128/256-bit versions of VPACKSSDW/USDW/SSWB/USWB and VPMADDUBSW/VPMADDWD. llvm-svn: 268188
2016-05-01 14:24:57 +08:00
let Predicates = [HasBWI] in
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
defm NAME#Z : avx512_packs_rm<opc, OpcodeStr, OpNode, v16i32_info,
v32i16_info>,
avx512_packs_rmb<opc, OpcodeStr, OpNode, v16i32_info,
v32i16_info>, EVEX_V512;
[AVX512] Add HasVLX to the 128/256-bit versions of VPACKSSDW/USDW/SSWB/USWB and VPMADDUBSW/VPMADDWD. llvm-svn: 268188
2016-05-01 14:24:57 +08:00
let Predicates = [HasBWI, HasVLX] in {
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
defm NAME#Z256 : avx512_packs_rm<opc, OpcodeStr, OpNode, v8i32x_info,
v16i16x_info>,
avx512_packs_rmb<opc, OpcodeStr, OpNode, v8i32x_info,
v16i16x_info>, EVEX_V256;
defm NAME#Z128 : avx512_packs_rm<opc, OpcodeStr, OpNode, v4i32x_info,
v8i16x_info>,
avx512_packs_rmb<opc, OpcodeStr, OpNode, v4i32x_info,
v8i16x_info>, EVEX_V128;
}
}
multiclass avx512_packs_all_i16_i8<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
[AVX512] Add HasVLX to the 128/256-bit versions of VPACKSSDW/USDW/SSWB/USWB and VPMADDUBSW/VPMADDWD. llvm-svn: 268188
2016-05-01 14:24:57 +08:00
let Predicates = [HasBWI] in
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
defm NAME#Z : avx512_packs_rm<opc, OpcodeStr, OpNode, v32i16_info,
v64i8_info>, EVEX_V512;
[AVX512] Add HasVLX to the 128/256-bit versions of VPACKSSDW/USDW/SSWB/USWB and VPMADDUBSW/VPMADDWD. llvm-svn: 268188
2016-05-01 14:24:57 +08:00
let Predicates = [HasBWI, HasVLX] in {
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
defm NAME#Z256 : avx512_packs_rm<opc, OpcodeStr, OpNode, v16i16x_info,
v32i8x_info>, EVEX_V256;
defm NAME#Z128 : avx512_packs_rm<opc, OpcodeStr, OpNode, v8i16x_info,
v16i8x_info>, EVEX_V128;
}
}
AVX512 : Implemented VPMADDUBSW and VPMADDWD instruction , Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11351 llvm-svn: 242761
2015-07-21 15:11:28 +08:00
multiclass avx512_vpmadd<bits<8> opc, string OpcodeStr,
SDNode OpNode, AVX512VLVectorVTInfo _Src,
AVX512VLVectorVTInfo _Dst> {
[AVX512] Add HasVLX to the 128/256-bit versions of VPACKSSDW/USDW/SSWB/USWB and VPMADDUBSW/VPMADDWD. llvm-svn: 268188
2016-05-01 14:24:57 +08:00
let Predicates = [HasBWI] in
AVX512 : Implemented VPMADDUBSW and VPMADDWD instruction , Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11351 llvm-svn: 242761
2015-07-21 15:11:28 +08:00
defm NAME#Z : avx512_packs_rm<opc, OpcodeStr, OpNode, _Src.info512,
_Dst.info512>, EVEX_V512;
[AVX512] Add HasVLX to the 128/256-bit versions of VPACKSSDW/USDW/SSWB/USWB and VPMADDUBSW/VPMADDWD. llvm-svn: 268188
2016-05-01 14:24:57 +08:00
let Predicates = [HasBWI, HasVLX] in {
AVX512 : Implemented VPMADDUBSW and VPMADDWD instruction , Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11351 llvm-svn: 242761
2015-07-21 15:11:28 +08:00
defm NAME#Z256 : avx512_packs_rm<opc, OpcodeStr, OpNode, _Src.info256,
_Dst.info256>, EVEX_V256;
defm NAME#Z128 : avx512_packs_rm<opc, OpcodeStr, OpNode, _Src.info128,
_Dst.info128>, EVEX_V128;
}
}
[AVX512] VPACKUSWB/VPACKSSWB should not be encoded with EVEX.W=1. While there fix the execution domain for VPACKSSDW/VPACKUSDW. llvm-svn: 268200
2016-05-02 01:38:32 +08:00
defm VPACKSSDW : avx512_packs_all_i32_i16<0x6B, "vpackssdw", X86Packss>, AVX512BIBase;
defm VPACKUSDW : avx512_packs_all_i32_i16<0x2b, "vpackusdw", X86Packus>, AVX5128IBase;
defm VPACKSSWB : avx512_packs_all_i16_i8 <0x63, "vpacksswb", X86Packss>, AVX512BIBase;
defm VPACKUSWB : avx512_packs_all_i16_i8 <0x67, "vpackuswb", X86Packus>, AVX512BIBase;
AVX512 : Implemented VPMADDUBSW and VPMADDWD instruction , Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11351 llvm-svn: 242761
2015-07-21 15:11:28 +08:00
[AVX512] Add HasVLX to the 128/256-bit versions of VPACKSSDW/USDW/SSWB/USWB and VPMADDUBSW/VPMADDWD. llvm-svn: 268188
2016-05-01 14:24:57 +08:00
defm VPMADDUBSW : avx512_vpmadd<0x04, "vpmaddubsw", X86vpmaddubsw,
avx512vl_i8_info, avx512vl_i16_info>, AVX512BIBase, T8PD;
defm VPMADDWD : avx512_vpmadd<0xF5, "vpmaddwd", X86vpmaddwd,
avx512vl_i16_info, avx512vl_i32_info>, AVX512BIBase;
AVX-512: Added VPACK* instructions forms for KNL and SKX and their intrinsics by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236414
2015-05-04 17:14:02 +08:00
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMAXSB : avx512_binop_rm_vl_b<0x3C, "vpmaxsb", smax,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>, T8PD;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMAXSW : avx512_binop_rm_vl_w<0xEE, "vpmaxsw", smax,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>;
[X86][SSE] Use the general SMAX/SMIN/UMAX/UMIN opcodes and remove the X86 implementation With the completion of D9746 there is now a common implementation of integer signed/unsigned min/max nodes, removing the need for the equivalent X86 specific implementations. This patch removes the old X86ISD nodes, legalizes the relevant SSE2/SSE41/AVX2/AVX512 instructions for the ISD versions and converts the small amount of existing X86 code. Differential Revision: http://reviews.llvm.org/D10947 llvm-svn: 241506
2015-07-07 04:30:47 +08:00
defm VPMAXS : avx512_binop_rm_vl_dq<0x3D, 0x3D, "vpmaxs", smax,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMAXUB : avx512_binop_rm_vl_b<0xDE, "vpmaxub", umax,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMAXUW : avx512_binop_rm_vl_w<0x3E, "vpmaxuw", umax,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>, T8PD;
[X86][SSE] Use the general SMAX/SMIN/UMAX/UMIN opcodes and remove the X86 implementation With the completion of D9746 there is now a common implementation of integer signed/unsigned min/max nodes, removing the need for the equivalent X86 specific implementations. This patch removes the old X86ISD nodes, legalizes the relevant SSE2/SSE41/AVX2/AVX512 instructions for the ISD versions and converts the small amount of existing X86 code. Differential Revision: http://reviews.llvm.org/D10947 llvm-svn: 241506
2015-07-07 04:30:47 +08:00
defm VPMAXU : avx512_binop_rm_vl_dq<0x3F, 0x3F, "vpmaxu", umax,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMINSB : avx512_binop_rm_vl_b<0x38, "vpminsb", smin,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>, T8PD;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMINSW : avx512_binop_rm_vl_w<0xEA, "vpminsw", smin,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>;
[X86][SSE] Use the general SMAX/SMIN/UMAX/UMIN opcodes and remove the X86 implementation With the completion of D9746 there is now a common implementation of integer signed/unsigned min/max nodes, removing the need for the equivalent X86 specific implementations. This patch removes the old X86ISD nodes, legalizes the relevant SSE2/SSE41/AVX2/AVX512 instructions for the ISD versions and converts the small amount of existing X86 code. Differential Revision: http://reviews.llvm.org/D10947 llvm-svn: 241506
2015-07-07 04:30:47 +08:00
defm VPMINS : avx512_binop_rm_vl_dq<0x39, 0x39, "vpmins", smin,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMINUB : avx512_binop_rm_vl_b<0xDA, "vpminub", umin,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPMINUW : avx512_binop_rm_vl_w<0x3A, "vpminuw", umin,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasBWI, 1>, T8PD;
[X86][SSE] Use the general SMAX/SMIN/UMAX/UMIN opcodes and remove the X86 implementation With the completion of D9746 there is now a common implementation of integer signed/unsigned min/max nodes, removing the need for the equivalent X86 specific implementations. This patch removes the old X86ISD nodes, legalizes the relevant SSE2/SSE41/AVX2/AVX512 instructions for the ISD versions and converts the small amount of existing X86 code. Differential Revision: http://reviews.llvm.org/D10947 llvm-svn: 241506
2015-07-07 04:30:47 +08:00
defm VPMINU : avx512_binop_rm_vl_dq<0x3B, 0x3B, "vpminu", umin,
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Logical Instructions
//===----------------------------------------------------------------------===//
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
defm VPAND : avx512_binop_rm_vl_dq<0xDB, 0xDB, "vpand", and,
SSE_INTALU_ITINS_P, HasAVX512, 1>;
defm VPOR : avx512_binop_rm_vl_dq<0xEB, 0xEB, "vpor", or,
SSE_INTALU_ITINS_P, HasAVX512, 1>;
defm VPXOR : avx512_binop_rm_vl_dq<0xEF, 0xEF, "vpxor", xor,
SSE_INTALU_ITINS_P, HasAVX512, 1>;
defm VPANDN : avx512_binop_rm_vl_dq<0xDF, 0xDF, "vpandn", X86andnp,
AVX-512: Fixed the "commutative" property flag in VPANDN instruction By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 233489
2015-03-29 17:14:29 +08:00
SSE_INTALU_ITINS_P, HasAVX512, 0>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 FP arithmetic
//===----------------------------------------------------------------------===//
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
multiclass avx512_fp_scalar<bits<8> opc, string OpcodeStr,X86VectorVTInfo _,
SDNode OpNode, SDNode VecNode, OpndItins itins,
bit IsCommutable> {
defm rr_Int : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(VecNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
(i32 FROUND_CURRENT)),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
itins.rr, IsCommutable>;
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
defm rm_Int : AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr,
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
"$src2, $src1", "$src1, $src2",
(VecNode (_.VT _.RC:$src1),
(_.VT (scalar_to_vector (_.ScalarLdFrag addr:$src2))),
(i32 FROUND_CURRENT)),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
itins.rm, IsCommutable>;
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
let isCodeGenOnly = 1, isCommutable = IsCommutable,
Predicates = [HasAVX512] in {
def rr : I< opc, MRMSrcReg, (outs _.FRC:$dst),
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(ins _.FRC:$src1, _.FRC:$src2),
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[(set _.FRC:$dst, (OpNode _.FRC:$src1, _.FRC:$src2))],
itins.rr>;
def rm : I< opc, MRMSrcMem, (outs _.FRC:$dst),
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(ins _.FRC:$src1, _.ScalarMemOp:$src2),
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[(set _.FRC:$dst, (OpNode _.FRC:$src1,
(_.ScalarLdFrag addr:$src2)))], itins.rr>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
multiclass avx512_fp_scalar_round<bits<8> opc, string OpcodeStr,X86VectorVTInfo _,
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
SDNode VecNode, OpndItins itins, bit IsCommutable = 0> {
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
defm rrb : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2, AVX512RC:$rc), OpcodeStr,
"$rc, $src2, $src1", "$src1, $src2, $rc",
(VecNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
(i32 imm:$rc)), itins.rr, IsCommutable>,
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
EVEX_B, EVEX_RC;
}
multiclass avx512_fp_scalar_sae<bits<8> opc, string OpcodeStr,X86VectorVTInfo _,
SDNode VecNode, OpndItins itins, bit IsCommutable> {
defm rrb : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
"{sae}, $src2, $src1", "$src1, $src2, {sae}",
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
(VecNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
(i32 FROUND_NO_EXC))>, EVEX_B;
AVX-512: Added mask and rounding mode for scalar arithmetics Added more tests for scalar instructions to destinguish between AVX and AVX-512 forms. llvm-svn: 230891
2015-03-01 15:44:04 +08:00
}
multiclass avx512_binop_s_round<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode VecNode,
SizeItins itins, bit IsCommutable> {
defm SSZ : avx512_fp_scalar<opc, OpcodeStr#"ss", f32x_info, OpNode, VecNode,
itins.s, IsCommutable>,
avx512_fp_scalar_round<opc, OpcodeStr#"ss", f32x_info, VecNode,
itins.s, IsCommutable>,
XS, EVEX_4V, VEX_LIG, EVEX_CD8<32, CD8VT1>;
defm SDZ : avx512_fp_scalar<opc, OpcodeStr#"sd", f64x_info, OpNode, VecNode,
itins.d, IsCommutable>,
avx512_fp_scalar_round<opc, OpcodeStr#"sd", f64x_info, VecNode,
itins.d, IsCommutable>,
XD, VEX_W, EVEX_4V, VEX_LIG, EVEX_CD8<64, CD8VT1>;
}
multiclass avx512_binop_s_sae<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode VecNode,
SizeItins itins, bit IsCommutable> {
defm SSZ : avx512_fp_scalar<opc, OpcodeStr#"ss", f32x_info, OpNode, VecNode,
itins.s, IsCommutable>,
avx512_fp_scalar_sae<opc, OpcodeStr#"ss", f32x_info, VecNode,
itins.s, IsCommutable>,
XS, EVEX_4V, VEX_LIG, EVEX_CD8<32, CD8VT1>;
defm SDZ : avx512_fp_scalar<opc, OpcodeStr#"sd", f64x_info, OpNode, VecNode,
itins.d, IsCommutable>,
avx512_fp_scalar_sae<opc, OpcodeStr#"sd", f64x_info, VecNode,
itins.d, IsCommutable>,
XD, VEX_W, EVEX_4V, VEX_LIG, EVEX_CD8<64, CD8VT1>;
}
defm VADD : avx512_binop_s_round<0x58, "vadd", fadd, X86faddRnd, SSE_ALU_ITINS_S, 1>;
defm VMUL : avx512_binop_s_round<0x59, "vmul", fmul, X86fmulRnd, SSE_ALU_ITINS_S, 1>;
defm VSUB : avx512_binop_s_round<0x5C, "vsub", fsub, X86fsubRnd, SSE_ALU_ITINS_S, 0>;
defm VDIV : avx512_binop_s_round<0x5E, "vdiv", fdiv, X86fdivRnd, SSE_ALU_ITINS_S, 0>;
defm VMIN : avx512_binop_s_sae <0x5D, "vmin", X86fmin, X86fminRnd, SSE_ALU_ITINS_S, 1>;
defm VMAX : avx512_binop_s_sae <0x5F, "vmax", X86fmax, X86fmaxRnd, SSE_ALU_ITINS_S, 1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_fp_packed<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Implemented AVX512VL FP bnary packed instructions (VADDP*, VSUBP*, VMULP*, VDIVP*, VMAXP*, VMINP*) Refactored through AVX512_maskable Added encoding tests for them. llvm-svn: 220858
2014-10-29 23:43:02 +08:00
X86VectorVTInfo _, bit IsCommutable> {
defm rr: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, _.RC:$src2))>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in {
[AVX512] Implemented AVX512VL FP bnary packed instructions (VADDP*, VSUBP*, VMULP*, VDIVP*, VMAXP*, VMINP*) Refactored through AVX512_maskable Added encoding tests for them. llvm-svn: 220858
2014-10-29 23:43:02 +08:00
defm rm: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(OpNode _.RC:$src1, (_.LdFrag addr:$src2))>, EVEX_4V;
defm rmb: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr##_.Suffix,
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(OpNode _.RC:$src1, (_.VT (X86VBroadcast
(_.ScalarLdFrag addr:$src2))))>,
EVEX_4V, EVEX_B;
}//let mayLoad = 1
}
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
multiclass avx512_fp_round_packed<bits<8> opc, string OpcodeStr, SDNode OpNodeRnd,
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
X86VectorVTInfo _> {
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
defm rb: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2, AVX512RC:$rc), OpcodeStr##_.Suffix,
"$rc, $src2, $src1", "$src1, $src2, $rc",
(_.VT (OpNodeRnd _.RC:$src1, _.RC:$src2, (i32 imm:$rc)))>,
EVEX_4V, EVEX_B, EVEX_RC;
}
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
multiclass avx512_fp_sae_packed<bits<8> opc, string OpcodeStr, SDNode OpNodeRnd,
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
X86VectorVTInfo _> {
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
defm rb: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr##_.Suffix,
"{sae}, $src2, $src1", "$src1, $src2, {sae}",
(_.VT (OpNodeRnd _.RC:$src1, _.RC:$src2, (i32 FROUND_NO_EXC)))>,
EVEX_4V, EVEX_B;
}
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
multiclass avx512_fp_binop_p<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
Predicate prd, bit IsCommutable = 0> {
let Predicates = [prd] in {
[AVX512] Implemented AVX512VL FP bnary packed instructions (VADDP*, VSUBP*, VMULP*, VDIVP*, VMAXP*, VMINP*) Refactored through AVX512_maskable Added encoding tests for them. llvm-svn: 220858
2014-10-29 23:43:02 +08:00
defm PSZ : avx512_fp_packed<opc, OpcodeStr, OpNode, v16f32_info,
IsCommutable>, EVEX_V512, PS,
EVEX_CD8<32, CD8VF>;
defm PDZ : avx512_fp_packed<opc, OpcodeStr, OpNode, v8f64_info,
IsCommutable>, EVEX_V512, PD, VEX_W,
EVEX_CD8<64, CD8VF>;
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
}
[AVX512] Implemented AVX512VL FP bnary packed instructions (VADDP*, VSUBP*, VMULP*, VDIVP*, VMAXP*, VMINP*) Refactored through AVX512_maskable Added encoding tests for them. llvm-svn: 220858
2014-10-29 23:43:02 +08:00
// Define only if AVX512VL feature is present.
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
let Predicates = [prd, HasVLX] in {
[AVX512] Implemented AVX512VL FP bnary packed instructions (VADDP*, VSUBP*, VMULP*, VDIVP*, VMAXP*, VMINP*) Refactored through AVX512_maskable Added encoding tests for them. llvm-svn: 220858
2014-10-29 23:43:02 +08:00
defm PSZ128 : avx512_fp_packed<opc, OpcodeStr, OpNode, v4f32x_info,
IsCommutable>, EVEX_V128, PS,
EVEX_CD8<32, CD8VF>;
defm PSZ256 : avx512_fp_packed<opc, OpcodeStr, OpNode, v8f32x_info,
IsCommutable>, EVEX_V256, PS,
EVEX_CD8<32, CD8VF>;
defm PDZ128 : avx512_fp_packed<opc, OpcodeStr, OpNode, v2f64x_info,
IsCommutable>, EVEX_V128, PD, VEX_W,
EVEX_CD8<64, CD8VF>;
defm PDZ256 : avx512_fp_packed<opc, OpcodeStr, OpNode, v4f64x_info,
IsCommutable>, EVEX_V256, PD, VEX_W,
EVEX_CD8<64, CD8VF>;
AVX-512: Added rrk, rrkz, rmk, rmkz, rmbk, rmbkz versions of AVX512 FP packed instructions, added encoding tests for them. By Robert Khazanov. llvm-svn: 203098
2014-03-06 16:45:30 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
multiclass avx512_fp_binop_p_round<bits<8> opc, string OpcodeStr, SDNode OpNodeRnd> {
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
defm PSZ : avx512_fp_round_packed<opc, OpcodeStr, OpNodeRnd, v16f32_info>,
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
EVEX_V512, PS, EVEX_CD8<32, CD8VF>;
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
defm PDZ : avx512_fp_round_packed<opc, OpcodeStr, OpNodeRnd, v8f64_info>,
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
EVEX_V512, PD, VEX_W,EVEX_CD8<64, CD8VF>;
}
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
multiclass avx512_fp_binop_p_sae<bits<8> opc, string OpcodeStr, SDNode OpNodeRnd> {
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
defm PSZ : avx512_fp_sae_packed<opc, OpcodeStr, OpNodeRnd, v16f32_info>,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
EVEX_V512, PS, EVEX_CD8<32, CD8VF>;
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
defm PDZ : avx512_fp_sae_packed<opc, OpcodeStr, OpNodeRnd, v8f64_info>,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
EVEX_V512, PD, VEX_W,EVEX_CD8<64, CD8VF>;
}
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
defm VADD : avx512_fp_binop_p<0x58, "vadd", fadd, HasAVX512, 1>,
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
avx512_fp_binop_p_round<0x58, "vadd", X86faddRnd>;
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
defm VMUL : avx512_fp_binop_p<0x59, "vmul", fmul, HasAVX512, 1>,
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
avx512_fp_binop_p_round<0x59, "vmul", X86fmulRnd>;
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
defm VSUB : avx512_fp_binop_p<0x5C, "vsub", fsub, HasAVX512>,
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
avx512_fp_binop_p_round<0x5C, "vsub", X86fsubRnd>;
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
defm VDIV : avx512_fp_binop_p<0x5E, "vdiv", fdiv, HasAVX512>,
AVX-512: Added support for FP instructions with embedded rounding mode. By Asaf Badouh <asaf.badouh@intel.com> llvm-svn: 229645
2015-02-18 15:59:20 +08:00
avx512_fp_binop_p_round<0x5E, "vdiv", X86fdivRnd>;
[AVX512] Add support for commutative MAX/MIN . In general VMAX{PS,PD} and VMIN{PS,PD} instruction are not commutative . In combine pass only if UnsafeFPMath are used VMAX/VMAX are converted to commutative nodes VMAXC/VMAXC. Differential Revision: http://reviews.llvm.org/D19860 llvm-svn: 268375
2016-05-03 19:51:45 +08:00
defm VMIN : avx512_fp_binop_p<0x5D, "vmin", X86fmin, HasAVX512, 0>,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
avx512_fp_binop_p_sae<0x5D, "vmin", X86fminRnd>;
[AVX512] Add support for commutative MAX/MIN . In general VMAX{PS,PD} and VMIN{PS,PD} instruction are not commutative . In combine pass only if UnsafeFPMath are used VMAX/VMAX are converted to commutative nodes VMAXC/VMAXC. Differential Revision: http://reviews.llvm.org/D19860 llvm-svn: 268375
2016-05-03 19:51:45 +08:00
defm VMAX : avx512_fp_binop_p<0x5F, "vmax", X86fmax, HasAVX512, 0>,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
avx512_fp_binop_p_sae<0x5F, "vmax", X86fmaxRnd>;
[AVX512] Add support for commutative MAX/MIN . In general VMAX{PS,PD} and VMIN{PS,PD} instruction are not commutative . In combine pass only if UnsafeFPMath are used VMAX/VMAX are converted to commutative nodes VMAXC/VMAXC. Differential Revision: http://reviews.llvm.org/D19860 llvm-svn: 268375
2016-05-03 19:51:45 +08:00
let isCodeGenOnly = 1 in {
defm VMINC : avx512_fp_binop_p<0x5D, "vmin", X86fminc, HasAVX512, 1>;
defm VMAXC : avx512_fp_binop_p<0x5F, "vmax", X86fmaxc, HasAVX512, 1>;
}
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
defm VAND : avx512_fp_binop_p<0x54, "vand", X86fand, HasDQI, 1>;
defm VANDN : avx512_fp_binop_p<0x55, "vandn", X86fandn, HasDQI, 0>;
defm VOR : avx512_fp_binop_p<0x56, "vor", X86for, HasDQI, 1>;
defm VXOR : avx512_fp_binop_p<0x57, "vxor", X86fxor, HasDQI, 1>;
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
multiclass avx512_fp_scalef_p<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm rr: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, _.RC:$src2, (i32 FROUND_CURRENT)))>, EVEX_4V;
let mayLoad = 1 in {
defm rm: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(OpNode _.RC:$src1, (_.LdFrag addr:$src2), (i32 FROUND_CURRENT))>, EVEX_4V;
defm rmb: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr##_.Suffix,
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(OpNode _.RC:$src1, (_.VT (X86VBroadcast
(_.ScalarLdFrag addr:$src2))), (i32 FROUND_CURRENT))>,
EVEX_4V, EVEX_B;
}//let mayLoad = 1
}
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
multiclass avx512_fp_scalef_scalar<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm rr: AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, _.RC:$src2, (i32 FROUND_CURRENT)))>;
let mayLoad = 1 in {
defm rm: AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr##_.Suffix,
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
"$src2, $src1", "$src1, $src2",
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(OpNode _.RC:$src1,
(_.VT (scalar_to_vector (_.ScalarLdFrag addr:$src2))),
(i32 FROUND_CURRENT))>;
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
}//let mayLoad = 1
}
multiclass avx512_fp_scalef_all<bits<8> opc, bits<8> opcScaler, string OpcodeStr, SDNode OpNode> {
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
defm PSZ : avx512_fp_scalef_p<opc, OpcodeStr, OpNode, v16f32_info>,
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
avx512_fp_round_packed<opc, OpcodeStr, OpNode, v16f32_info>,
EVEX_V512, EVEX_CD8<32, CD8VF>;
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
defm PDZ : avx512_fp_scalef_p<opc, OpcodeStr, OpNode, v8f64_info>,
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
avx512_fp_round_packed<opc, OpcodeStr, OpNode, v8f64_info>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
defm SSZ128 : avx512_fp_scalef_scalar<opcScaler, OpcodeStr, OpNode, f32x_info>,
avx512_fp_scalar_round<opcScaler, OpcodeStr##"ss", f32x_info, OpNode, SSE_ALU_ITINS_S.s>,
EVEX_4V,EVEX_CD8<32, CD8VT1>;
defm SDZ128 : avx512_fp_scalef_scalar<opcScaler, OpcodeStr, OpNode, f64x_info>,
avx512_fp_scalar_round<opcScaler, OpcodeStr##"sd", f64x_info, OpNode, SSE_ALU_ITINS_S.d>,
EVEX_4V, EVEX_CD8<64, CD8VT1>, VEX_W;
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
// Define only if AVX512VL feature is present.
let Predicates = [HasVLX] in {
defm PSZ128 : avx512_fp_scalef_p<opc, OpcodeStr, OpNode, v4f32x_info>,
EVEX_V128, EVEX_CD8<32, CD8VF>;
defm PSZ256 : avx512_fp_scalef_p<opc, OpcodeStr, OpNode, v8f32x_info>,
EVEX_V256, EVEX_CD8<32, CD8VF>;
defm PDZ128 : avx512_fp_scalef_p<opc, OpcodeStr, OpNode, v2f64x_info>,
EVEX_V128, VEX_W, EVEX_CD8<64, CD8VF>;
defm PDZ256 : avx512_fp_scalef_p<opc, OpcodeStr, OpNode, v4f64x_info>,
EVEX_V256, VEX_W, EVEX_CD8<64, CD8VF>;
}
}
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
defm VSCALEF : avx512_fp_scalef_all<0x2C, 0x2D, "vscalef", X86scalef>, T8PD;
[x86][AVX512] Add vscalef support include encoding and intrinsics review: http://reviews.llvm.org/D10730 llvm-svn: 240906
2015-06-28 22:30:39 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 VPTESTM instructions
//===----------------------------------------------------------------------===//
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
multiclass avx512_vptest<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
AVX512: Combine AND + TESTM instructions . Differential Revision: http://reviews.llvm.org/D17844 llvm-svn: 262621
2016-03-03 22:18:38 +08:00
let isCommutable = 1 in
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
defm rr : AVX512_maskable_cmp<opc, MRMSrcReg, _, (outs _.KRC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2))>,
EVEX_4V;
let mayLoad = 1 in
defm rm : AVX512_maskable_cmp<opc, MRMSrcMem, _, (outs _.KRC:$dst),
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(OpNode (_.VT _.RC:$src1),
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
(_.VT (bitconvert (_.LdFrag addr:$src2))))>,
EVEX_4V,
EVEX_CD8<_.EltSize, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
multiclass avx512_vptest_mb<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
let mayLoad = 1 in
defm rmb : AVX512_maskable_cmp<opc, MRMSrcMem, _, (outs _.KRC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr,
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(OpNode (_.VT _.RC:$src1), (_.VT (X86VBroadcast
(_.ScalarLdFrag addr:$src2))))>,
EVEX_B, EVEX_4V, EVEX_CD8<_.EltSize, CD8VF>;
}
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
// Use 512bit version to implement 128/256 bit in case NoVLX.
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
multiclass avx512_vptest_lowering<SDNode OpNode, X86VectorVTInfo ExtendInfo,
X86VectorVTInfo _, string Suffix> {
def : Pat<(_.KVT (OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2))),
(_.KVT (COPY_TO_REGCLASS
(!cast<Instruction>(NAME # Suffix # "Zrr")
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
_.RC:$src1, _.SubRegIdx),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
_.RC:$src2, _.SubRegIdx)),
_.KRC))>;
}
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
multiclass avx512_vptest_dq_sizes<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
AVX512VLVectorVTInfo _, string Suffix> {
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
let Predicates = [HasAVX512] in
defm Z : avx512_vptest<opc, OpcodeStr, OpNode, _.info512>,
avx512_vptest_mb<opc, OpcodeStr, OpNode, _.info512>, EVEX_V512;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
let Predicates = [HasAVX512, HasVLX] in {
defm Z256 : avx512_vptest<opc, OpcodeStr, OpNode, _.info256>,
avx512_vptest_mb<opc, OpcodeStr, OpNode, _.info256>, EVEX_V256;
defm Z128 : avx512_vptest<opc, OpcodeStr, OpNode, _.info128>,
avx512_vptest_mb<opc, OpcodeStr, OpNode, _.info128>, EVEX_V128;
}
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
let Predicates = [HasAVX512, NoVLX] in {
defm Z256_Alt : avx512_vptest_lowering< OpNode, _.info512, _.info256, Suffix>;
defm Z128_Alt : avx512_vptest_lowering< OpNode, _.info512, _.info128, Suffix>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
}
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
}
multiclass avx512_vptest_dq<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm D : avx512_vptest_dq_sizes<opc, OpcodeStr#"d", OpNode,
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
avx512vl_i32_info, "D">;
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
defm Q : avx512_vptest_dq_sizes<opc, OpcodeStr#"q", OpNode,
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
avx512vl_i64_info, "Q">, VEX_W;
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
}
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
multiclass avx512_vptest_wb<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
let Predicates = [HasBWI] in {
defm WZ: avx512_vptest<opc, OpcodeStr#"w", OpNode, v32i16_info>,
EVEX_V512, VEX_W;
defm BZ: avx512_vptest<opc, OpcodeStr#"b", OpNode, v64i8_info>,
EVEX_V512;
}
let Predicates = [HasVLX, HasBWI] in {
defm WZ256: avx512_vptest<opc, OpcodeStr#"w", OpNode, v16i16x_info>,
EVEX_V256, VEX_W;
defm WZ128: avx512_vptest<opc, OpcodeStr#"w", OpNode, v8i16x_info>,
EVEX_V128, VEX_W;
defm BZ256: avx512_vptest<opc, OpcodeStr#"b", OpNode, v32i8x_info>,
EVEX_V256;
defm BZ128: avx512_vptest<opc, OpcodeStr#"b", OpNode, v16i8x_info>,
EVEX_V128;
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
let Predicates = [HasAVX512, NoVLX] in {
defm BZ256_Alt : avx512_vptest_lowering< OpNode, v64i8_info, v32i8x_info, "B">;
defm BZ128_Alt : avx512_vptest_lowering< OpNode, v64i8_info, v16i8x_info, "B">;
defm WZ256_Alt : avx512_vptest_lowering< OpNode, v32i16_info, v16i16x_info, "W">;
defm WZ128_Alt : avx512_vptest_lowering< OpNode, v32i16_info, v8i16x_info, "W">;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
}
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
AVX-512: Added VPTESTM and VPTESTNM instructions for SKX llvm-svn: 235383
2015-04-21 21:13:46 +08:00
}
multiclass avx512_vptest_all_forms<bits<8> opc_wb, bits<8> opc_dq, string OpcodeStr,
SDNode OpNode> :
avx512_vptest_wb <opc_wb, OpcodeStr, OpNode>,
avx512_vptest_dq<opc_dq, OpcodeStr, OpNode>;
defm VPTESTM : avx512_vptest_all_forms<0x26, 0x27, "vptestm", X86testm>, T8PD;
defm VPTESTNM : avx512_vptest_all_forms<0x26, 0x27, "vptestnm", X86testnm>, T8XS;
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Shift instructions
//===----------------------------------------------------------------------===//
multiclass avx512_shift_rmi<bits<8> opc, Format ImmFormR, Format ImmFormM,
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
string OpcodeStr, SDNode OpNode, X86VectorVTInfo _> {
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
defm ri : AVX512_maskable<opc, ImmFormR, _, (outs _.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins _.RC:$src1, u8imm:$src2), OpcodeStr,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, (i8 imm:$src2))),
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
SSE_INTSHIFT_ITINS_P.rr>;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
let mayLoad = 1 in
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
defm mi : AVX512_maskable<opc, ImmFormM, _, (outs _.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins _.MemOp:$src1, u8imm:$src2), OpcodeStr,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
"$src2, $src1", "$src1, $src2",
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
(_.VT (OpNode (_.VT (bitconvert (_.LdFrag addr:$src1))),
(i8 imm:$src2))),
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
SSE_INTSHIFT_ITINS_P.rm>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
multiclass avx512_shift_rmbi<bits<8> opc, Format ImmFormM,
string OpcodeStr, SDNode OpNode, X86VectorVTInfo _> {
let mayLoad = 1 in
defm mbi : AVX512_maskable<opc, ImmFormM, _, (outs _.RC:$dst),
(ins _.ScalarMemOp:$src1, u8imm:$src2), OpcodeStr,
"$src2, ${src1}"##_.BroadcastStr, "${src1}"##_.BroadcastStr##", $src2",
(_.VT (OpNode (X86VBroadcast (_.ScalarLdFrag addr:$src1)), (i8 imm:$src2))),
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
SSE_INTSHIFT_ITINS_P.rm>, EVEX_B;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_shift_rrm<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
ValueType SrcVT, PatFrag bc_frag, X86VectorVTInfo _> {
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
// src2 is always 128-bit
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, VR128X:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, (SrcVT VR128X:$src2))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
SSE_INTSHIFT_ITINS_P.rr>, AVX512BIBase, EVEX_4V;
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, i128mem:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(_.VT (OpNode _.RC:$src1, (bc_frag (loadv2i64 addr:$src2)))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
SSE_INTSHIFT_ITINS_P.rm>, AVX512BIBase,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
EVEX_4V;
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
}
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
multiclass avx512_shift_sizes<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
ValueType SrcVT, PatFrag bc_frag,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_shift_rrm<opc, OpcodeStr, OpNode, SrcVT, bc_frag,
VTInfo.info512>, EVEX_V512,
EVEX_CD8<VTInfo.info512.EltSize, CD8VQ> ;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_shift_rrm<opc, OpcodeStr, OpNode, SrcVT, bc_frag,
VTInfo.info256>, EVEX_V256,
EVEX_CD8<VTInfo.info256.EltSize, CD8VH>;
defm Z128 : avx512_shift_rrm<opc, OpcodeStr, OpNode, SrcVT, bc_frag,
VTInfo.info128>, EVEX_V128,
EVEX_CD8<VTInfo.info128.EltSize, CD8VF>;
}
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
}
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
multiclass avx512_shift_types<bits<8> opcd, bits<8> opcq, bits<8> opcw,
string OpcodeStr, SDNode OpNode> {
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
defm D : avx512_shift_sizes<opcd, OpcodeStr#"d", OpNode, v4i32, bc_v4i32,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
avx512vl_i32_info, HasAVX512>;
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
defm Q : avx512_shift_sizes<opcq, OpcodeStr#"q", OpNode, v2i64, bc_v2i64,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
avx512vl_i64_info, HasAVX512>, VEX_W;
defm W : avx512_shift_sizes<opcw, OpcodeStr#"w", OpNode, v8i16, bc_v8i16,
avx512vl_i16_info, HasBWI>;
}
multiclass avx512_shift_rmi_sizes<bits<8> opc, Format ImmFormR, Format ImmFormM,
string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo> {
let Predicates = [HasAVX512] in
defm Z: avx512_shift_rmi<opc, ImmFormR, ImmFormM, OpcodeStr, OpNode,
VTInfo.info512>,
avx512_shift_rmbi<opc, ImmFormM, OpcodeStr, OpNode,
VTInfo.info512>, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in {
defm Z256: avx512_shift_rmi<opc, ImmFormR, ImmFormM, OpcodeStr, OpNode,
VTInfo.info256>,
avx512_shift_rmbi<opc, ImmFormM, OpcodeStr, OpNode,
VTInfo.info256>, EVEX_V256;
defm Z128: avx512_shift_rmi<opc, ImmFormR, ImmFormM, OpcodeStr, OpNode,
VTInfo.info128>,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
avx512_shift_rmbi<opc, ImmFormM, OpcodeStr, OpNode,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
VTInfo.info128>, EVEX_V128;
}
}
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
multiclass avx512_shift_rmi_w<bits<8> opcw,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
Format ImmFormR, Format ImmFormM,
string OpcodeStr, SDNode OpNode> {
let Predicates = [HasBWI] in
defm WZ: avx512_shift_rmi<opcw, ImmFormR, ImmFormM, OpcodeStr, OpNode,
v32i16_info>, EVEX_V512;
let Predicates = [HasVLX, HasBWI] in {
defm WZ256: avx512_shift_rmi<opcw, ImmFormR, ImmFormM, OpcodeStr, OpNode,
v16i16x_info>, EVEX_V256;
defm WZ128: avx512_shift_rmi<opcw, ImmFormR, ImmFormM, OpcodeStr, OpNode,
v8i16x_info>, EVEX_V128;
}
}
multiclass avx512_shift_rmi_dq<bits<8> opcd, bits<8> opcq,
Format ImmFormR, Format ImmFormM,
string OpcodeStr, SDNode OpNode> {
defm D: avx512_shift_rmi_sizes<opcd, ImmFormR, ImmFormM, OpcodeStr#"d", OpNode,
avx512vl_i32_info>, EVEX_CD8<32, CD8VF>;
defm Q: avx512_shift_rmi_sizes<opcq, ImmFormR, ImmFormM, OpcodeStr#"q", OpNode,
avx512vl_i64_info>, EVEX_CD8<64, CD8VF>, VEX_W;
}
defm VPSRL : avx512_shift_rmi_dq<0x72, 0x73, MRM2r, MRM2m, "vpsrl", X86vsrli>,
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
avx512_shift_rmi_w<0x71, MRM2r, MRM2m, "vpsrlw", X86vsrli>, AVX512BIi8Base, EVEX_4V;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
defm VPSLL : avx512_shift_rmi_dq<0x72, 0x73, MRM6r, MRM6m, "vpsll", X86vshli>,
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
avx512_shift_rmi_w<0x71, MRM6r, MRM6m, "vpsllw", X86vshli>, AVX512BIi8Base, EVEX_4V;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
AVX-512: fixed a bug in encoding of VPSRAQ instrcution, added a bunch of encoding tests. llvm-svn: 237232
2015-05-13 15:35:05 +08:00
defm VPSRA : avx512_shift_rmi_dq<0x72, 0x72, MRM4r, MRM4m, "vpsra", X86vsrai>,
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
avx512_shift_rmi_w<0x71, MRM4r, MRM4m, "vpsraw", X86vsrai>, AVX512BIi8Base, EVEX_4V;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
[AVX512] adding PRORQ , PRORD , PRORLVQ and PRORLVD Intrinsics Differential Revision: http://reviews.llvm.org/D16052 llvm-svn: 257594
2016-01-13 20:39:33 +08:00
defm VPROR : avx512_shift_rmi_dq<0x72, 0x72, MRM0r, MRM0m, "vpror", X86vrotri>, AVX512BIi8Base, EVEX_4V;
[AVX512] adding PROLQ and PROLD Intrinsics Differential Revision: http://reviews.llvm.org/D16048 llvm-svn: 257523
2016-01-13 05:19:17 +08:00
defm VPROL : avx512_shift_rmi_dq<0x72, 0x72, MRM1r, MRM1m, "vprol", X86vrotli>, AVX512BIi8Base, EVEX_4V;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
defm VPSLL : avx512_shift_types<0xF2, 0xF3, 0xF1, "vpsll", X86vshl>;
defm VPSRA : avx512_shift_types<0xE2, 0xE2, 0xE1, "vpsra", X86vsra>;
defm VPSRL : avx512_shift_types<0xD2, 0xD3, 0xD1, "vpsrl", X86vsrl>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===-------------------------------------------------------------------===//
// Variable Bit Shifts
//===-------------------------------------------------------------------===//
multiclass avx512_var_shift<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
X86VectorVTInfo _> {
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, (_.VT _.RC:$src2))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
SSE_INTSHIFT_ITINS_P.rr>, AVX5128IBase, EVEX_4V;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
let mayLoad = 1 in
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
AVX-512: added all SKX forms of VPERMW/D/Q instructions. Added all forms of VPERMPS/PD instrcuctions. Added encoding tests. llvm-svn: 239016
2015-06-04 15:07:13 +08:00
(_.VT (OpNode _.RC:$src1,
(_.VT (bitconvert (_.LdFrag addr:$src2))))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
SSE_INTSHIFT_ITINS_P.rm>, AVX5128IBase, EVEX_4V,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
EVEX_CD8<_.EltSize, CD8VF>;
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
}
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
multiclass avx512_var_shift_mb<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
let mayLoad = 1 in
defm rmb : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr,
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(_.VT (OpNode _.RC:$src1, (_.VT (X86VBroadcast
(_.ScalarLdFrag addr:$src2))))),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
SSE_INTSHIFT_ITINS_P.rm>, AVX5128IBase, EVEX_B,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
EVEX_4V, EVEX_CD8<_.EltSize, CD8VF>;
}
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
multiclass avx512_var_shift_sizes<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo _> {
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
let Predicates = [HasAVX512] in
defm Z : avx512_var_shift<opc, OpcodeStr, OpNode, _.info512>,
avx512_var_shift_mb<opc, OpcodeStr, OpNode, _.info512>, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in {
defm Z256 : avx512_var_shift<opc, OpcodeStr, OpNode, _.info256>,
avx512_var_shift_mb<opc, OpcodeStr, OpNode, _.info256>, EVEX_V256;
defm Z128 : avx512_var_shift<opc, OpcodeStr, OpNode, _.info128>,
avx512_var_shift_mb<opc, OpcodeStr, OpNode, _.info128>, EVEX_V128;
}
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
}
multiclass avx512_var_shift_types<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
defm D : avx512_var_shift_sizes<opc, OpcodeStr#"d", OpNode,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
avx512vl_i32_info>;
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
defm Q : avx512_var_shift_sizes<opc, OpcodeStr#"q", OpNode,
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
avx512vl_i64_info>, VEX_W;
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
// Use 512bit version to implement 128/256 bit in case NoVLX.
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
multiclass avx512_var_shift_w_lowering<AVX512VLVectorVTInfo _, SDNode OpNode> {
let Predicates = [HasBWI, NoVLX] in {
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
def : Pat<(_.info256.VT (OpNode (_.info256.VT _.info256.RC:$src1),
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
(_.info256.VT _.info256.RC:$src2))),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(EXTRACT_SUBREG
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
(!cast<Instruction>(NAME#"WZrr")
(INSERT_SUBREG (_.info512.VT (IMPLICIT_DEF)), VR256X:$src1, sub_ymm),
(INSERT_SUBREG (_.info512.VT (IMPLICIT_DEF)), VR256X:$src2, sub_ymm)),
sub_ymm)>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
def : Pat<(_.info128.VT (OpNode (_.info128.VT _.info128.RC:$src1),
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
(_.info128.VT _.info128.RC:$src2))),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(EXTRACT_SUBREG
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
(!cast<Instruction>(NAME#"WZrr")
(INSERT_SUBREG (_.info512.VT (IMPLICIT_DEF)), VR128X:$src1, sub_xmm),
(INSERT_SUBREG (_.info512.VT (IMPLICIT_DEF)), VR128X:$src2, sub_xmm)),
sub_xmm)>;
}
}
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
multiclass avx512_var_shift_w<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
let Predicates = [HasBWI] in
defm WZ: avx512_var_shift<opc, OpcodeStr, OpNode, v32i16_info>,
EVEX_V512, VEX_W;
let Predicates = [HasVLX, HasBWI] in {
defm WZ256: avx512_var_shift<opc, OpcodeStr, OpNode, v16i16x_info>,
EVEX_V256, VEX_W;
defm WZ128: avx512_var_shift<opc, OpcodeStr, OpNode, v8i16x_info>,
EVEX_V128, VEX_W;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
defm VPSLLV : avx512_var_shift_types<0x47, "vpsllv", shl>,
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
avx512_var_shift_w<0x12, "vpsllvw", shl>,
avx512_var_shift_w_lowering<avx512vl_i16_info, shl>;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
defm VPSRAV : avx512_var_shift_types<0x46, "vpsrav", sra>,
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
avx512_var_shift_w<0x11, "vpsravw", sra>,
avx512_var_shift_w_lowering<avx512vl_i16_info, sra>;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
defm VPSRLV : avx512_var_shift_types<0x45, "vpsrlv", srl>,
AVX512BW: Enable packed word shift for 512bit vector. Enable lowering scalar immidiate shift v64i8 .Fix predicate for AVX1/2 shifts. Differential Revision: http://reviews.llvm.org/D15713 llvm-svn: 256324
2015-12-23 16:06:50 +08:00
avx512_var_shift_w<0x10, "vpsrlvw", srl>,
avx512_var_shift_w_lowering<avx512vl_i16_info, srl>;
AVX-512: Added SKX forms of shift instructions. Added rotation instructions, encoding only. Added encoding tests for all these forms. llvm-svn: 231916
2015-03-11 18:25:42 +08:00
defm VPRORV : avx512_var_shift_types<0x14, "vprorv", rotr>;
defm VPROLV : avx512_var_shift_types<0x15, "vprolv", rotl>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: added all SKX forms of VPERMW/D/Q instructions. Added all forms of VPERMPS/PD instrcuctions. Added encoding tests. llvm-svn: 239016
2015-06-04 15:07:13 +08:00
//===-------------------------------------------------------------------===//
// 1-src variable permutation VPERMW/D/Q
//===-------------------------------------------------------------------===//
multiclass avx512_vperm_dq_sizes<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in
defm Z : avx512_var_shift<opc, OpcodeStr, OpNode, _.info512>,
avx512_var_shift_mb<opc, OpcodeStr, OpNode, _.info512>, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in
defm Z256 : avx512_var_shift<opc, OpcodeStr, OpNode, _.info256>,
avx512_var_shift_mb<opc, OpcodeStr, OpNode, _.info256>, EVEX_V256;
}
multiclass avx512_vpermi_dq_sizes<bits<8> opc, Format ImmFormR, Format ImmFormM,
string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo> {
let Predicates = [HasAVX512] in
defm Z: avx512_shift_rmi<opc, ImmFormR, ImmFormM, OpcodeStr, OpNode,
VTInfo.info512>,
avx512_shift_rmbi<opc, ImmFormM, OpcodeStr, OpNode,
VTInfo.info512>, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in
defm Z256: avx512_shift_rmi<opc, ImmFormR, ImmFormM, OpcodeStr, OpNode,
VTInfo.info256>,
avx512_shift_rmbi<opc, ImmFormM, OpcodeStr, OpNode,
VTInfo.info256>, EVEX_V256;
}
[AVX512] Adding VPERMB instruction Differential Revision: http://reviews.llvm.org/D16294 llvm-svn: 258144
2016-01-20 01:07:43 +08:00
multiclass avx512_vperm_bw<bits<8> opc, string OpcodeStr,
Predicate prd, SDNode OpNode,
AVX512VLVectorVTInfo _> {
let Predicates = [prd] in
defm Z: avx512_var_shift<opc, OpcodeStr, OpNode, _.info512>,
EVEX_V512 ;
let Predicates = [HasVLX, prd] in {
defm Z256: avx512_var_shift<opc, OpcodeStr, OpNode, _.info256>,
EVEX_V256 ;
defm Z128: avx512_var_shift<opc, OpcodeStr, OpNode, _.info128>,
EVEX_V128 ;
}
}
AVX-512: added all SKX forms of VPERMW/D/Q instructions. Added all forms of VPERMPS/PD instrcuctions. Added encoding tests. llvm-svn: 239016
2015-06-04 15:07:13 +08:00
[AVX512] Adding VPERMB instruction Differential Revision: http://reviews.llvm.org/D16294 llvm-svn: 258144
2016-01-20 01:07:43 +08:00
defm VPERMW : avx512_vperm_bw<0x8D, "vpermw", HasBWI, X86VPermv,
avx512vl_i16_info>, VEX_W;
defm VPERMB : avx512_vperm_bw<0x8D, "vpermb", HasVBMI, X86VPermv,
avx512vl_i8_info>;
AVX-512: added all SKX forms of VPERMW/D/Q instructions. Added all forms of VPERMPS/PD instrcuctions. Added encoding tests. llvm-svn: 239016
2015-06-04 15:07:13 +08:00
defm VPERMD : avx512_vperm_dq_sizes<0x36, "vpermd", X86VPermv,
avx512vl_i32_info>;
defm VPERMQ : avx512_vperm_dq_sizes<0x36, "vpermq", X86VPermv,
avx512vl_i64_info>, VEX_W;
defm VPERMPS : avx512_vperm_dq_sizes<0x16, "vpermps", X86VPermv,
avx512vl_f32_info>;
defm VPERMPD : avx512_vperm_dq_sizes<0x16, "vpermpd", X86VPermv,
avx512vl_f64_info>, VEX_W;
defm VPERMQ : avx512_vpermi_dq_sizes<0x00, MRMSrcReg, MRMSrcMem, "vpermq",
X86VPermi, avx512vl_i64_info>,
EVEX, AVX512AIi8Base, EVEX_CD8<64, CD8VF>, VEX_W;
defm VPERMPD : avx512_vpermi_dq_sizes<0x01, MRMSrcReg, MRMSrcMem, "vpermpd",
X86VPermi, avx512vl_f64_info>,
EVEX, AVX512AIi8Base, EVEX_CD8<64, CD8VF>, VEX_W;
AVX512: Implemented encoding and intrinsics for VPERMILPS/PD instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12690 llvm-svn: 249261
2015-10-04 15:20:41 +08:00
//===----------------------------------------------------------------------===//
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
// AVX-512 - VPERMIL
AVX512: Implemented encoding and intrinsics for VPERMILPS/PD instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12690 llvm-svn: 249261
2015-10-04 15:20:41 +08:00
//===----------------------------------------------------------------------===//
multiclass avx512_permil_vec<bits<8> OpcVar, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _, X86VectorVTInfo Ctrl> {
defm rr: AVX512_maskable<OpcVar, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, Ctrl.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1,
(Ctrl.VT Ctrl.RC:$src2)))>,
T8PD, EVEX_4V;
let mayLoad = 1 in {
defm rm: AVX512_maskable<OpcVar, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, Ctrl.MemOp:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode
_.RC:$src1,
(Ctrl.VT (bitconvert(Ctrl.LdFrag addr:$src2)))))>,
T8PD, EVEX_4V, EVEX_CD8<_.EltSize, CD8VF>;
defm rmb: AVX512_maskable<OpcVar, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr,
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(_.VT (OpNode
_.RC:$src1,
(Ctrl.VT (X86VBroadcast
(Ctrl.ScalarLdFrag addr:$src2)))))>,
T8PD, EVEX_4V, EVEX_B, EVEX_CD8<_.EltSize, CD8VF>;
}//let mayLoad = 1
}
multiclass avx512_permil_vec_common<string OpcodeStr, bits<8> OpcVar,
AVX512VLVectorVTInfo _, AVX512VLVectorVTInfo Ctrl>{
let Predicates = [HasAVX512] in {
defm Z : avx512_permil_vec<OpcVar, OpcodeStr, X86VPermilpv, _.info512,
Ctrl.info512>, EVEX_V512;
}
let Predicates = [HasAVX512, HasVLX] in {
defm Z128 : avx512_permil_vec<OpcVar, OpcodeStr, X86VPermilpv, _.info128,
Ctrl.info128>, EVEX_V128;
defm Z256 : avx512_permil_vec<OpcVar, OpcodeStr, X86VPermilpv, _.info256,
Ctrl.info256>, EVEX_V256;
}
}
multiclass avx512_permil<string OpcodeStr, bits<8> OpcImm, bits<8> OpcVar,
AVX512VLVectorVTInfo _, AVX512VLVectorVTInfo Ctrl>{
defm NAME: avx512_permil_vec_common<OpcodeStr, OpcVar, _, Ctrl>;
defm NAME: avx512_shift_rmi_sizes<OpcImm, MRMSrcReg, MRMSrcMem, OpcodeStr,
X86VPermilpi, _>,
EVEX, AVX512AIi8Base, EVEX_CD8<_.info128.EltSize, CD8VF>;
}
AVX-512: added all SKX forms of VPERMW/D/Q instructions. Added all forms of VPERMPS/PD instrcuctions. Added encoding tests. llvm-svn: 239016
2015-06-04 15:07:13 +08:00
AVX512: Implemented encoding and intrinsics for VPERMILPS/PD instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12690 llvm-svn: 249261
2015-10-04 15:20:41 +08:00
defm VPERMILPS : avx512_permil<"vpermilps", 0x04, 0x0C, avx512vl_f32_info,
avx512vl_i32_info>;
defm VPERMILPD : avx512_permil<"vpermilpd", 0x05, 0x0D, avx512vl_f64_info,
avx512vl_i64_info>, VEX_W;
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - VPSHUFD, VPSHUFLW, VPSHUFHW
//===----------------------------------------------------------------------===//
defm VPSHUFD : avx512_shift_rmi_sizes<0x70, MRMSrcReg, MRMSrcMem, "vpshufd",
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
X86PShufd, avx512vl_i32_info>,
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
EVEX, AVX512BIi8Base, EVEX_CD8<32, CD8VF>;
defm VPSHUFH : avx512_shift_rmi_w<0x70, MRMSrcReg, MRMSrcMem, "vpshufhw",
AVX512: Change encoding of vpshuflw and vpshufhw instructions. Implement WIG as W0 and not W1, like all other instruction have been implemented. Add encoding tests. Differential Revision: http://reviews.llvm.org/D13471 llvm-svn: 249521
2015-10-07 14:31:18 +08:00
X86PShufhw>, EVEX, AVX512XSIi8Base;
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
defm VPSHUFL : avx512_shift_rmi_w<0x70, MRMSrcReg, MRMSrcMem, "vpshuflw",
AVX512: Change encoding of vpshuflw and vpshufhw instructions. Implement WIG as W0 and not W1, like all other instruction have been implemented. Add encoding tests. Differential Revision: http://reviews.llvm.org/D13471 llvm-svn: 249521
2015-10-07 14:31:18 +08:00
X86PShuflw>, EVEX, AVX512XDIi8Base;
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
AVX-512: added VPSHUFB instruction - all SKX forms Added intrinsics and encoding tests. llvm-svn: 240277
2015-06-22 21:00:42 +08:00
multiclass avx512_pshufb_sizes<bits<8> opc, string OpcodeStr, SDNode OpNode> {
let Predicates = [HasBWI] in
defm Z: avx512_var_shift<opc, OpcodeStr, OpNode, v64i8_info>, EVEX_V512;
let Predicates = [HasVLX, HasBWI] in {
defm Z256: avx512_var_shift<opc, OpcodeStr, OpNode, v32i8x_info>, EVEX_V256;
defm Z128: avx512_var_shift<opc, OpcodeStr, OpNode, v16i8x_info>, EVEX_V128;
}
}
defm VPSHUFB: avx512_pshufb_sizes<0x00, "vpshufb", X86pshufb>;
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
//===----------------------------------------------------------------------===//
// Move Low to High and High to Low packed FP Instructions
//===----------------------------------------------------------------------===//
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def VMOVLHPSZrr : AVX512PSI<0x16, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src1, VR128X:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovlhps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst, (v4f32 (X86Movlhps VR128X:$src1, VR128X:$src2)))],
IIC_SSE_MOV_LH>, EVEX_4V;
def VMOVHLPSZrr : AVX512PSI<0x12, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src1, VR128X:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovhlps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst, (v4f32 (X86Movhlps VR128X:$src1, VR128X:$src2)))],
IIC_SSE_MOV_LH>, EVEX_4V;
Put HasAVX512 predicate on some patterns to properly disable them when AVX512 isn't enabled. Currently it works simply because the SSE and AVX version of the same patterns are checked first in the DAG isel table. llvm-svn: 191490
2013-09-27 15:20:47 +08:00
let Predicates = [HasAVX512] in {
// MOVLHPS patterns
def : Pat<(v4i32 (X86Movlhps VR128X:$src1, VR128X:$src2)),
(VMOVLHPSZrr VR128X:$src1, VR128X:$src2)>;
def : Pat<(v2i64 (X86Movlhps VR128X:$src1, VR128X:$src2)),
(VMOVLHPSZrr (v2i64 VR128X:$src1), VR128X:$src2)>;
// MOVHLPS patterns
def : Pat<(v4i32 (X86Movhlps VR128X:$src1, VR128X:$src2)),
(VMOVHLPSZrr VR128X:$src1, VR128X:$src2)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX512 : Implemented encoding and DAG lowering for VMOVHPS/PD and VMOVLPS/PD instructions. Differential Revision: http://reviews.llvm.org/D14492 llvm-svn: 252592
2015-11-10 15:09:07 +08:00
//===----------------------------------------------------------------------===//
// VMOVHPS/PD VMOVLPS Instructions
// All patterns was taken from SSS implementation.
//===----------------------------------------------------------------------===//
multiclass avx512_mov_hilo_packed<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
let mayLoad = 1 in
def rm : AVX512<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, f64mem:$src2),
!strconcat(OpcodeStr,
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set _.RC:$dst,
(OpNode _.RC:$src1,
(_.VT (bitconvert
(v2f64 (scalar_to_vector (loadf64 addr:$src2)))))))],
IIC_SSE_MOV_LH>, EVEX_4V;
}
defm VMOVHPSZ128 : avx512_mov_hilo_packed<0x16, "vmovhps", X86Movlhps,
v4f32x_info>, EVEX_CD8<32, CD8VT2>, PS;
defm VMOVHPDZ128 : avx512_mov_hilo_packed<0x16, "vmovhpd", X86Movlhpd,
v2f64x_info>, EVEX_CD8<64, CD8VT1>, PD, VEX_W;
defm VMOVLPSZ128 : avx512_mov_hilo_packed<0x12, "vmovlps", X86Movlps,
v4f32x_info>, EVEX_CD8<32, CD8VT2>, PS;
defm VMOVLPDZ128 : avx512_mov_hilo_packed<0x12, "vmovlpd", X86Movlpd,
v2f64x_info>, EVEX_CD8<64, CD8VT1>, PD, VEX_W;
let Predicates = [HasAVX512] in {
// VMOVHPS patterns
def : Pat<(X86Movlhps VR128X:$src1,
(bc_v4f32 (v2i64 (scalar_to_vector (loadi64 addr:$src2))))),
(VMOVHPSZ128rm VR128X:$src1, addr:$src2)>;
def : Pat<(X86Movlhps VR128X:$src1,
(bc_v4i32 (v2i64 (X86vzload addr:$src2)))),
(VMOVHPSZ128rm VR128X:$src1, addr:$src2)>;
// VMOVHPD patterns
def : Pat<(v2f64 (X86Unpckl VR128X:$src1,
(scalar_to_vector (loadf64 addr:$src2)))),
(VMOVHPDZ128rm VR128X:$src1, addr:$src2)>;
def : Pat<(v2f64 (X86Unpckl VR128X:$src1,
(bc_v2f64 (v2i64 (scalar_to_vector (loadi64 addr:$src2)))))),
(VMOVHPDZ128rm VR128X:$src1, addr:$src2)>;
// VMOVLPS patterns
def : Pat<(v4f32 (X86Movlps VR128X:$src1, (load addr:$src2))),
(VMOVLPSZ128rm VR128X:$src1, addr:$src2)>;
def : Pat<(v4i32 (X86Movlps VR128X:$src1, (load addr:$src2))),
(VMOVLPSZ128rm VR128X:$src1, addr:$src2)>;
// VMOVLPD patterns
def : Pat<(v2f64 (X86Movlpd VR128X:$src1, (load addr:$src2))),
(VMOVLPDZ128rm VR128X:$src1, addr:$src2)>;
def : Pat<(v2i64 (X86Movlpd VR128X:$src1, (load addr:$src2))),
(VMOVLPDZ128rm VR128X:$src1, addr:$src2)>;
def : Pat<(v2f64 (X86Movsd VR128X:$src1,
(v2f64 (scalar_to_vector (loadf64 addr:$src2))))),
(VMOVLPDZ128rm VR128X:$src1, addr:$src2)>;
}
let mayStore = 1 in {
def VMOVHPSZ128mr : AVX512PSI<0x17, MRMDestMem, (outs),
(ins f64mem:$dst, VR128X:$src),
"vmovhps\t{$src, $dst|$dst, $src}",
[AVX512] Replace vector_extract with extractelt in some patterns. They mean the same thing but vector_extract is deprecated. NFC llvm-svn: 268179
2016-05-01 12:59:44 +08:00
[(store (f64 (extractelt
AVX512 : Implemented encoding and DAG lowering for VMOVHPS/PD and VMOVLPS/PD instructions. Differential Revision: http://reviews.llvm.org/D14492 llvm-svn: 252592
2015-11-10 15:09:07 +08:00
(X86Unpckh (bc_v2f64 (v4f32 VR128X:$src)),
(bc_v2f64 (v4f32 VR128X:$src))),
(iPTR 0))), addr:$dst)], IIC_SSE_MOV_LH>,
EVEX, EVEX_CD8<32, CD8VT2>;
def VMOVHPDZ128mr : AVX512PDI<0x17, MRMDestMem, (outs),
(ins f64mem:$dst, VR128X:$src),
"vmovhpd\t{$src, $dst|$dst, $src}",
[AVX512] Replace vector_extract with extractelt in some patterns. They mean the same thing but vector_extract is deprecated. NFC llvm-svn: 268179
2016-05-01 12:59:44 +08:00
[(store (f64 (extractelt
AVX512 : Implemented encoding and DAG lowering for VMOVHPS/PD and VMOVLPS/PD instructions. Differential Revision: http://reviews.llvm.org/D14492 llvm-svn: 252592
2015-11-10 15:09:07 +08:00
(v2f64 (X86Unpckh VR128X:$src, VR128X:$src)),
(iPTR 0))), addr:$dst)], IIC_SSE_MOV_LH>,
EVEX, EVEX_CD8<64, CD8VT1>, VEX_W;
def VMOVLPSZ128mr : AVX512PSI<0x13, MRMDestMem, (outs),
(ins f64mem:$dst, VR128X:$src),
"vmovlps\t{$src, $dst|$dst, $src}",
[AVX512] Replace vector_extract with extractelt in some patterns. They mean the same thing but vector_extract is deprecated. NFC llvm-svn: 268179
2016-05-01 12:59:44 +08:00
[(store (f64 (extractelt (bc_v2f64 (v4f32 VR128X:$src)),
AVX512 : Implemented encoding and DAG lowering for VMOVHPS/PD and VMOVLPS/PD instructions. Differential Revision: http://reviews.llvm.org/D14492 llvm-svn: 252592
2015-11-10 15:09:07 +08:00
(iPTR 0))), addr:$dst)],
IIC_SSE_MOV_LH>,
EVEX, EVEX_CD8<32, CD8VT2>;
def VMOVLPDZ128mr : AVX512PDI<0x13, MRMDestMem, (outs),
(ins f64mem:$dst, VR128X:$src),
"vmovlpd\t{$src, $dst|$dst, $src}",
[AVX512] Replace vector_extract with extractelt in some patterns. They mean the same thing but vector_extract is deprecated. NFC llvm-svn: 268179
2016-05-01 12:59:44 +08:00
[(store (f64 (extractelt (v2f64 VR128X:$src),
AVX512 : Implemented encoding and DAG lowering for VMOVHPS/PD and VMOVLPS/PD instructions. Differential Revision: http://reviews.llvm.org/D14492 llvm-svn: 252592
2015-11-10 15:09:07 +08:00
(iPTR 0))), addr:$dst)],
IIC_SSE_MOV_LH>,
EVEX, EVEX_CD8<64, CD8VT1>, VEX_W;
}
let Predicates = [HasAVX512] in {
// VMOVHPD patterns
[AVX512] Replace vector_extract with extractelt in some patterns. They mean the same thing but vector_extract is deprecated. NFC llvm-svn: 268179
2016-05-01 12:59:44 +08:00
def : Pat<(store (f64 (extractelt
AVX512 : Implemented encoding and DAG lowering for VMOVHPS/PD and VMOVLPS/PD instructions. Differential Revision: http://reviews.llvm.org/D14492 llvm-svn: 252592
2015-11-10 15:09:07 +08:00
(v2f64 (X86VPermilpi VR128X:$src, (i8 1))),
(iPTR 0))), addr:$dst),
(VMOVHPDZ128mr addr:$dst, VR128X:$src)>;
// VMOVLPS patterns
def : Pat<(store (v4f32 (X86Movlps (load addr:$src1), VR128X:$src2)),
addr:$src1),
(VMOVLPSZ128mr addr:$src1, VR128X:$src2)>;
def : Pat<(store (v4i32 (X86Movlps
(bc_v4i32 (loadv2i64 addr:$src1)), VR128X:$src2)), addr:$src1),
(VMOVLPSZ128mr addr:$src1, VR128X:$src2)>;
// VMOVLPD patterns
def : Pat<(store (v2f64 (X86Movlpd (load addr:$src1), VR128X:$src2)),
addr:$src1),
(VMOVLPDZ128mr addr:$src1, VR128X:$src2)>;
def : Pat<(store (v2i64 (X86Movlpd (load addr:$src1), VR128X:$src2)),
addr:$src1),
(VMOVLPDZ128mr addr:$src1, VR128X:$src2)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// FMA - Fused Multiply Operations
//
[AVX512] Introduce fma3p_forms from AVX This multiclass generates the different forms: 213, 231, 132 in AVX. 132 in AVX512 is a separate class but I am planning to use this same multiclass to generate 231 relying on the nice the null_frag trick from AVX to disable codegen pattern for 231. No functionality change, no change in X86.td.expanded except for the different instruction definition names. llvm-svn: 220539
2014-10-24 08:02:55 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Constraints = "$src1 = $dst" in {
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
multiclass avx512_fma3p_213_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm r: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
(ins _.RC:$src2, _.RC:$src3),
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
OpcodeStr, "$src3, $src2", "$src2, $src3",
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
(_.VT (OpNode _.RC:$src1, _.RC:$src2, _.RC:$src3))>,
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
AVX512FMA3Base;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
let mayLoad = 1 in {
defm m: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
(ins _.RC:$src2, _.MemOp:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
(_.VT (OpNode _.RC:$src1, _.RC:$src2, (_.LdFrag addr:$src3)))>,
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
AVX512FMA3Base;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
defm mb: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
(ins _.RC:$src2, _.ScalarMemOp:$src3),
AVX-512: blank lines, duplicated tests, no functional changes see comments http://reviews.llvm.org/D6835 llvm-svn: 233528
2015-03-30 17:29:28 +08:00
OpcodeStr, !strconcat("${src3}", _.BroadcastStr,", $src2"),
!strconcat("$src2, ${src3}", _.BroadcastStr ),
(OpNode _.RC:$src1,
[X86][SSE] Use the general SMAX/SMIN/UMAX/UMIN opcodes and remove the X86 implementation With the completion of D9746 there is now a common implementation of integer signed/unsigned min/max nodes, removing the need for the equivalent X86 specific implementations. This patch removes the old X86ISD nodes, legalizes the relevant SSE2/SSE41/AVX2/AVX512 instructions for the ISD versions and converts the small amount of existing X86 code. Differential Revision: http://reviews.llvm.org/D10947 llvm-svn: 241506
2015-07-07 04:30:47 +08:00
_.RC:$src2,(_.VT (X86VBroadcast (_.ScalarLdFrag addr:$src3))))>,
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
AVX512FMA3Base, EVEX_B;
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
}
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
multiclass avx512_fma3_213_round<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm rb: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
AVX-512: Added FMA intrinsics with rounding mode By Asaf Badouh and Elena Demikhovsky Added special nodes for rounding: FMADD_RND, FMSUB_RND.. It will prevent merge between nodes with rounding and other standard nodes. llvm-svn: 227303
2015-01-28 18:21:27 +08:00
(ins _.RC:$src2, _.RC:$src3, AVX512RC:$rc),
OpcodeStr, "$rc, $src3, $src2", "$src2, $src3, $rc",
(_.VT ( OpNode _.RC:$src1, _.RC:$src2, _.RC:$src3, (i32 imm:$rc)))>,
AVX512FMA3Base, EVEX_B, EVEX_RC;
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
}
AVX-512: Added FMA intrinsics with rounding mode By Asaf Badouh and Elena Demikhovsky Added special nodes for rounding: FMADD_RND, FMSUB_RND.. It will prevent merge between nodes with rounding and other standard nodes. llvm-svn: 227303
2015-01-28 18:21:27 +08:00
} // Constraints = "$src1 = $dst"
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
multiclass avx512_fma3p_213_common<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd, AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in {
defm Z : avx512_fma3p_213_rm<opc, OpcodeStr, OpNode, _.info512>,
avx512_fma3_213_round<opc, OpcodeStr, OpNodeRnd, _.info512>,
EVEX_V512, EVEX_CD8<_.info512.EltSize, CD8VF>;
}
let Predicates = [HasVLX, HasAVX512] in {
defm Z256 : avx512_fma3p_213_rm<opc, OpcodeStr, OpNode, _.info256>,
EVEX_V256, EVEX_CD8<_.info256.EltSize, CD8VF>;
defm Z128 : avx512_fma3p_213_rm<opc, OpcodeStr, OpNode, _.info128>,
EVEX_V128, EVEX_CD8<_.info128.EltSize, CD8VF>;
}
AVX-512: Added FMA intrinsics with rounding mode By Asaf Badouh and Elena Demikhovsky Added special nodes for rounding: FMADD_RND, FMSUB_RND.. It will prevent merge between nodes with rounding and other standard nodes. llvm-svn: 227303
2015-01-28 18:21:27 +08:00
}
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
multiclass avx512_fma3p_213_f<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd > {
defm PS : avx512_fma3p_213_common<opc, OpcodeStr#"ps", OpNode, OpNodeRnd,
avx512vl_f32_info>;
defm PD : avx512_fma3p_213_common<opc, OpcodeStr#"pd", OpNode, OpNodeRnd,
avx512vl_f64_info>, VEX_W;
[AVX512] Introduce fma3p_forms from AVX This multiclass generates the different forms: 213, 231, 132 in AVX. 132 in AVX512 is a separate class but I am planning to use this same multiclass to generate 231 relying on the nice the null_frag trick from AVX to disable codegen pattern for 231. No functionality change, no change in X86.td.expanded except for the different instruction definition names. llvm-svn: 220539
2014-10-24 08:02:55 +08:00
}
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
defm VFMADD213 : avx512_fma3p_213_f<0xA8, "vfmadd213", X86Fmadd, X86FmaddRnd>;
defm VFMSUB213 : avx512_fma3p_213_f<0xAA, "vfmsub213", X86Fmsub, X86FmsubRnd>;
defm VFMADDSUB213 : avx512_fma3p_213_f<0xA6, "vfmaddsub213", X86Fmaddsub, X86FmaddsubRnd>;
defm VFMSUBADD213 : avx512_fma3p_213_f<0xA7, "vfmsubadd213", X86Fmsubadd, X86FmsubaddRnd>;
defm VFNMADD213 : avx512_fma3p_213_f<0xAC, "vfnmadd213", X86Fnmadd, X86FnmaddRnd>;
defm VFNMSUB213 : avx512_fma3p_213_f<0xAE, "vfnmsub213", X86Fnmsub, X86FnmsubRnd>;
let Constraints = "$src1 = $dst" in {
multiclass avx512_fma3p_231_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm r: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
(_.VT (OpNode _.RC:$src2, _.RC:$src3, _.RC:$src1))>,
AVX512FMA3Base;
let mayLoad = 1 in {
defm m: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.MemOp:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
(_.VT (OpNode _.RC:$src2, (_.LdFrag addr:$src3), _.RC:$src1))>,
AVX512FMA3Base;
defm mb: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.ScalarMemOp:$src3),
OpcodeStr, "${src3}"##_.BroadcastStr##", $src2",
"$src2, ${src3}"##_.BroadcastStr,
(_.VT (OpNode _.RC:$src2,
(_.VT (X86VBroadcast(_.ScalarLdFrag addr:$src3))),
_.RC:$src1))>, AVX512FMA3Base, EVEX_B;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
}
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
}
multiclass avx512_fma3_231_round<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm rb: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, AVX512RC:$rc),
OpcodeStr, "$rc, $src3, $src2", "$src2, $src3, $rc",
(_.VT ( OpNode _.RC:$src2, _.RC:$src3, _.RC:$src1, (i32 imm:$rc)))>,
AVX512FMA3Base, EVEX_B, EVEX_RC;
}
} // Constraints = "$src1 = $dst"
multiclass avx512_fma3p_231_common<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd, AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in {
defm Z : avx512_fma3p_231_rm<opc, OpcodeStr, OpNode, _.info512>,
avx512_fma3_231_round<opc, OpcodeStr, OpNodeRnd, _.info512>,
EVEX_V512, EVEX_CD8<_.info512.EltSize, CD8VF>;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
}
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
let Predicates = [HasVLX, HasAVX512] in {
defm Z256 : avx512_fma3p_231_rm<opc, OpcodeStr, OpNode, _.info256>,
EVEX_V256, EVEX_CD8<_.info256.EltSize, CD8VF>;
defm Z128 : avx512_fma3p_231_rm<opc, OpcodeStr, OpNode, _.info128>,
EVEX_V128, EVEX_CD8<_.info128.EltSize, CD8VF>;
}
}
multiclass avx512_fma3p_231_f<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd > {
defm PS : avx512_fma3p_231_common<opc, OpcodeStr#"ps", OpNode, OpNodeRnd,
avx512vl_f32_info>;
defm PD : avx512_fma3p_231_common<opc, OpcodeStr#"pd", OpNode, OpNodeRnd,
avx512vl_f64_info>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
defm VFMADD231 : avx512_fma3p_231_f<0xB8, "vfmadd231", X86Fmadd, X86FmaddRnd>;
defm VFMSUB231 : avx512_fma3p_231_f<0xBA, "vfmsub231", X86Fmsub, X86FmsubRnd>;
defm VFMADDSUB231 : avx512_fma3p_231_f<0xB6, "vfmaddsub231", X86Fmaddsub, X86FmaddsubRnd>;
defm VFMSUBADD231 : avx512_fma3p_231_f<0xB7, "vfmsubadd231", X86Fmsubadd, X86FmsubaddRnd>;
defm VFNMADD231 : avx512_fma3p_231_f<0xBC, "vfnmadd231", X86Fnmadd, X86FnmaddRnd>;
defm VFNMSUB231 : avx512_fma3p_231_f<0xBE, "vfnmsub231", X86Fnmsub, X86FnmsubRnd>;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Constraints = "$src1 = $dst" in {
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
multiclass avx512_fma3p_132_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm r: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src3, _.RC:$src2),
OpcodeStr, "$src2, $src3", "$src3, $src2",
(_.VT (OpNode _.RC:$src1, _.RC:$src2, _.RC:$src3))>,
AVX512FMA3Base;
let mayLoad = 1 in {
defm m: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src3, _.MemOp:$src2),
OpcodeStr, "$src2, $src3", "$src3, $src2",
(_.VT (OpNode _.RC:$src1, (_.LdFrag addr:$src2), _.RC:$src3))>,
AVX512FMA3Base;
defm mb: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src3, _.ScalarMemOp:$src2),
OpcodeStr, "${src2}"##_.BroadcastStr##", $src3",
"$src3, ${src2}"##_.BroadcastStr,
(_.VT (OpNode _.RC:$src1,
(_.VT (X86VBroadcast(_.ScalarLdFrag addr:$src2))),
_.RC:$src3))>, AVX512FMA3Base, EVEX_B;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
multiclass avx512_fma3_132_round<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm rb: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src3, _.RC:$src2, AVX512RC:$rc),
OpcodeStr, "$rc, $src2, $src3", "$src3, $src2, $rc",
(_.VT ( OpNode _.RC:$src1, _.RC:$src2, _.RC:$src3, (i32 imm:$rc)))>,
AVX512FMA3Base, EVEX_B, EVEX_RC;
}
} // Constraints = "$src1 = $dst"
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
multiclass avx512_fma3p_132_common<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd, AVX512VLVectorVTInfo _> {
let Predicates = [HasAVX512] in {
defm Z : avx512_fma3p_132_rm<opc, OpcodeStr, OpNode, _.info512>,
avx512_fma3_132_round<opc, OpcodeStr, OpNodeRnd, _.info512>,
EVEX_V512, EVEX_CD8<_.info512.EltSize, CD8VF>;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
}
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
let Predicates = [HasVLX, HasAVX512] in {
defm Z256 : avx512_fma3p_132_rm<opc, OpcodeStr, OpNode, _.info256>,
EVEX_V256, EVEX_CD8<_.info256.EltSize, CD8VF>;
defm Z128 : avx512_fma3p_132_rm<opc, OpcodeStr, OpNode, _.info128>,
EVEX_V128, EVEX_CD8<_.info128.EltSize, CD8VF>;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Implemented missing encoding and intrinsics for FMA instructions Added tests for DAG lowering ,encoding and intrinsics Differential Revision: http://reviews.llvm.org/D10796 llvm-svn: 240926
2015-06-29 17:10:00 +08:00
multiclass avx512_fma3p_132_f<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd > {
defm PS : avx512_fma3p_132_common<opc, OpcodeStr#"ps", OpNode, OpNodeRnd,
avx512vl_f32_info>;
defm PD : avx512_fma3p_132_common<opc, OpcodeStr#"pd", OpNode, OpNodeRnd,
avx512vl_f64_info>, VEX_W;
}
defm VFMADD132 : avx512_fma3p_132_f<0x98, "vfmadd132", X86Fmadd, X86FmaddRnd>;
defm VFMSUB132 : avx512_fma3p_132_f<0x9A, "vfmsub132", X86Fmsub, X86FmsubRnd>;
defm VFMADDSUB132 : avx512_fma3p_132_f<0x96, "vfmaddsub132", X86Fmaddsub, X86FmaddsubRnd>;
defm VFMSUBADD132 : avx512_fma3p_132_f<0x97, "vfmsubadd132", X86Fmsubadd, X86FmsubaddRnd>;
defm VFNMADD132 : avx512_fma3p_132_f<0x9C, "vfnmadd132", X86Fnmadd, X86FnmaddRnd>;
defm VFNMSUB132 : avx512_fma3p_132_f<0x9E, "vfnmsub132", X86Fnmsub, X86FnmsubRnd>;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Scalar FMA
let Constraints = "$src1 = $dst" in {
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
multiclass avx512_fma3s_common<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
dag RHS_VEC_r, dag RHS_VEC_m, dag RHS_VEC_rb,
dag RHS_r, dag RHS_m > {
defm r_Int: AVX512_maskable_3src_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3), OpcodeStr,
"$src3, $src2", "$src2, $src3", RHS_VEC_r>, AVX512FMA3Base;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
defm m_Int: AVX512_maskable_3src_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src2, _.ScalarMemOp:$src3), OpcodeStr,
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
"$src3, $src2", "$src2, $src3", RHS_VEC_m>, AVX512FMA3Base;
defm rb_Int: AVX512_maskable_3src_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, AVX512RC:$rc),
OpcodeStr, "$rc, $src3, $src2", "$src2, $src3, $rc", RHS_VEC_rb>,
AVX512FMA3Base, EVEX_B, EVEX_RC;
let isCodeGenOnly = 1 in {
def r : AVX512FMA3<opc, MRMSrcReg, (outs _.FRC:$dst),
(ins _.FRC:$src1, _.FRC:$src2, _.FRC:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
[RHS_r]>;
let mayLoad = 1 in
def m : AVX512FMA3<opc, MRMSrcMem, (outs _.FRC:$dst),
(ins _.FRC:$src1, _.FRC:$src2, _.ScalarMemOp:$src3),
!strconcat(OpcodeStr,
"\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
[RHS_m]>;
}// isCodeGenOnly = 1
}
}// Constraints = "$src1 = $dst"
multiclass avx512_fma3s_all<bits<8> opc213, bits<8> opc231, bits<8> opc132,
string OpcodeStr, SDNode OpNode, SDNode OpNodeRnd, X86VectorVTInfo _ ,
string SUFF> {
defm NAME#213#SUFF: avx512_fma3s_common<opc213, OpcodeStr#"213"#_.Suffix , _ ,
[AVX512] add vfmadd132ss and vfmadd132sd Intrinsic Differential Revision: http://reviews.llvm.org/D16589 llvm-svn: 259789
2016-02-04 22:41:08 +08:00
(_.VT (OpNodeRnd _.RC:$src2, _.RC:$src1, _.RC:$src3, (i32 FROUND_CURRENT))),
(_.VT (OpNodeRnd _.RC:$src2, _.RC:$src1,
(_.VT (scalar_to_vector(_.ScalarLdFrag addr:$src3))), (i32 FROUND_CURRENT))),
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
(_.VT ( OpNodeRnd _.RC:$src2, _.RC:$src1, _.RC:$src3,
(i32 imm:$rc))),
(set _.FRC:$dst, (_.EltVT (OpNode _.FRC:$src2, _.FRC:$src1,
_.FRC:$src3))),
(set _.FRC:$dst, (_.EltVT (OpNode _.FRC:$src2, _.FRC:$src1,
(_.ScalarLdFrag addr:$src3))))>;
defm NAME#231#SUFF: avx512_fma3s_common<opc231, OpcodeStr#"231"#_.Suffix , _ ,
[AVX512] add vfmadd132ss and vfmadd132sd Intrinsic Differential Revision: http://reviews.llvm.org/D16589 llvm-svn: 259789
2016-02-04 22:41:08 +08:00
(_.VT (OpNodeRnd _.RC:$src2, _.RC:$src3, _.RC:$src1, (i32 FROUND_CURRENT))),
(_.VT (OpNodeRnd _.RC:$src2,
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
(_.VT (scalar_to_vector(_.ScalarLdFrag addr:$src3))),
[AVX512] add vfmadd132ss and vfmadd132sd Intrinsic Differential Revision: http://reviews.llvm.org/D16589 llvm-svn: 259789
2016-02-04 22:41:08 +08:00
_.RC:$src1, (i32 FROUND_CURRENT))),
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
(_.VT ( OpNodeRnd _.RC:$src2, _.RC:$src3, _.RC:$src1,
(i32 imm:$rc))),
(set _.FRC:$dst, (_.EltVT (OpNode _.FRC:$src2, _.FRC:$src3,
_.FRC:$src1))),
(set _.FRC:$dst, (_.EltVT (OpNode _.FRC:$src2,
(_.ScalarLdFrag addr:$src3), _.FRC:$src1)))>;
defm NAME#132#SUFF: avx512_fma3s_common<opc132, OpcodeStr#"132"#_.Suffix , _ ,
[AVX512] add vfmadd132ss and vfmadd132sd Intrinsic Differential Revision: http://reviews.llvm.org/D16589 llvm-svn: 259789
2016-02-04 22:41:08 +08:00
(_.VT (OpNodeRnd _.RC:$src1, _.RC:$src3, _.RC:$src2, (i32 FROUND_CURRENT))),
(_.VT (OpNodeRnd _.RC:$src1,
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
(_.VT (scalar_to_vector(_.ScalarLdFrag addr:$src3))),
[AVX512] add vfmadd132ss and vfmadd132sd Intrinsic Differential Revision: http://reviews.llvm.org/D16589 llvm-svn: 259789
2016-02-04 22:41:08 +08:00
_.RC:$src2, (i32 FROUND_CURRENT))),
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
(_.VT ( OpNodeRnd _.RC:$src1, _.RC:$src3, _.RC:$src2,
(i32 imm:$rc))),
(set _.FRC:$dst, (_.EltVT (OpNode _.FRC:$src1, _.FRC:$src3,
_.FRC:$src2))),
(set _.FRC:$dst, (_.EltVT (OpNode _.FRC:$src1,
(_.ScalarLdFrag addr:$src3), _.FRC:$src2)))>;
}
multiclass avx512_fma3s<bits<8> opc213, bits<8> opc231, bits<8> opc132,
string OpcodeStr, SDNode OpNode, SDNode OpNodeRnd>{
let Predicates = [HasAVX512] in {
defm NAME : avx512_fma3s_all<opc213, opc231, opc132, OpcodeStr, OpNode,
OpNodeRnd, f32x_info, "SS">,
EVEX_CD8<32, CD8VT1>, VEX_LIG;
defm NAME : avx512_fma3s_all<opc213, opc231, opc132, OpcodeStr, OpNode,
OpNodeRnd, f64x_info, "SD">,
EVEX_CD8<64, CD8VT1>, VEX_LIG, VEX_W;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Implemented missing encoding for FMA scalar instructions Added tests for encoding Differential Revision: http://reviews.llvm.org/D10865 llvm-svn: 241159
2015-07-01 21:24:28 +08:00
defm VFMADD : avx512_fma3s<0xA9, 0xB9, 0x99, "vfmadd", X86Fmadd, X86FmaddRnd>;
defm VFMSUB : avx512_fma3s<0xAB, 0xBB, 0x9B, "vfmsub", X86Fmsub, X86FmsubRnd>;
defm VFNMADD : avx512_fma3s<0xAD, 0xBD, 0x9D, "vfnmadd", X86Fnmadd, X86FnmaddRnd>;
defm VFNMSUB : avx512_fma3s<0xAF, 0xBF, 0x9F, "vfnmsub", X86Fnmsub, X86FnmsubRnd>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[X86][IFMA] adding intrinsics and encoding for multiply and add of unsigned 52bit integer VPMADD52LUQ - Packed Multiply of Unsigned 52-bit Integers and Add the Low 52-bit Products to Qword Accumulators VPMADD52HUQ - Packed Multiply of Unsigned 52-bit Unsigned Integers and Add High 52-bit Products to 64-bit Accumulators Differential Revision: http://reviews.llvm.org/D16407 llvm-svn: 258680
2016-01-25 19:14:24 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Packed Multiply of Unsigned 52-bit Integers and Add the Low 52-bit IFMA
//===----------------------------------------------------------------------===//
let Constraints = "$src1 = $dst" in {
multiclass avx512_pmadd52_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm r: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
(_.VT (OpNode _.RC:$src1, _.RC:$src2, _.RC:$src3))>,
AVX512FMA3Base;
let mayLoad = 1 in {
defm m: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.MemOp:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
(_.VT (OpNode _.RC:$src1, _.RC:$src2, (_.LdFrag addr:$src3)))>,
AVX512FMA3Base;
defm mb: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.ScalarMemOp:$src3),
OpcodeStr, !strconcat("${src3}", _.BroadcastStr,", $src2"),
!strconcat("$src2, ${src3}", _.BroadcastStr ),
(OpNode _.RC:$src1,
_.RC:$src2,(_.VT (X86VBroadcast (_.ScalarLdFrag addr:$src3))))>,
AVX512FMA3Base, EVEX_B;
}
}
} // Constraints = "$src1 = $dst"
multiclass avx512_pmadd52_common<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo _> {
let Predicates = [HasIFMA] in {
defm Z : avx512_pmadd52_rm<opc, OpcodeStr, OpNode, _.info512>,
EVEX_V512, EVEX_CD8<_.info512.EltSize, CD8VF>;
}
let Predicates = [HasVLX, HasIFMA] in {
defm Z256 : avx512_pmadd52_rm<opc, OpcodeStr, OpNode, _.info256>,
EVEX_V256, EVEX_CD8<_.info256.EltSize, CD8VF>;
defm Z128 : avx512_pmadd52_rm<opc, OpcodeStr, OpNode, _.info128>,
EVEX_V128, EVEX_CD8<_.info128.EltSize, CD8VF>;
}
}
defm VPMADD52LUQ : avx512_pmadd52_common<0xb4, "vpmadd52luq", x86vpmadd52l,
avx512vl_i64_info>, VEX_W;
defm VPMADD52HUQ : avx512_pmadd52_common<0xb5, "vpmadd52huq", x86vpmadd52h,
avx512vl_i64_info>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Scalar convert from sign integer to float/double
//===----------------------------------------------------------------------===//
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
multiclass avx512_vcvtsi<bits<8> opc, SDNode OpNode, RegisterClass SrcRC,
X86VectorVTInfo DstVT, X86MemOperand x86memop,
PatFrag ld_frag, string asm> {
let hasSideEffects = 0 in {
def rr : SI<opc, MRMSrcReg, (outs DstVT.FRC:$dst),
(ins DstVT.FRC:$src1, SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $src1, $dst|$dst, $src1, $src}"), []>,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_4V;
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
let mayLoad = 1 in
def rm : SI<opc, MRMSrcMem, (outs DstVT.FRC:$dst),
(ins DstVT.FRC:$src1, x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $src1, $dst|$dst, $src1, $src}"), []>,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_4V;
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
} // hasSideEffects = 0
let isCodeGenOnly = 1 in {
def rr_Int : SI<opc, MRMSrcReg, (outs DstVT.RC:$dst),
(ins DstVT.RC:$src1, SrcRC:$src2),
!strconcat(asm,"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set DstVT.RC:$dst,
(OpNode (DstVT.VT DstVT.RC:$src1),
SrcRC:$src2,
(i32 FROUND_CURRENT)))]>, EVEX_4V;
def rm_Int : SI<opc, MRMSrcMem, (outs DstVT.RC:$dst),
(ins DstVT.RC:$src1, x86memop:$src2),
!strconcat(asm,"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set DstVT.RC:$dst,
(OpNode (DstVT.VT DstVT.RC:$src1),
(ld_frag addr:$src2),
(i32 FROUND_CURRENT)))]>, EVEX_4V;
}//isCodeGenOnly = 1
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: blank lines, duplicated tests, no functional changes see comments http://reviews.llvm.org/D6835 llvm-svn: 233528
2015-03-30 17:29:28 +08:00
AVX-512: Implemented cvtsi2ss/d cvtusi2ss/d instructions with round control for KNL. Added intrinsics for cvtsi2ss/d instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D10430 llvm-svn: 239694
2015-06-14 20:44:55 +08:00
multiclass avx512_vcvtsi_round<bits<8> opc, SDNode OpNode, RegisterClass SrcRC,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
X86VectorVTInfo DstVT, string asm> {
AVX-512: Implemented cvtsi2ss/d cvtusi2ss/d instructions with round control for KNL. Added intrinsics for cvtsi2ss/d instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D10430 llvm-svn: 239694
2015-06-14 20:44:55 +08:00
def rrb_Int : SI<opc, MRMSrcReg, (outs DstVT.RC:$dst),
(ins DstVT.RC:$src1, SrcRC:$src2, AVX512RC:$rc),
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
!strconcat(asm,
"\t{$src2, $rc, $src1, $dst|$dst, $src1, $rc, $src2}"),
AVX-512: Implemented cvtsi2ss/d cvtusi2ss/d instructions with round control for KNL. Added intrinsics for cvtsi2ss/d instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D10430 llvm-svn: 239694
2015-06-14 20:44:55 +08:00
[(set DstVT.RC:$dst,
(OpNode (DstVT.VT DstVT.RC:$src1),
SrcRC:$src2,
(i32 imm:$rc)))]>, EVEX_4V, EVEX_B, EVEX_RC;
}
multiclass avx512_vcvtsi_common<bits<8> opc, SDNode OpNode, RegisterClass SrcRC,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
X86VectorVTInfo DstVT, X86MemOperand x86memop,
PatFrag ld_frag, string asm> {
defm NAME : avx512_vcvtsi_round<opc, OpNode, SrcRC, DstVT, asm>,
avx512_vcvtsi<opc, OpNode, SrcRC, DstVT, x86memop, ld_frag, asm>,
VEX_LIG;
AVX-512: Implemented cvtsi2ss/d cvtusi2ss/d instructions with round control for KNL. Added intrinsics for cvtsi2ss/d instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D10430 llvm-svn: 239694
2015-06-14 20:44:55 +08:00
}
Add missing HasAVX512 predicate. This was only working because AVX had cheaper rules in all cases. I'm sure there are other places in this file where predicates are missing. llvm-svn: 192276
2013-10-09 13:11:10 +08:00
let Predicates = [HasAVX512] in {
AVX-512: Implemented cvtsi2ss/d cvtusi2ss/d instructions with round control for KNL. Added intrinsics for cvtsi2ss/d instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D10430 llvm-svn: 239694
2015-06-14 20:44:55 +08:00
defm VCVTSI2SSZ : avx512_vcvtsi_common<0x2A, X86SintToFpRnd, GR32,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
v4f32x_info, i32mem, loadi32, "cvtsi2ss{l}">,
XS, EVEX_CD8<32, CD8VT1>;
AVX-512: Implemented cvtsi2ss/d cvtusi2ss/d instructions with round control for KNL. Added intrinsics for cvtsi2ss/d instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D10430 llvm-svn: 239694
2015-06-14 20:44:55 +08:00
defm VCVTSI642SSZ: avx512_vcvtsi_common<0x2A, X86SintToFpRnd, GR64,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
v4f32x_info, i64mem, loadi64, "cvtsi2ss{q}">,
XS, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Implemented cvtsi2ss/d cvtusi2ss/d instructions with round control for KNL. Added intrinsics for cvtsi2ss/d instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D10430 llvm-svn: 239694
2015-06-14 20:44:55 +08:00
defm VCVTSI2SDZ : avx512_vcvtsi_common<0x2A, X86SintToFpRnd, GR32,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
v2f64x_info, i32mem, loadi32, "cvtsi2sd{l}">,
XD, EVEX_CD8<32, CD8VT1>;
AVX-512: Implemented cvtsi2ss/d cvtusi2ss/d instructions with round control for KNL. Added intrinsics for cvtsi2ss/d instructions. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D10430 llvm-svn: 239694
2015-06-14 20:44:55 +08:00
defm VCVTSI642SDZ: avx512_vcvtsi_common<0x2A, X86SintToFpRnd, GR64,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
v2f64x_info, i64mem, loadi64, "cvtsi2sd{q}">,
XD, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(f32 (sint_to_fp (loadi32 addr:$src))),
(VCVTSI2SSZrm (f32 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f32 (sint_to_fp (loadi64 addr:$src))),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
(VCVTSI642SSZrm (f32 (IMPLICIT_DEF)), addr:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(f64 (sint_to_fp (loadi32 addr:$src))),
(VCVTSI2SDZrm (f64 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f64 (sint_to_fp (loadi64 addr:$src))),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
(VCVTSI642SDZrm (f64 (IMPLICIT_DEF)), addr:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(f32 (sint_to_fp GR32:$src)),
(VCVTSI2SSZrr (f32 (IMPLICIT_DEF)), GR32:$src)>;
def : Pat<(f32 (sint_to_fp GR64:$src)),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
(VCVTSI642SSZrr (f32 (IMPLICIT_DEF)), GR64:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(f64 (sint_to_fp GR32:$src)),
(VCVTSI2SDZrr (f64 (IMPLICIT_DEF)), GR32:$src)>;
def : Pat<(f64 (sint_to_fp GR64:$src)),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
(VCVTSI642SDZrr (f64 (IMPLICIT_DEF)), GR64:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTUSI2SSZ : avx512_vcvtsi_common<0x7B, X86UintToFpRnd, GR32,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
v4f32x_info, i32mem, loadi32,
"cvtusi2ss{l}">, XS, EVEX_CD8<32, CD8VT1>;
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTUSI642SSZ : avx512_vcvtsi_common<0x7B, X86UintToFpRnd, GR64,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
v4f32x_info, i64mem, loadi64, "cvtusi2ss{q}">,
XS, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTUSI2SDZ : avx512_vcvtsi<0x7B, X86UintToFpRnd, GR32, v2f64x_info,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
i32mem, loadi32, "cvtusi2sd{l}">,
XD, VEX_LIG, EVEX_CD8<32, CD8VT1>;
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTUSI642SDZ : avx512_vcvtsi_common<0x7B, X86UintToFpRnd, GR64,
AVX-512: cvtusi2ss/d intrinsics. Change builtin function name and signature ( add third parameter - rounding mode ). Added tests for intrinsics. Differential Revision: http://reviews.llvm.org/D10473 llvm-svn: 239888
2015-06-17 15:23:57 +08:00
v2f64x_info, i64mem, loadi64, "cvtusi2sd{q}">,
XD, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
def : Pat<(f32 (uint_to_fp (loadi32 addr:$src))),
(VCVTUSI2SSZrm (f32 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f32 (uint_to_fp (loadi64 addr:$src))),
(VCVTUSI642SSZrm (f32 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f64 (uint_to_fp (loadi32 addr:$src))),
(VCVTUSI2SDZrm (f64 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f64 (uint_to_fp (loadi64 addr:$src))),
(VCVTUSI642SDZrm (f64 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f32 (uint_to_fp GR32:$src)),
(VCVTUSI2SSZrr (f32 (IMPLICIT_DEF)), GR32:$src)>;
def : Pat<(f32 (uint_to_fp GR64:$src)),
(VCVTUSI642SSZrr (f32 (IMPLICIT_DEF)), GR64:$src)>;
def : Pat<(f64 (uint_to_fp GR32:$src)),
(VCVTUSI2SDZrr (f64 (IMPLICIT_DEF)), GR32:$src)>;
def : Pat<(f64 (uint_to_fp GR64:$src)),
(VCVTUSI642SDZrr (f64 (IMPLICIT_DEF)), GR64:$src)>;
Add missing HasAVX512 predicate. This was only working because AVX had cheaper rules in all cases. I'm sure there are other places in this file where predicates are missing. llvm-svn: 192276
2013-10-09 13:11:10 +08:00
}
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Scalar convert from float/double to integer
//===----------------------------------------------------------------------===//
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
multiclass avx512_cvt_s_int_round<bits<8> opc, X86VectorVTInfo SrcVT ,
X86VectorVTInfo DstVT, SDNode OpNode, string asm> {
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
let hasSideEffects = 0, Predicates = [HasAVX512] in {
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
def rr : SI<opc, MRMSrcReg, (outs DstVT.RC:$dst), (ins SrcVT.RC:$src),
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
[(set DstVT.RC:$dst, (OpNode (SrcVT.VT SrcVT.RC:$src),(i32 FROUND_CURRENT)))]>,
EVEX, VEX_LIG;
def rb : SI<opc, MRMSrcReg, (outs DstVT.RC:$dst), (ins SrcVT.RC:$src, AVX512RC:$rc),
!strconcat(asm,"\t{$rc, $src, $dst|$dst, $src, $rc}"),
[(set DstVT.RC:$dst, (OpNode (SrcVT.VT SrcVT.RC:$src),(i32 imm:$rc)))]>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
EVEX, VEX_LIG, EVEX_B, EVEX_RC;
let mayLoad = 1 in
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
def rm : SI<opc, MRMSrcMem, (outs DstVT.RC:$dst), (ins SrcVT.ScalarMemOp:$src),
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
[(set DstVT.RC:$dst, (OpNode
(SrcVT.VT (scalar_to_vector (SrcVT.ScalarLdFrag addr:$src))),
(i32 FROUND_CURRENT)))]>,
EVEX, VEX_LIG;
} // hasSideEffects = 0, Predicates = [HasAVX512]
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
}
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
// Convert float/double to signed/unsigned int 32/64
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
defm VCVTSS2SIZ: avx512_cvt_s_int_round<0x2D, f32x_info, i32x_info,
X86cvtss2si, "cvtss2si">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XS, EVEX_CD8<32, CD8VT1>;
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
defm VCVTSS2SI64Z: avx512_cvt_s_int_round<0x2D, f32x_info, i64x_info,
X86cvtss2si, "cvtss2si">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XS, VEX_W, EVEX_CD8<32, CD8VT1>;
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
defm VCVTSS2USIZ: avx512_cvt_s_int_round<0x79, f32x_info, i32x_info,
X86cvtss2usi, "cvtss2usi">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XS, EVEX_CD8<32, CD8VT1>;
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
defm VCVTSS2USI64Z: avx512_cvt_s_int_round<0x79, f32x_info, i64x_info,
X86cvtss2usi, "cvtss2usi">, XS, VEX_W,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<32, CD8VT1>;
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
defm VCVTSD2SIZ: avx512_cvt_s_int_round<0x2D, f64x_info, i32x_info,
X86cvtsd2si, "cvtsd2si">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XD, EVEX_CD8<64, CD8VT1>;
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
defm VCVTSD2SI64Z: avx512_cvt_s_int_round<0x2D, f64x_info, i64x_info,
X86cvtsd2si, "cvtsd2si">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XD, VEX_W, EVEX_CD8<64, CD8VT1>;
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
defm VCVTSD2USIZ: avx512_cvt_s_int_round<0x79, f64x_info, i32x_info,
X86cvtsd2usi, "cvtsd2usi">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XD, EVEX_CD8<64, CD8VT1>;
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
defm VCVTSD2USI64Z: avx512_cvt_s_int_round<0x79, f64x_info, i64x_info,
X86cvtsd2usi, "cvtsd2usi">, XD, VEX_W,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<64, CD8VT1>;
[X86][AVX512] add intrinsics of Scalar FP to integer conversion with rounding mode Differential Revision: http://reviews.llvm.org/D16629 llvm-svn: 260033
2016-02-07 22:59:13 +08:00
// The SSE version of these instructions are disabled for AVX512.
// Therefore, the SSE intrinsics are mapped to the AVX512 instructions.
let Predicates = [HasAVX512] in {
def : Pat<(i32 (int_x86_sse_cvtss2si (v4f32 VR128X:$src))),
(VCVTSS2SIZrr (COPY_TO_REGCLASS VR128X:$src, FR32X))>;
def : Pat<(i64 (int_x86_sse_cvtss2si64 (v4f32 VR128X:$src))),
(VCVTSS2SI64Zrr (COPY_TO_REGCLASS VR128X:$src, FR32X))>;
def : Pat<(i32 (int_x86_sse2_cvtsd2si (v2f64 VR128X:$src))),
(VCVTSD2SIZrr (COPY_TO_REGCLASS VR128X:$src, FR64X))>;
def : Pat<(i64 (int_x86_sse2_cvtsd2si64 (v2f64 VR128X:$src))),
(VCVTSD2SI64Zrr (COPY_TO_REGCLASS VR128X:$src, FR64X))>;
} // HasAVX512
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
let isCodeGenOnly = 1 , Predicates = [HasAVX512] in {
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
defm Int_VCVTSI2SSZ : sse12_cvt_sint_3addr<0x2A, GR32, VR128X,
int_x86_sse_cvtsi2ss, i32mem, loadi32, "cvtsi2ss{l}",
SSE_CVT_Scalar, 0>, XS, EVEX_4V;
defm Int_VCVTSI2SS64Z : sse12_cvt_sint_3addr<0x2A, GR64, VR128X,
int_x86_sse_cvtsi642ss, i64mem, loadi64, "cvtsi2ss{q}",
SSE_CVT_Scalar, 0>, XS, EVEX_4V, VEX_W;
defm Int_VCVTSI2SDZ : sse12_cvt_sint_3addr<0x2A, GR32, VR128X,
int_x86_sse2_cvtsi2sd, i32mem, loadi32, "cvtsi2sd{l}",
SSE_CVT_Scalar, 0>, XD, EVEX_4V;
defm Int_VCVTSI2SD64Z : sse12_cvt_sint_3addr<0x2A, GR64, VR128X,
int_x86_sse2_cvtsi642sd, i64mem, loadi64, "cvtsi2sd{q}",
SSE_CVT_Scalar, 0>, XD, EVEX_4V, VEX_W;
defm Int_VCVTUSI2SDZ : sse12_cvt_sint_3addr<0x2A, GR32, VR128X,
int_x86_avx512_cvtusi2sd, i32mem, loadi32, "cvtusi2sd{l}",
SSE_CVT_Scalar, 0>, XD, EVEX_4V;
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
} // isCodeGenOnly = 1, Predicates = [HasAVX512]
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
// Convert float/double to signed/unsigned int 32/64 with truncation
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_cvt_s_all<bits<8> opc, string asm, X86VectorVTInfo _SrcRC,
X86VectorVTInfo _DstRC, SDNode OpNode,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
SDNode OpNodeRnd>{
let Predicates = [HasAVX512] in {
def rr : SI<opc, MRMSrcReg, (outs _DstRC.RC:$dst), (ins _SrcRC.FRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
[(set _DstRC.RC:$dst, (OpNode _SrcRC.FRC:$src))]>, EVEX;
def rb : SI<opc, MRMSrcReg, (outs _DstRC.RC:$dst), (ins _SrcRC.FRC:$src),
!strconcat(asm,"\t{{sae}, $src, $dst|$dst, $src, {sae}}"),
[]>, EVEX, EVEX_B;
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
def rm : SI<opc, MRMSrcMem, (outs _DstRC.RC:$dst), (ins _SrcRC.ScalarMemOp:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
[(set _DstRC.RC:$dst, (OpNode (_SrcRC.ScalarLdFrag addr:$src)))]>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
EVEX;
let isCodeGenOnly = 1,hasSideEffects = 0 in {
def rr_Int : SI<opc, MRMSrcReg, (outs _DstRC.RC:$dst), (ins _SrcRC.RC:$src),
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
[(set _DstRC.RC:$dst, (OpNodeRnd _SrcRC.RC:$src,
(i32 FROUND_CURRENT)))]>, EVEX, VEX_LIG;
def rb_Int : SI<opc, MRMSrcReg, (outs _DstRC.RC:$dst), (ins _SrcRC.RC:$src),
!strconcat(asm,"\t{{sae}, $src, $dst|$dst, $src, {sae}}"),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
[(set _DstRC.RC:$dst, (OpNodeRnd _SrcRC.RC:$src,
(i32 FROUND_NO_EXC)))]>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
EVEX,VEX_LIG , EVEX_B;
let mayLoad = 1 in
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
def rm_Int : SI<opc, MRMSrcMem, (outs _DstRC.RC:$dst),
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
(ins _SrcRC.MemOp:$src),
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
[]>, EVEX, VEX_LIG;
} // isCodeGenOnly = 1, hasSideEffects = 0
} //HasAVX512
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTTSS2SIZ: avx512_cvt_s_all<0x2C, "cvttss2si", f32x_info, i32x_info,
fp_to_sint,X86cvttss2IntRnd>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
XS, EVEX_CD8<32, CD8VT1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTTSS2SI64Z: avx512_cvt_s_all<0x2C, "cvttss2si", f32x_info, i64x_info,
fp_to_sint,X86cvttss2IntRnd>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
VEX_W, XS, EVEX_CD8<32, CD8VT1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTTSD2SIZ: avx512_cvt_s_all<0x2C, "cvttsd2si", f64x_info, i32x_info,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
fp_to_sint,X86cvttsd2IntRnd>,
XD, EVEX_CD8<64, CD8VT1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTTSD2SI64Z: avx512_cvt_s_all<0x2C, "cvttsd2si", f64x_info, i64x_info,
fp_to_sint,X86cvttsd2IntRnd>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
VEX_W, XD, EVEX_CD8<64, CD8VT1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTTSS2USIZ: avx512_cvt_s_all<0x78, "cvttss2usi", f32x_info, i32x_info,
fp_to_uint,X86cvttss2UIntRnd>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
XS, EVEX_CD8<32, CD8VT1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTTSS2USI64Z: avx512_cvt_s_all<0x78, "cvttss2usi", f32x_info, i64x_info,
fp_to_uint,X86cvttss2UIntRnd>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
XS,VEX_W, EVEX_CD8<32, CD8VT1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTTSD2USIZ: avx512_cvt_s_all<0x78, "cvttsd2usi", f64x_info, i32x_info,
fp_to_uint,X86cvttsd2UIntRnd>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
XD, EVEX_CD8<64, CD8VT1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTTSD2USI64Z: avx512_cvt_s_all<0x78, "cvttsd2usi", f64x_info, i64x_info,
fp_to_uint,X86cvttsd2UIntRnd>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
XD, VEX_W, EVEX_CD8<64, CD8VT1>;
let Predicates = [HasAVX512] in {
def : Pat<(i32 (int_x86_sse_cvttss2si (v4f32 VR128X:$src))),
(VCVTTSS2SIZrr_Int (COPY_TO_REGCLASS VR128X:$src, FR32X))>;
def : Pat<(i64 (int_x86_sse_cvttss2si64 (v4f32 VR128X:$src))),
(VCVTTSS2SI64Zrr_Int (COPY_TO_REGCLASS VR128X:$src, FR32X))>;
def : Pat<(i32 (int_x86_sse2_cvttsd2si (v2f64 VR128X:$src))),
(VCVTTSD2SIZrr_Int (COPY_TO_REGCLASS VR128X:$src, FR64X))>;
def : Pat<(i64 (int_x86_sse2_cvttsd2si64 (v2f64 VR128X:$src))),
(VCVTTSD2SI64Zrr_Int (COPY_TO_REGCLASS VR128X:$src, FR64X))>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
} // HasAVX512
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Convert form float to double and back
//===----------------------------------------------------------------------===//
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
multiclass avx512_cvt_fp_scalar<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
X86VectorVTInfo _Src, SDNode OpNode> {
defm rr : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
[AVX512] Fix up types for arguments of int_x86_avx512_mask_cvtsd2ss_round and int_x86_avx512_mask_cvtss2sd_round. Only the argument being converted should be a different type. The other 2 argument should have the same type as the result. llvm-svn: 268891
2016-05-09 13:34:12 +08:00
(ins _.RC:$src1, _Src.RC:$src2), OpcodeStr,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
"$src2, $src1", "$src1, $src2",
[AVX512] Fix up types for arguments of int_x86_avx512_mask_cvtsd2ss_round and int_x86_avx512_mask_cvtss2sd_round. Only the argument being converted should be a different type. The other 2 argument should have the same type as the result. llvm-svn: 268891
2016-05-09 13:34:12 +08:00
(_.VT (OpNode (_.VT _.RC:$src1),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(_Src.VT _Src.RC:$src2)))>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
EVEX_4V, VEX_LIG, Sched<[WriteCvtF2F]>;
defm rm : AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _Src.RC:$src1, _Src.ScalarMemOp:$src2), OpcodeStr,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
"$src2, $src1", "$src1, $src2",
[AVX512] Fix up types for arguments of int_x86_avx512_mask_cvtsd2ss_round and int_x86_avx512_mask_cvtss2sd_round. Only the argument being converted should be a different type. The other 2 argument should have the same type as the result. llvm-svn: 268891
2016-05-09 13:34:12 +08:00
(_.VT (OpNode (_.VT _.RC:$src1),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(_Src.VT (scalar_to_vector
(_Src.ScalarLdFrag addr:$src2)))))>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
EVEX_4V, VEX_LIG, Sched<[WriteCvtF2FLd, ReadAfterLd]>;
}
// Scalar Coversion with SAE - suppress all exceptions
multiclass avx512_cvt_fp_sae_scalar<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
X86VectorVTInfo _Src, SDNode OpNodeRnd> {
defm rrb : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
[AVX512] Fix up types for arguments of int_x86_avx512_mask_cvtsd2ss_round and int_x86_avx512_mask_cvtss2sd_round. Only the argument being converted should be a different type. The other 2 argument should have the same type as the result. llvm-svn: 268891
2016-05-09 13:34:12 +08:00
(ins _.RC:$src1, _Src.RC:$src2), OpcodeStr,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
"{sae}, $src2, $src1", "$src1, $src2, {sae}",
[AVX512] Fix up types for arguments of int_x86_avx512_mask_cvtsd2ss_round and int_x86_avx512_mask_cvtss2sd_round. Only the argument being converted should be a different type. The other 2 argument should have the same type as the result. llvm-svn: 268891
2016-05-09 13:34:12 +08:00
(_.VT (OpNodeRnd (_.VT _.RC:$src1),
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
(_Src.VT _Src.RC:$src2),
(i32 FROUND_NO_EXC)))>,
EVEX_4V, VEX_LIG, EVEX_B;
}
// Scalar Conversion with rounding control (RC)
multiclass avx512_cvt_fp_rc_scalar<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
X86VectorVTInfo _Src, SDNode OpNodeRnd> {
defm rrb : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
[AVX512] Fix up types for arguments of int_x86_avx512_mask_cvtsd2ss_round and int_x86_avx512_mask_cvtss2sd_round. Only the argument being converted should be a different type. The other 2 argument should have the same type as the result. llvm-svn: 268891
2016-05-09 13:34:12 +08:00
(ins _.RC:$src1, _Src.RC:$src2, AVX512RC:$rc), OpcodeStr,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
"$rc, $src2, $src1", "$src1, $src2, $rc",
[AVX512] Fix up types for arguments of int_x86_avx512_mask_cvtsd2ss_round and int_x86_avx512_mask_cvtss2sd_round. Only the argument being converted should be a different type. The other 2 argument should have the same type as the result. llvm-svn: 268891
2016-05-09 13:34:12 +08:00
(_.VT (OpNodeRnd (_.VT _.RC:$src1),
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
(_Src.VT _Src.RC:$src2), (i32 imm:$rc)))>,
EVEX_4V, VEX_LIG, Sched<[WriteCvtF2FLd, ReadAfterLd]>,
EVEX_B, EVEX_RC;
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_cvt_fp_scalar_sd2ss<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd, X86VectorVTInfo _src,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
X86VectorVTInfo _dst> {
let Predicates = [HasAVX512] in {
defm Z : avx512_cvt_fp_scalar<opc, OpcodeStr, _dst, _src, OpNode>,
avx512_cvt_fp_rc_scalar<opc, OpcodeStr, _dst, _src,
OpNodeRnd>, VEX_W, EVEX_CD8<64, CD8VT1>,
EVEX_V512, XD;
}
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_cvt_fp_scalar_ss2sd<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd, X86VectorVTInfo _src,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
X86VectorVTInfo _dst> {
let Predicates = [HasAVX512] in {
defm Z : avx512_cvt_fp_scalar<opc, OpcodeStr, _dst, _src, OpNode>,
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
avx512_cvt_fp_sae_scalar<opc, OpcodeStr, _dst, _src, OpNodeRnd>,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
EVEX_CD8<32, CD8VT1>, XS, EVEX_V512;
}
}
defm VCVTSD2SS : avx512_cvt_fp_scalar_sd2ss<0x5A, "vcvtsd2ss", X86fround,
X86froundRnd, f64x_info, f32x_info>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTSS2SD : avx512_cvt_fp_scalar_ss2sd<0x5A, "vcvtss2sd", X86fpext,
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
X86fpextRnd,f32x_info, f64x_info >;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
def : Pat<(f64 (fextend FR32X:$src)),
(COPY_TO_REGCLASS (VCVTSS2SDZrr (COPY_TO_REGCLASS FR32X:$src, VR128X),
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
(COPY_TO_REGCLASS FR32X:$src, VR128X)), VR128X)>,
Requires<[HasAVX512]>;
def : Pat<(f64 (fextend (loadf32 addr:$src))),
(COPY_TO_REGCLASS (VCVTSS2SDZrm (v4f32 (IMPLICIT_DEF)), addr:$src), VR128X)>,
Requires<[HasAVX512]>;
def : Pat<(f64 (extloadf32 addr:$src)),
(COPY_TO_REGCLASS (VCVTSS2SDZrm (v4f32 (IMPLICIT_DEF)), addr:$src), VR128X)>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Requires<[HasAVX512, OptForSize]>;
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
def : Pat<(f64 (extloadf32 addr:$src)),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(COPY_TO_REGCLASS (VCVTSS2SDZrr (v4f32 (IMPLICIT_DEF)),
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
(COPY_TO_REGCLASS (VMOVSSZrm addr:$src), VR128X)), VR128X)>,
Requires<[HasAVX512, OptForSpeed]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
def : Pat<(f32 (fround FR64X:$src)),
(COPY_TO_REGCLASS (VCVTSD2SSZrr (COPY_TO_REGCLASS FR64X:$src, VR128X),
[X86][AVX512] extend support in Scalar conversion add scalar FP to Int conversion with truncation intrinsics add scalar conversion FP32 from/to FP64 intrinsics add rounding mode and SAE mode encoding for these intrinsics Differential Revision: http://reviews.llvm.org/D12665 llvm-svn: 248117
2015-09-20 22:31:19 +08:00
(COPY_TO_REGCLASS FR64X:$src, VR128X)), VR128X)>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Requires<[HasAVX512]>;
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Vector convert from signed/unsigned integer to float/double
// and from float/double to signed/unsigned integer
//===----------------------------------------------------------------------===//
multiclass avx512_vcvt_fp<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
X86VectorVTInfo _Src, SDNode OpNode,
string Broadcast = _.BroadcastStr,
string Alias = ""> {
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _Src.RC:$src), OpcodeStr, "$src", "$src",
(_.VT (OpNode (_Src.VT _Src.RC:$src)))>, EVEX;
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _Src.MemOp:$src), OpcodeStr#Alias, "$src", "$src",
(_.VT (OpNode (_Src.VT
(bitconvert (_Src.LdFrag addr:$src)))))>, EVEX;
defm rmb : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _Src.ScalarMemOp:$src), OpcodeStr,
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
"${src}"##Broadcast, "${src}"##Broadcast,
(_.VT (OpNode (_Src.VT
(X86VBroadcast (_Src.ScalarLdFrag addr:$src)))
))>, EVEX, EVEX_B;
}
// Coversion with SAE - suppress all exceptions
multiclass avx512_vcvt_fp_sae<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
X86VectorVTInfo _Src, SDNode OpNodeRnd> {
defm rrb : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _Src.RC:$src), OpcodeStr,
"{sae}, $src", "$src, {sae}",
(_.VT (OpNodeRnd (_Src.VT _Src.RC:$src),
(i32 FROUND_NO_EXC)))>,
EVEX, EVEX_B;
}
// Conversion with rounding control (RC)
multiclass avx512_vcvt_fp_rc<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
X86VectorVTInfo _Src, SDNode OpNodeRnd> {
defm rrb : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _Src.RC:$src, AVX512RC:$rc), OpcodeStr,
"$rc, $src", "$src, $rc",
(_.VT (OpNodeRnd (_Src.VT _Src.RC:$src), (i32 imm:$rc)))>,
EVEX, EVEX_B, EVEX_RC;
}
// Extend Float to Double
multiclass avx512_cvtps2pd<bits<8> opc, string OpcodeStr> {
let Predicates = [HasAVX512] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8f64_info, v8f32x_info, fextend>,
avx512_vcvt_fp_sae<opc, OpcodeStr, v8f64_info, v8f32x_info,
X86vfpextRnd>, EVEX_V512;
}
let Predicates = [HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2f64x_info, v4f32x_info,
X86vfpext, "{1to2}">, EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4f64x_info, v4f32x_info, fextend>,
EVEX_V256;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
// Truncate Double to Float
multiclass avx512_cvtpd2ps<bits<8> opc, string OpcodeStr> {
let Predicates = [HasAVX512] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8f32x_info, v8f64_info, fround>,
avx512_vcvt_fp_rc<opc, OpcodeStr, v8f32x_info, v8f64_info,
X86vfproundRnd>, EVEX_V512;
}
let Predicates = [HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4f32x_info, v2f64x_info,
X86vfpround, "{1to2}", "{x}">, EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4f32x_info, v4f64x_info, fround,
"{1to4}", "{y}">, EVEX_V256;
}
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
}
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTPD2PS : avx512_cvtpd2ps<0x5A, "vcvtpd2ps">,
VEX_W, PD, EVEX_CD8<64, CD8VF>;
defm VCVTPS2PD : avx512_cvtps2pd<0x5A, "vcvtps2pd">,
PS, EVEX_CD8<32, CD8VH>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (extloadv8f32 addr:$src)),
(VCVTPS2PDZrm addr:$src)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
let Predicates = [HasVLX] in {
def : Pat<(v4f64 (extloadv4f32 addr:$src)),
(VCVTPS2PDZ256rm addr:$src)>;
}
AVX-512: added intrinsic vcvtpd2ps (with rounding mode and without) llvm-svn: 198593
2014-01-06 16:45:54 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
// Convert Signed/Unsigned Doubleword to Double
multiclass avx512_cvtdq2pd<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNode128> {
// No rounding in this op
let Predicates = [HasAVX512] in
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8f64_info, v8i32x_info, OpNode>,
EVEX_V512;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
let Predicates = [HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2f64x_info, v4i32x_info,
OpNode128, "{1to2}">, EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4f64x_info, v4i32x_info, OpNode>,
EVEX_V256;
}
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
// Convert Signed/Unsigned Doubleword to Float
multiclass avx512_cvtdq2ps<bits<8> opc, string OpcodeStr, SDNode OpNode,
SDNode OpNodeRnd> {
let Predicates = [HasAVX512] in
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v16f32_info, v16i32_info, OpNode>,
avx512_vcvt_fp_rc<opc, OpcodeStr, v16f32_info, v16i32_info,
OpNodeRnd>, EVEX_V512;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
let Predicates = [HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4f32x_info, v4i32x_info, OpNode>,
EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v8f32x_info, v8i32x_info, OpNode>,
EVEX_V256;
}
}
// Convert Float to Signed/Unsigned Doubleword with truncation
multiclass avx512_cvttps2dq<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasAVX512] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v16i32_info, v16f32_info, OpNode>,
avx512_vcvt_fp_sae<opc, OpcodeStr, v16i32_info, v16f32_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v4f32x_info, OpNode>,
EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v8i32x_info, v8f32x_info, OpNode>,
EVEX_V256;
}
}
// Convert Float to Signed/Unsigned Doubleword
multiclass avx512_cvtps2dq<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasAVX512] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v16i32_info, v16f32_info, OpNode>,
avx512_vcvt_fp_rc<opc, OpcodeStr, v16i32_info, v16f32_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v4f32x_info, OpNode>,
EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v8i32x_info, v8f32x_info, OpNode>,
EVEX_V256;
}
}
// Convert Double to Signed/Unsigned Doubleword with truncation
multiclass avx512_cvttpd2dq<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasAVX512] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8i32x_info, v8f64_info, OpNode>,
avx512_vcvt_fp_sae<opc, OpcodeStr, v8i32x_info, v8f64_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasVLX] in {
// we need "x"/"y" suffixes in order to distinguish between 128 and 256
// memory forms of these instructions in Asm Parcer. They have the same
// dest type - 'v4i32x_info'. We also specify the broadcast string explicitly
// due to the same reason.
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v2f64x_info, OpNode,
"{1to2}", "{x}">, EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v4f64x_info, OpNode,
"{1to4}", "{y}">, EVEX_V256;
}
}
// Convert Double to Signed/Unsigned Doubleword
multiclass avx512_cvtpd2dq<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasAVX512] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8i32x_info, v8f64_info, OpNode>,
avx512_vcvt_fp_rc<opc, OpcodeStr, v8i32x_info, v8f64_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasVLX] in {
// we need "x"/"y" suffixes in order to distinguish between 128 and 256
// memory forms of these instructions in Asm Parcer. They have the same
// dest type - 'v4i32x_info'. We also specify the broadcast string explicitly
// due to the same reason.
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v2f64x_info, OpNode,
"{1to2}", "{x}">, EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i32x_info, v4f64x_info, OpNode,
"{1to4}", "{y}">, EVEX_V256;
}
}
// Convert Double to Signed/Unsigned Quardword
multiclass avx512_cvtpd2qq<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasDQI] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8i64_info, v8f64_info, OpNode>,
avx512_vcvt_fp_rc<opc, OpcodeStr, v8i64_info, v8f64_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasDQI, HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2i64x_info, v2f64x_info, OpNode>,
EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i64x_info, v4f64x_info, OpNode>,
EVEX_V256;
}
}
// Convert Double to Signed/Unsigned Quardword with truncation
multiclass avx512_cvttpd2qq<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasDQI] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8i64_info, v8f64_info, OpNode>,
avx512_vcvt_fp_sae<opc, OpcodeStr, v8i64_info, v8f64_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasDQI, HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2i64x_info, v2f64x_info, OpNode>,
EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i64x_info, v4f64x_info, OpNode>,
EVEX_V256;
}
}
// Convert Signed/Unsigned Quardword to Double
multiclass avx512_cvtqq2pd<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasDQI] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8f64_info, v8i64_info, OpNode>,
avx512_vcvt_fp_rc<opc, OpcodeStr, v8f64_info, v8i64_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasDQI, HasVLX] in {
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2f64x_info, v2i64x_info, OpNode>,
EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4f64x_info, v4i64x_info, OpNode>,
EVEX_V256;
}
}
// Convert Float to Signed/Unsigned Quardword
multiclass avx512_cvtps2qq<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasDQI] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8i64_info, v8f32x_info, OpNode>,
avx512_vcvt_fp_rc<opc, OpcodeStr, v8i64_info, v8f32x_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasDQI, HasVLX] in {
// Explicitly specified broadcast string, since we take only 2 elements
// from v4f32x_info source
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2i64x_info, v4f32x_info, OpNode,
"{1to2}">, EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i64x_info, v4f32x_info, OpNode>,
EVEX_V256;
}
}
// Convert Float to Signed/Unsigned Quardword with truncation
multiclass avx512_cvttps2qq<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasDQI] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8i64_info, v8f32x_info, OpNode>,
avx512_vcvt_fp_sae<opc, OpcodeStr, v8i64_info, v8f32x_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasDQI, HasVLX] in {
// Explicitly specified broadcast string, since we take only 2 elements
// from v4f32x_info source
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v2i64x_info, v4f32x_info, OpNode,
"{1to2}">, EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4i64x_info, v4f32x_info, OpNode>,
EVEX_V256;
}
}
// Convert Signed/Unsigned Quardword to Float
multiclass avx512_cvtqq2ps<bits<8> opc, string OpcodeStr,
SDNode OpNode, SDNode OpNodeRnd> {
let Predicates = [HasDQI] in {
defm Z : avx512_vcvt_fp<opc, OpcodeStr, v8f32x_info, v8i64_info, OpNode>,
avx512_vcvt_fp_rc<opc, OpcodeStr, v8f32x_info, v8i64_info,
OpNodeRnd>, EVEX_V512;
}
let Predicates = [HasDQI, HasVLX] in {
// we need "x"/"y" suffixes in order to distinguish between 128 and 256
// memory forms of these instructions in Asm Parcer. They have the same
// dest type - 'v4i32x_info'. We also specify the broadcast string explicitly
// due to the same reason.
defm Z128 : avx512_vcvt_fp<opc, OpcodeStr, v4f32x_info, v2i64x_info, OpNode,
"{1to2}", "{x}">, EVEX_V128;
defm Z256 : avx512_vcvt_fp<opc, OpcodeStr, v4f32x_info, v4i64x_info, OpNode,
"{1to4}", "{y}">, EVEX_V256;
}
}
defm VCVTDQ2PD : avx512_cvtdq2pd<0xE6, "vcvtdq2pd", sint_to_fp, X86cvtdq2pd>, XS,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VH>;
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTDQ2PS : avx512_cvtdq2ps<0x5B, "vcvtdq2ps", sint_to_fp,
X86VSintToFpRnd>,
PS, EVEX_CD8<32, CD8VF>;
defm VCVTTPS2DQ : avx512_cvttps2dq<0x5B, "vcvttps2dq", fp_to_sint,
X86VFpToSintRnd>,
XS, EVEX_CD8<32, CD8VF>;
defm VCVTTPD2DQ : avx512_cvttpd2dq<0xE6, "vcvttpd2dq", fp_to_sint,
X86VFpToSintRnd>,
PD, VEX_W, EVEX_CD8<64, CD8VF>;
defm VCVTTPS2UDQ : avx512_cvttps2dq<0x78, "vcvttps2udq", fp_to_uint,
X86VFpToUintRnd>, PS,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VF>;
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTTPD2UDQ : avx512_cvttpd2dq<0x78, "vcvttpd2udq", fp_to_uint,
X86VFpToUintRnd>, PS, VEX_W,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>;
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTUDQ2PD : avx512_cvtdq2pd<0x7A, "vcvtudq2pd", uint_to_fp, X86cvtudq2pd>,
XS, EVEX_CD8<32, CD8VH>;
defm VCVTUDQ2PS : avx512_cvtdq2ps<0x7A, "vcvtudq2ps", uint_to_fp,
X86VUintToFpRnd>, XD,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VF>;
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTPS2DQ : avx512_cvtps2dq<0x5B, "vcvtps2dq", X86cvtps2Int,
X86cvtps2IntRnd>, PD, EVEX_CD8<32, CD8VF>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTPD2DQ : avx512_cvtpd2dq<0xE6, "vcvtpd2dq", X86cvtpd2Int,
X86cvtpd2IntRnd>, XD, VEX_W,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTPS2UDQ : avx512_cvtps2dq<0x79, "vcvtps2udq", X86cvtps2UInt,
X86cvtps2UIntRnd>,
PS, EVEX_CD8<32, CD8VF>;
defm VCVTPD2UDQ : avx512_cvtpd2dq<0x79, "vcvtpd2udq", X86cvtpd2UInt,
X86cvtpd2UIntRnd>, VEX_W,
PS, EVEX_CD8<64, CD8VF>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTPD2QQ : avx512_cvtpd2qq<0x7B, "vcvtpd2qq", X86cvtpd2Int,
X86cvtpd2IntRnd>, VEX_W,
PD, EVEX_CD8<64, CD8VF>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTPS2QQ : avx512_cvtps2qq<0x7B, "vcvtps2qq", X86cvtps2Int,
X86cvtps2IntRnd>, PD, EVEX_CD8<32, CD8VH>;
defm VCVTPD2UQQ : avx512_cvtpd2qq<0x79, "vcvtpd2uqq", X86cvtpd2UInt,
X86cvtpd2UIntRnd>, VEX_W,
PD, EVEX_CD8<64, CD8VF>;
defm VCVTPS2UQQ : avx512_cvtps2qq<0x79, "vcvtps2uqq", X86cvtps2UInt,
X86cvtps2UIntRnd>, PD, EVEX_CD8<32, CD8VH>;
defm VCVTTPD2QQ : avx512_cvttpd2qq<0x7A, "vcvttpd2qq", fp_to_sint,
X86VFpToSlongRnd>, VEX_W,
PD, EVEX_CD8<64, CD8VF>;
defm VCVTTPS2QQ : avx512_cvttps2qq<0x7A, "vcvttps2qq", fp_to_sint,
X86VFpToSlongRnd>, PD, EVEX_CD8<32, CD8VH>;
defm VCVTTPD2UQQ : avx512_cvttpd2qq<0x78, "vcvttpd2uqq", fp_to_uint,
X86VFpToUlongRnd>, VEX_W,
PD, EVEX_CD8<64, CD8VF>;
defm VCVTTPS2UQQ : avx512_cvttps2qq<0x78, "vcvttps2uqq", fp_to_uint,
X86VFpToUlongRnd>, PD, EVEX_CD8<32, CD8VH>;
defm VCVTQQ2PD : avx512_cvtqq2pd<0xE6, "vcvtqq2pd", sint_to_fp,
X86VSlongToFpRnd>, VEX_W, XS, EVEX_CD8<64, CD8VF>;
defm VCVTUQQ2PD : avx512_cvtqq2pd<0x7A, "vcvtuqq2pd", uint_to_fp,
X86VUlongToFpRnd>, VEX_W, XS, EVEX_CD8<64, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added all AVX-512 forms of Vector Convert for Float/Double/Int/Long types. In this patch I have only encoding. Intrinsics and DAG lowering will be in the next patch. I temporary removed the old intrinsics test (just to split this patch). Half types are not covered here. Differential Revision: http://reviews.llvm.org/D11134 llvm-svn: 242023
2015-07-13 21:26:20 +08:00
defm VCVTQQ2PS : avx512_cvtqq2ps<0x5B, "vcvtqq2ps", sint_to_fp,
X86VSlongToFpRnd>, VEX_W, PS, EVEX_CD8<64, CD8VF>;
defm VCVTUQQ2PS : avx512_cvtqq2ps<0x7A, "vcvtuqq2ps", uint_to_fp,
X86VUlongToFpRnd>, VEX_W, XD, EVEX_CD8<64, CD8VF>;
[X86] Pair a NoVLX with HasAVX512 to match the others and remove a unique predicate check in the isel tables. NFC llvm-svn: 254191
2015-11-27 13:44:02 +08:00
let Predicates = [HasAVX512, NoVLX] in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i32 (fp_to_uint (v8f32 VR256X:$src1))),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(EXTRACT_SUBREG (v16i32 (VCVTTPS2UDQZrr
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_ymm)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added fp_to_uint and uint_to_fp patterns. llvm-svn: 205754
2014-04-08 15:24:02 +08:00
def : Pat<(v4i32 (fp_to_uint (v4f32 VR128X:$src1))),
(EXTRACT_SUBREG (v16i32 (VCVTTPS2UDQZrr
(v16f32 (SUBREG_TO_REG (i32 0), VR128X:$src1, sub_xmm)))), sub_xmm)>;
AVX-512: fixed a bug in fp_to_uint pattern on KNL Fixed fp_to_uint instruction selection on KNL. One pattern was missing for <4 x double> to <4 x i32> Differential Revision: http://reviews.llvm.org/D18512 llvm-svn: 264701
2016-03-29 14:33:41 +08:00
def : Pat<(v4i32 (fp_to_uint (v4f64 VR256X:$src1))),
(EXTRACT_SUBREG (v8i32 (VCVTTPD2UDQZrr
(v8f64 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_xmm)>;
AVX-512: Added fp_to_uint and uint_to_fp patterns. llvm-svn: 205754
2014-04-08 15:24:02 +08:00
def : Pat<(v8f32 (uint_to_fp (v8i32 VR256X:$src1))),
(EXTRACT_SUBREG (v16f32 (VCVTUDQ2PSZrr
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_ymm)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added fp_to_uint and uint_to_fp patterns. llvm-svn: 205754
2014-04-08 15:24:02 +08:00
def : Pat<(v4f32 (uint_to_fp (v4i32 VR128X:$src1))),
(EXTRACT_SUBREG (v16f32 (VCVTUDQ2PSZrr
(v16i32 (SUBREG_TO_REG (i32 0), VR128X:$src1, sub_xmm)))), sub_xmm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Add pattern for unsigned v4i32->v4f64 convert on AVX512. llvm-svn: 211164
2014-06-18 22:04:37 +08:00
def : Pat<(v4f64 (uint_to_fp (v4i32 VR128X:$src1))),
(EXTRACT_SUBREG (v8f64 (VCVTUDQ2PDZrr
(v8i32 (SUBREG_TO_REG (i32 0), VR128X:$src1, sub_xmm)))), sub_ymm)>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(v8f32 (fround (loadv8f64 addr:$src))),
(VCVTPD2PSZrm addr:$src)>;
def : Pat<(v8f64 (extloadv8f32 addr:$src)),
(VCVTPS2PDZrm addr:$src)>;
}
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
//===----------------------------------------------------------------------===//
// Half precision conversion instructions
//===----------------------------------------------------------------------===//
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_cvtph2ps<X86VectorVTInfo _dest, X86VectorVTInfo _src,
[X86][AVX512] extend vcvtph2ps to support xmm/ymm and sae versions Differential Revision: http://reviews.llvm.org/D13945 llvm-svn: 251018
2015-10-22 22:01:16 +08:00
X86MemOperand x86memop, PatFrag ld_frag> {
defm rr : AVX512_maskable<0x13, MRMSrcReg, _dest ,(outs _dest.RC:$dst), (ins _src.RC:$src),
"vcvtph2ps", "$src", "$src",
(X86cvtph2ps (_src.VT _src.RC:$src),
(i32 FROUND_CURRENT))>, T8PD;
let hasSideEffects = 0, mayLoad = 1 in {
defm rm : AVX512_maskable<0x13, MRMSrcMem, _dest, (outs _dest.RC:$dst), (ins x86memop:$src),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
"vcvtph2ps", "$src", "$src",
[X86][AVX512] extend vcvtph2ps to support xmm/ymm and sae versions Differential Revision: http://reviews.llvm.org/D13945 llvm-svn: 251018
2015-10-22 22:01:16 +08:00
(X86cvtph2ps (_src.VT (bitconvert (ld_frag addr:$src))),
(i32 FROUND_CURRENT))>, T8PD;
}
}
[X86][AVX512] [X86][AVX512] add convert float to half convert float to half with mask/maskz for the reg to reg version and mask for the reg to mem version (there is no maskz version for reg to mem). Differential Revision: http://reviews.llvm.org/D14113 llvm-svn: 251409
2015-10-27 23:37:17 +08:00
multiclass avx512_cvtph2ps_sae<X86VectorVTInfo _dest, X86VectorVTInfo _src> {
[X86][AVX512] extend vcvtph2ps to support xmm/ymm and sae versions Differential Revision: http://reviews.llvm.org/D13945 llvm-svn: 251018
2015-10-22 22:01:16 +08:00
defm rb : AVX512_maskable<0x13, MRMSrcReg, _dest ,(outs _dest.RC:$dst), (ins _src.RC:$src),
"vcvtph2ps", "{sae}, $src", "$src, {sae}",
(X86cvtph2ps (_src.VT _src.RC:$src),
(i32 FROUND_NO_EXC))>, T8PD, EVEX_B;
}
let Predicates = [HasAVX512] in {
defm VCVTPH2PSZ : avx512_cvtph2ps<v16f32_info, v16i16x_info, f256mem, loadv4i64>,
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
avx512_cvtph2ps_sae<v16f32_info, v16i16x_info>,
[X86][AVX512] extend vcvtph2ps to support xmm/ymm and sae versions Differential Revision: http://reviews.llvm.org/D13945 llvm-svn: 251018
2015-10-22 22:01:16 +08:00
EVEX, EVEX_V512, EVEX_CD8<32, CD8VH>;
let Predicates = [HasVLX] in {
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VCVTPH2PSZ256 : avx512_cvtph2ps<v8f32x_info, v8i16x_info, f128mem,
[X86][AVX512] extend vcvtph2ps to support xmm/ymm and sae versions Differential Revision: http://reviews.llvm.org/D13945 llvm-svn: 251018
2015-10-22 22:01:16 +08:00
loadv2i64>,EVEX, EVEX_V256, EVEX_CD8<32, CD8VH>;
defm VCVTPH2PSZ128 : avx512_cvtph2ps<v4f32x_info, v8i16x_info, f64mem,
loadv2i64>, EVEX, EVEX_V128, EVEX_CD8<32, CD8VH>;
}
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_cvtps2ph<X86VectorVTInfo _dest, X86VectorVTInfo _src,
[X86][AVX512] [X86][AVX512] add convert float to half convert float to half with mask/maskz for the reg to reg version and mask for the reg to mem version (there is no maskz version for reg to mem). Differential Revision: http://reviews.llvm.org/D14113 llvm-svn: 251409
2015-10-27 23:37:17 +08:00
X86MemOperand x86memop> {
defm rr : AVX512_maskable<0x1D, MRMDestReg, _dest ,(outs _dest.RC:$dst),
(ins _src.RC:$src1, i32u8imm:$src2),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
"vcvtps2ph", "$src2, $src1", "$src1, $src2",
[X86][AVX512] [X86][AVX512] add convert float to half convert float to half with mask/maskz for the reg to reg version and mask for the reg to mem version (there is no maskz version for reg to mem). Differential Revision: http://reviews.llvm.org/D14113 llvm-svn: 251409
2015-10-27 23:37:17 +08:00
(X86cvtps2ph (_src.VT _src.RC:$src1),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(i32 imm:$src2),
[X86][AVX512] [X86][AVX512] add convert float to half convert float to half with mask/maskz for the reg to reg version and mask for the reg to mem version (there is no maskz version for reg to mem). Differential Revision: http://reviews.llvm.org/D14113 llvm-svn: 251409
2015-10-27 23:37:17 +08:00
(i32 FROUND_CURRENT))>, AVX512AIi8Base;
let hasSideEffects = 0, mayStore = 1 in {
def mr : AVX512AIi8<0x1D, MRMDestMem, (outs),
(ins x86memop:$dst, _src.RC:$src1, i32u8imm:$src2),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
"vcvtps2ph\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[X86][AVX512] [X86][AVX512] add convert float to half convert float to half with mask/maskz for the reg to reg version and mask for the reg to mem version (there is no maskz version for reg to mem). Differential Revision: http://reviews.llvm.org/D14113 llvm-svn: 251409
2015-10-27 23:37:17 +08:00
[(store (_dest.VT (X86cvtps2ph (_src.VT _src.RC:$src1),
(i32 imm:$src2), (i32 FROUND_CURRENT) )),
addr:$dst)]>;
def mrk : AVX512AIi8<0x1D, MRMDestMem, (outs),
(ins x86memop:$dst, _dest.KRCWM:$mask, _src.RC:$src1, i32u8imm:$src2),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
"vcvtps2ph\t{$src2, $src1, $dst {${mask}}|$dst {${mask}}, $src1, $src2}",
[X86][AVX512] [X86][AVX512] add convert float to half convert float to half with mask/maskz for the reg to reg version and mask for the reg to mem version (there is no maskz version for reg to mem). Differential Revision: http://reviews.llvm.org/D14113 llvm-svn: 251409
2015-10-27 23:37:17 +08:00
[]>, EVEX_K;
}
}
multiclass avx512_cvtps2ph_sae<X86VectorVTInfo _dest, X86VectorVTInfo _src> {
defm rb : AVX512_maskable<0x1D, MRMDestReg, _dest ,(outs _dest.RC:$dst),
(ins _src.RC:$src1, i32u8imm:$src2),
AVX512F: Add assembler Intel syntax tests for knl, fix minor bugs. Differential Revision: http://reviews.llvm.org/D17498 llvm-svn: 261521
2016-02-22 20:37:41 +08:00
"vcvtps2ph", "$src2, {sae}, $src1", "$src1, {sae}, $src2",
[X86][AVX512] [X86][AVX512] add convert float to half convert float to half with mask/maskz for the reg to reg version and mask for the reg to mem version (there is no maskz version for reg to mem). Differential Revision: http://reviews.llvm.org/D14113 llvm-svn: 251409
2015-10-27 23:37:17 +08:00
(X86cvtps2ph (_src.VT _src.RC:$src1),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(i32 imm:$src2),
[X86][AVX512] [X86][AVX512] add convert float to half convert float to half with mask/maskz for the reg to reg version and mask for the reg to mem version (there is no maskz version for reg to mem). Differential Revision: http://reviews.llvm.org/D14113 llvm-svn: 251409
2015-10-27 23:37:17 +08:00
(i32 FROUND_NO_EXC))>, EVEX_B, AVX512AIi8Base;
}
let Predicates = [HasAVX512] in {
defm VCVTPS2PHZ : avx512_cvtps2ph<v16i16x_info, v16f32_info, f256mem>,
avx512_cvtps2ph_sae<v16i16x_info, v16f32_info>,
EVEX, EVEX_V512, EVEX_CD8<32, CD8VH>;
let Predicates = [HasVLX] in {
defm VCVTPS2PHZ256 : avx512_cvtps2ph<v8i16x_info, v8f32x_info, f128mem>,
EVEX, EVEX_V256, EVEX_CD8<32, CD8VH>;
defm VCVTPS2PHZ128 : avx512_cvtps2ph<v8i16x_info, v4f32x_info, f128mem>,
EVEX, EVEX_V128, EVEX_CD8<32, CD8VH>;
}
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
}
[X86][AVX512] add comi with Sae add builtin_ia32_vcomisd and builtin_ia32_vcomisd Differential Revision: http://reviews.llvm.org/D14331 llvm-svn: 254493
2015-12-02 16:17:51 +08:00
// Unordered/Ordered scalar fp compare with Sea and set EFLAGS
multiclass avx512_ord_cmp_sae<bits<8> opc, X86VectorVTInfo _, SDNode OpNode,
string OpcodeStr> {
def rb: AVX512<opc, MRMSrcReg, (outs), (ins _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr, "\t{{sae}, $src2, $src1|$src1, $src2, {sae}}"),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
[(set EFLAGS, (OpNode (_.VT _.RC:$src1), _.RC:$src2,
[X86][AVX512] add comi with Sae add builtin_ia32_vcomisd and builtin_ia32_vcomisd Differential Revision: http://reviews.llvm.org/D14331 llvm-svn: 254493
2015-12-02 16:17:51 +08:00
(i32 FROUND_NO_EXC)))],
IIC_SSE_COMIS_RR>, EVEX, EVEX_B, VEX_LIG, EVEX_V128,
Sched<[WriteFAdd]>;
}
let Defs = [EFLAGS], Predicates = [HasAVX512] in {
defm VUCOMISSZ : avx512_ord_cmp_sae<0x2E, v4f32x_info, X86ucomiSae, "vucomiss">,
AVX512PSIi8Base, EVEX_CD8<32, CD8VT1>;
defm VUCOMISDZ : avx512_ord_cmp_sae<0x2E, v2f64x_info, X86ucomiSae, "vucomisd">,
AVX512PDIi8Base, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VCOMISSZ : avx512_ord_cmp_sae<0x2F, v4f32x_info, X86comiSae, "vcomiss">,
AVX512PSIi8Base, EVEX_CD8<32, CD8VT1>;
defm VCOMISDZ : avx512_ord_cmp_sae<0x2F, v2f64x_info, X86comiSae, "vcomisd">,
AVX512PDIi8Base, VEX_W, EVEX_CD8<64, CD8VT1>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Defs = [EFLAGS], Predicates = [HasAVX512] in {
defm VUCOMISSZ : sse12_ord_cmp<0x2E, FR32X, X86cmp, f32, f32mem, loadf32,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
"ucomiss">, PS, EVEX, VEX_LIG,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VT1>;
defm VUCOMISDZ : sse12_ord_cmp<0x2E, FR64X, X86cmp, f64, f64mem, loadf64,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
"ucomisd">, PD, EVEX,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
let Pattern = []<dag> in {
[X86] Fix bug in COMISD and COMISS definition in td files COMISD should receive QWORD because it is defined as (V)COMISD xmm1, xmm2/m64 COMISS should receive DWORD because it is defined as (V)COMISS xmm1, xmm2/m32 Differential Revision: http://reviews.llvm.org/D11712 llvm-svn: 245551
2015-08-20 19:21:36 +08:00
defm VCOMISSZ : sse12_ord_cmp<0x2F, FR32X, undef, f32, f32mem, loadf32,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
"comiss">, PS, EVEX, VEX_LIG,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VT1>;
[X86] Fix bug in COMISD and COMISS definition in td files COMISD should receive QWORD because it is defined as (V)COMISD xmm1, xmm2/m64 COMISS should receive DWORD because it is defined as (V)COMISS xmm1, xmm2/m32 Differential Revision: http://reviews.llvm.org/D11712 llvm-svn: 245551
2015-08-20 19:21:36 +08:00
defm VCOMISDZ : sse12_ord_cmp<0x2F, FR64X, undef, f64, f64mem, loadf64,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
"comisd">, PD, EVEX,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
}
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
let isCodeGenOnly = 1 in {
defm Int_VUCOMISSZ : sse12_ord_cmp<0x2E, VR128X, X86ucomi, v4f32, f128mem,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
load, "ucomiss">, PS, EVEX, VEX_LIG,
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
EVEX_CD8<32, CD8VT1>;
defm Int_VUCOMISDZ : sse12_ord_cmp<0x2E, VR128X, X86ucomi, v2f64, f128mem,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
load, "ucomisd">, PD, EVEX,
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
defm Int_VCOMISSZ : sse12_ord_cmp<0x2F, VR128X, X86comi, v4f32, f128mem,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
load, "comiss">, PS, EVEX, VEX_LIG,
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
EVEX_CD8<32, CD8VT1>;
defm Int_VCOMISDZ : sse12_ord_cmp<0x2F, VR128X, X86comi, v2f64, f128mem,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
load, "comisd">, PD, EVEX,
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
/// avx512_fp14_s rcp14ss, rcp14sd, rsqrt14ss, rsqrt14sd
[X86][AVX512] add masked version for RSQRT14 & RCP14 Scalar FP Differential Revision: http://reviews.llvm.org/D12524 llvm-svn: 248147
2015-09-21 18:23:53 +08:00
multiclass avx512_fp14_s<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
let hasSideEffects = 0, AddedComplexity = 20 , Predicates = [HasAVX512] in {
defm rr : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2))>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in {
[X86][AVX512] add masked version for RSQRT14 & RCP14 Scalar FP Differential Revision: http://reviews.llvm.org/D12524 llvm-svn: 248147
2015-09-21 18:23:53 +08:00
defm rm : AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr,
[X86][AVX512] add masked version for RSQRT14 & RCP14 Scalar FP Differential Revision: http://reviews.llvm.org/D12524 llvm-svn: 248147
2015-09-21 18:23:53 +08:00
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1),
(_.VT (scalar_to_vector (_.ScalarLdFrag addr:$src2))))>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
}
}
[X86][AVX512] add masked version for RSQRT14 & RCP14 Scalar FP Differential Revision: http://reviews.llvm.org/D12524 llvm-svn: 248147
2015-09-21 18:23:53 +08:00
defm VRCP14SS : avx512_fp14_s<0x4D, "vrcp14ss", X86frcp14s, f32x_info>,
EVEX_CD8<32, CD8VT1>, T8PD;
defm VRCP14SD : avx512_fp14_s<0x4D, "vrcp14sd", X86frcp14s, f64x_info>,
VEX_W, EVEX_CD8<64, CD8VT1>, T8PD;
defm VRSQRT14SS : avx512_fp14_s<0x4F, "vrsqrt14ss", X86frsqrt14s, f32x_info>,
EVEX_CD8<32, CD8VT1>, T8PD;
defm VRSQRT14SD : avx512_fp14_s<0x4F, "vrsqrt14sd", X86frsqrt14s, f64x_info>,
VEX_W, EVEX_CD8<64, CD8VT1>, T8PD;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
/// avx512_fp14_p rcp14ps, rcp14pd, rsqrt14ps, rsqrt14pd
multiclass avx512_fp14_p<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX-512] Expanded rsqrt/rcp instructions to VL subset. Refactored multiclass through AVX512_maskable llvm-svn: 220783
2014-10-29 00:37:13 +08:00
X86VectorVTInfo _> {
defm r: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src), OpcodeStr, "$src", "$src",
(_.FloatVT (OpNode _.RC:$src))>, EVEX, T8PD;
let mayLoad = 1 in {
defm m: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.MemOp:$src), OpcodeStr, "$src", "$src",
(OpNode (_.FloatVT
(bitconvert (_.LdFrag addr:$src))))>, EVEX, T8PD;
defm mb: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.ScalarMemOp:$src), OpcodeStr,
"${src}"##_.BroadcastStr, "${src}"##_.BroadcastStr,
(OpNode (_.FloatVT
(X86VBroadcast (_.ScalarLdFrag addr:$src))))>,
EVEX, T8PD, EVEX_B;
}
}
multiclass avx512_fp14_p_vl_all<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm PSZ : avx512_fp14_p<opc, !strconcat(OpcodeStr, "ps"), OpNode, v16f32_info>,
EVEX_V512, EVEX_CD8<32, CD8VF>;
defm PDZ : avx512_fp14_p<opc, !strconcat(OpcodeStr, "pd"), OpNode, v8f64_info>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
// Define only if AVX512VL feature is present.
let Predicates = [HasVLX] in {
defm PSZ128 : avx512_fp14_p<opc, !strconcat(OpcodeStr, "ps"),
OpNode, v4f32x_info>,
EVEX_V128, EVEX_CD8<32, CD8VF>;
defm PSZ256 : avx512_fp14_p<opc, !strconcat(OpcodeStr, "ps"),
OpNode, v8f32x_info>,
EVEX_V256, EVEX_CD8<32, CD8VF>;
defm PDZ128 : avx512_fp14_p<opc, !strconcat(OpcodeStr, "pd"),
OpNode, v2f64x_info>,
EVEX_V128, VEX_W, EVEX_CD8<64, CD8VF>;
defm PDZ256 : avx512_fp14_p<opc, !strconcat(OpcodeStr, "pd"),
OpNode, v4f64x_info>,
EVEX_V256, VEX_W, EVEX_CD8<64, CD8VF>;
}
}
defm VRSQRT14 : avx512_fp14_p_vl_all<0x4E, "vrsqrt14", X86frsqrt>;
defm VRCP14 : avx512_fp14_p_vl_all<0x4C, "vrcp14", X86frcp>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
/// avx512_fp28_s rcp28ss, rcp28sd, rsqrt28ss, rsqrt28sd
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
multiclass avx512_fp28_s<bits<8> opc, string OpcodeStr,X86VectorVTInfo _,
SDNode OpNode> {
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
defm r : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
(i32 FROUND_CURRENT))>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
defm rb : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
"{sae}, $src2, $src1", "$src1, $src2, {sae}",
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
(i32 FROUND_NO_EXC))>, EVEX_B;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
defm m : AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr,
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1),
(_.VT (scalar_to_vector (_.ScalarLdFrag addr:$src2))),
(i32 FROUND_CURRENT))>;
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
multiclass avx512_eri_s<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm SS : avx512_fp28_s<opc, OpcodeStr#"ss", f32x_info, OpNode>,
EVEX_CD8<32, CD8VT1>;
defm SD : avx512_fp28_s<opc, OpcodeStr#"sd", f64x_info, OpNode>,
EVEX_CD8<64, CD8VT1>, VEX_W;
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
let hasSideEffects = 0, Predicates = [HasERI] in {
defm VRCP28 : avx512_eri_s<0xCB, "vrcp28", X86rcp28s>, T8PD, EVEX_4V;
defm VRSQRT28 : avx512_eri_s<0xCD, "vrsqrt28", X86rsqrt28s>, T8PD, EVEX_4V;
}
AVX512: Implemented encoding and intrinsics for VGETEXPSS/D instructions Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11528 llvm-svn: 243390
2015-07-28 14:53:28 +08:00
defm VGETEXP : avx512_eri_s<0x43, "vgetexp", X86fgetexpRnds>, T8PD, EVEX_4V;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
/// avx512_fp28_p rcp28ps, rcp28pd, rsqrt28ps, rsqrt28pd
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
multiclass avx512_fp28_p<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
SDNode OpNode> {
defm r : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src), OpcodeStr, "$src", "$src",
(OpNode (_.VT _.RC:$src), (i32 FROUND_CURRENT))>;
defm m : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.MemOp:$src), OpcodeStr, "$src", "$src",
(OpNode (_.FloatVT
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
(bitconvert (_.LdFrag addr:$src))),
(i32 FROUND_CURRENT))>;
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
defm mb : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.ScalarMemOp:$src), OpcodeStr,
re-apply 238809 AVX-512: Implemented GETEXP instruction for KNL and SKX Added rounding mode modifier for SQRTPS/PD Added tests for encoding and intrinsics. CR: http://reviews.llvm.org/D9991 llvm-svn: 238923
2015-06-03 21:41:48 +08:00
"${src}"##_.BroadcastStr, "${src}"##_.BroadcastStr,
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
(OpNode (_.FloatVT
(X86VBroadcast (_.ScalarLdFrag addr:$src))),
(i32 FROUND_CURRENT))>, EVEX_B;
}
re-apply 238809 AVX-512: Implemented GETEXP instruction for KNL and SKX Added rounding mode modifier for SQRTPS/PD Added tests for encoding and intrinsics. CR: http://reviews.llvm.org/D9991 llvm-svn: 238923
2015-06-03 21:41:48 +08:00
multiclass avx512_fp28_p_round<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
SDNode OpNode> {
defm rb : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src), OpcodeStr,
"{sae}, $src", "$src, {sae}",
(OpNode (_.VT _.RC:$src), (i32 FROUND_NO_EXC))>, EVEX_B;
}
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
multiclass avx512_eri<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm PS : avx512_fp28_p<opc, OpcodeStr#"ps", v16f32_info, OpNode>,
re-apply 238809 AVX-512: Implemented GETEXP instruction for KNL and SKX Added rounding mode modifier for SQRTPS/PD Added tests for encoding and intrinsics. CR: http://reviews.llvm.org/D9991 llvm-svn: 238923
2015-06-03 21:41:48 +08:00
avx512_fp28_p_round<opc, OpcodeStr#"ps", v16f32_info, OpNode>,
T8PD, EVEX_V512, EVEX_CD8<32, CD8VF>;
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
defm PD : avx512_fp28_p<opc, OpcodeStr#"pd", v8f64_info, OpNode>,
re-apply 238809 AVX-512: Implemented GETEXP instruction for KNL and SKX Added rounding mode modifier for SQRTPS/PD Added tests for encoding and intrinsics. CR: http://reviews.llvm.org/D9991 llvm-svn: 238923
2015-06-03 21:41:48 +08:00
avx512_fp28_p_round<opc, OpcodeStr#"pd", v8f64_info, OpNode>,
T8PD, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
}
re-apply 238809 AVX-512: Implemented GETEXP instruction for KNL and SKX Added rounding mode modifier for SQRTPS/PD Added tests for encoding and intrinsics. CR: http://reviews.llvm.org/D9991 llvm-svn: 238923
2015-06-03 21:41:48 +08:00
multiclass avx512_fp_unaryop_packed<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
// Define only if AVX512VL feature is present.
let Predicates = [HasVLX] in {
defm PSZ128 : avx512_fp28_p<opc, OpcodeStr#"ps", v4f32x_info, OpNode>,
EVEX_V128, T8PD, EVEX_CD8<32, CD8VF>;
defm PSZ256 : avx512_fp28_p<opc, OpcodeStr#"ps", v8f32x_info, OpNode>,
EVEX_V256, T8PD, EVEX_CD8<32, CD8VF>;
defm PDZ128 : avx512_fp28_p<opc, OpcodeStr#"pd", v2f64x_info, OpNode>,
EVEX_V128, VEX_W, T8PD, EVEX_CD8<64, CD8VF>;
defm PDZ256 : avx512_fp28_p<opc, OpcodeStr#"pd", v4f64x_info, OpNode>,
EVEX_V256, VEX_W, T8PD, EVEX_CD8<64, CD8VF>;
}
}
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
let Predicates = [HasERI], hasSideEffects = 0 in {
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
re-apply 238809 AVX-512: Implemented GETEXP instruction for KNL and SKX Added rounding mode modifier for SQRTPS/PD Added tests for encoding and intrinsics. CR: http://reviews.llvm.org/D9991 llvm-svn: 238923
2015-06-03 21:41:48 +08:00
defm VRSQRT28 : avx512_eri<0xCC, "vrsqrt28", X86rsqrt28>, EVEX;
defm VRCP28 : avx512_eri<0xCA, "vrcp28", X86rcp28>, EVEX;
defm VEXP2 : avx512_eri<0xC8, "vexp2", X86exp2>, EVEX;
}
defm VGETEXP : avx512_eri<0x42, "vgetexp", X86fgetexpRnd>,
avx512_fp_unaryop_packed<0x42, "vgetexp", X86fgetexpRnd> , EVEX;
multiclass avx512_sqrt_packed_round<bits<8> opc, string OpcodeStr,
SDNode OpNodeRnd, X86VectorVTInfo _>{
defm rb: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src, AVX512RC:$rc), OpcodeStr, "$rc, $src", "$src, $rc",
(_.VT (OpNodeRnd _.RC:$src, (i32 imm:$rc)))>,
EVEX, EVEX_B, EVEX_RC;
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
multiclass avx512_sqrt_packed<bits<8> opc, string OpcodeStr,
SDNode OpNode, X86VectorVTInfo _>{
[AVX512] Fix VSQRT packed instructions internal names. No functional change llvm-svn: 220808
2014-10-29 02:22:41 +08:00
defm r: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
(ins _.RC:$src), OpcodeStr, "$src", "$src",
(_.FloatVT (OpNode _.RC:$src))>, EVEX;
let mayLoad = 1 in {
[AVX512] Fix VSQRT packed instructions internal names. No functional change llvm-svn: 220808
2014-10-29 02:22:41 +08:00
defm m: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
(ins _.MemOp:$src), OpcodeStr, "$src", "$src",
(OpNode (_.FloatVT
(bitconvert (_.LdFrag addr:$src))))>, EVEX;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Fix VSQRT packed instructions internal names. No functional change llvm-svn: 220808
2014-10-29 02:22:41 +08:00
defm mb: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
(ins _.ScalarMemOp:$src), OpcodeStr,
"${src}"##_.BroadcastStr, "${src}"##_.BroadcastStr,
(OpNode (_.FloatVT
(X86VBroadcast (_.ScalarLdFrag addr:$src))))>,
EVEX, EVEX_B;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
multiclass avx512_sqrt_packed_all<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
defm PSZ : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "ps"), OpNode,
v16f32_info>,
EVEX_V512, PS, EVEX_CD8<32, CD8VF>;
defm PDZ : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "pd"), OpNode,
v8f64_info>,
EVEX_V512, VEX_W, PD, EVEX_CD8<64, CD8VF>;
// Define only if AVX512VL feature is present.
let Predicates = [HasVLX] in {
defm PSZ128 : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "ps"),
OpNode, v4f32x_info>,
EVEX_V128, PS, EVEX_CD8<32, CD8VF>;
defm PSZ256 : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "ps"),
OpNode, v8f32x_info>,
EVEX_V256, PS, EVEX_CD8<32, CD8VF>;
defm PDZ128 : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "pd"),
OpNode, v2f64x_info>,
EVEX_V128, VEX_W, PD, EVEX_CD8<64, CD8VF>;
defm PDZ256 : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "pd"),
OpNode, v4f64x_info>,
EVEX_V256, VEX_W, PD, EVEX_CD8<64, CD8VF>;
}
}
re-apply 238809 AVX-512: Implemented GETEXP instruction for KNL and SKX Added rounding mode modifier for SQRTPS/PD Added tests for encoding and intrinsics. CR: http://reviews.llvm.org/D9991 llvm-svn: 238923
2015-06-03 21:41:48 +08:00
multiclass avx512_sqrt_packed_all_round<bits<8> opc, string OpcodeStr,
SDNode OpNodeRnd> {
defm PSZ : avx512_sqrt_packed_round<opc, !strconcat(OpcodeStr, "ps"), OpNodeRnd,
v16f32_info>, EVEX_V512, PS, EVEX_CD8<32, CD8VF>;
defm PDZ : avx512_sqrt_packed_round<opc, !strconcat(OpcodeStr, "pd"), OpNodeRnd,
v8f64_info>, EVEX_V512, VEX_W, PD, EVEX_CD8<64, CD8VF>;
}
AVX512: vsqrtss/sd encoding and intrinsics implementation. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12102 llvm-svn: 248116
2015-09-20 17:13:41 +08:00
multiclass avx512_sqrt_scalar<bits<8> opc, string OpcodeStr,X86VectorVTInfo _,
string SUFF, SDNode OpNode, SDNode OpNodeRnd> {
defm r_Int : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(OpNodeRnd (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(i32 FROUND_CURRENT))>;
let mayLoad = 1 in
defm m_Int : AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr,
AVX512: vsqrtss/sd encoding and intrinsics implementation. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12102 llvm-svn: 248116
2015-09-20 17:13:41 +08:00
"$src2, $src1", "$src1, $src2",
(OpNodeRnd (_.VT _.RC:$src1),
(_.VT (scalar_to_vector
(_.ScalarLdFrag addr:$src2))),
(i32 FROUND_CURRENT))>;
defm rb_Int : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2, AVX512RC:$rc), OpcodeStr,
"$rc, $src2, $src1", "$src1, $src2, $rc",
(OpNodeRnd (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(i32 imm:$rc))>,
EVEX_B, EVEX_RC;
let isCodeGenOnly = 1 in {
AVX-512: fixed asm string of vsqrtss (vvsqrtss was generated before) llvm-svn: 254411
2015-12-01 20:43:46 +08:00
def r : I<opc, MRMSrcReg, (outs _.FRC:$dst),
AVX512: vsqrtss/sd encoding and intrinsics implementation. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12102 llvm-svn: 248116
2015-09-20 17:13:41 +08:00
(ins _.FRC:$src1, _.FRC:$src2),
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}", []>;
let mayLoad = 1 in
AVX-512: fixed asm string of vsqrtss (vvsqrtss was generated before) llvm-svn: 254411
2015-12-01 20:43:46 +08:00
def m : I<opc, MRMSrcMem, (outs _.FRC:$dst),
AVX512: vsqrtss/sd encoding and intrinsics implementation. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12102 llvm-svn: 248116
2015-09-20 17:13:41 +08:00
(ins _.FRC:$src1, _.ScalarMemOp:$src2),
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}", []>;
}
def : Pat<(_.EltVT (OpNode _.FRC:$src)),
(!cast<Instruction>(NAME#SUFF#Zr)
(_.EltVT (IMPLICIT_DEF)), _.FRC:$src)>;
def : Pat<(_.EltVT (OpNode (load addr:$src))),
(!cast<Instruction>(NAME#SUFF#Zm)
Fix incorrect selection of AVX512 sqrt when OptForSize is on Summary: When optimizing for size, sqrt calls can be incorrectly selected as AVX512 VSQRT instructions. This is because X86InstrAVX512.td has a `Requires<[OptForSize]>` in its `avx512_sqrt_scalar` multiclass definition. Even if the target does not support AVX512, the class can apparently still be chosen, leading to an incorrect selection of `vsqrtss`. In PR26625, this lead to an assertion: Reg >= X86::FP0 && Reg <= X86::FP6 && "Expected FP register!", because the `vsqrtss` instruction requires an XMM register, which is not available on i686 CPUs. Reviewers: grosbach, resistor, joker.eph Subscribers: spatel, emaste, llvm-commits Differential Revision: http://reviews.llvm.org/D17414 llvm-svn: 261360
2016-02-20 04:14:11 +08:00
(_.EltVT (IMPLICIT_DEF)), addr:$src)>, Requires<[HasAVX512, OptForSize]>;
AVX512: vsqrtss/sd encoding and intrinsics implementation. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12102 llvm-svn: 248116
2015-09-20 17:13:41 +08:00
}
multiclass avx512_sqrt_scalar_all<bits<8> opc, string OpcodeStr> {
defm SSZ : avx512_sqrt_scalar<opc, OpcodeStr#"ss", f32x_info, "SS", fsqrt,
X86fsqrtRnds>, EVEX_CD8<32, CD8VT1>, EVEX_4V, XS;
defm SDZ : avx512_sqrt_scalar<opc, OpcodeStr#"sd", f64x_info, "SD", fsqrt,
X86fsqrtRnds>, EVEX_CD8<64, CD8VT1>, EVEX_4V, XD, VEX_W;
}
re-apply 238809 AVX-512: Implemented GETEXP instruction for KNL and SKX Added rounding mode modifier for SQRTPS/PD Added tests for encoding and intrinsics. CR: http://reviews.llvm.org/D9991 llvm-svn: 238923
2015-06-03 21:41:48 +08:00
defm VSQRT : avx512_sqrt_packed_all<0x51, "vsqrt", fsqrt>,
avx512_sqrt_packed_all_round<0x51, "vsqrt", X86fsqrtRnd>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX512: vsqrtss/sd encoding and intrinsics implementation. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12102 llvm-svn: 248116
2015-09-20 17:13:41 +08:00
defm VSQRT : avx512_sqrt_scalar_all<0x51, "vsqrt">, VEX_LIG;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(f32 (X86frsqrt FR32X:$src)),
[X86][AVX512] add masked version for RSQRT14 & RCP14 Scalar FP Differential Revision: http://reviews.llvm.org/D12524 llvm-svn: 248147
2015-09-21 18:23:53 +08:00
(COPY_TO_REGCLASS (VRSQRT14SSrr (v4f32 (IMPLICIT_DEF)), (COPY_TO_REGCLASS FR32X:$src, VR128X)), VR128X)>;
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
def : Pat<(f32 (X86frsqrt (load addr:$src))),
[X86][AVX512] add masked version for RSQRT14 & RCP14 Scalar FP Differential Revision: http://reviews.llvm.org/D12524 llvm-svn: 248147
2015-09-21 18:23:53 +08:00
(COPY_TO_REGCLASS (VRSQRT14SSrm (v4f32 (IMPLICIT_DEF)), addr:$src), VR128X)>,
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
Requires<[OptForSize]>;
def : Pat<(f32 (X86frcp FR32X:$src)),
[X86][AVX512] add masked version for RSQRT14 & RCP14 Scalar FP Differential Revision: http://reviews.llvm.org/D12524 llvm-svn: 248147
2015-09-21 18:23:53 +08:00
(COPY_TO_REGCLASS (VRCP14SSrr (v4f32 (IMPLICIT_DEF)), (COPY_TO_REGCLASS FR32X:$src, VR128X)), VR128X )>;
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
def : Pat<(f32 (X86frcp (load addr:$src))),
[X86][AVX512] add masked version for RSQRT14 & RCP14 Scalar FP Differential Revision: http://reviews.llvm.org/D12524 llvm-svn: 248147
2015-09-21 18:23:53 +08:00
(COPY_TO_REGCLASS (VRCP14SSrm (v4f32 (IMPLICIT_DEF)), addr:$src), VR128X)>,
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
Requires<[OptForSize]>;
}
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
multiclass
avx512_rndscale_scalar<bits<8> opc, string OpcodeStr, X86VectorVTInfo _> {
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
let ExeDomain = _.ExeDomain in {
defm r : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2, i32u8imm:$src3), OpcodeStr,
"$src3, $src2, $src1", "$src1, $src2, $src3",
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(_.VT (X86RndScales (_.VT _.RC:$src1), (_.VT _.RC:$src2),
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
(i32 imm:$src3), (i32 FROUND_CURRENT)))>;
defm rb : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2, i32u8imm:$src3), OpcodeStr,
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
"$src3, {sae}, $src2, $src1", "$src1, $src2, {sae}, $src3",
(_.VT (X86RndScales (_.VT _.RC:$src1), (_.VT _.RC:$src2),
AVX-512: Added SKX instructions and intrinsics: {add/sub/mul/div/} x {ps/pd} x {128/256} 2. max/min with sae By Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236971
2015-05-11 14:05:05 +08:00
(i32 imm:$src3), (i32 FROUND_NO_EXC)))>, EVEX_B;
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
let mayLoad = 1 in
defm m : AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Fix scalar mem operands. Differential Revision: http://reviews.llvm.org/D17500 llvm-svn: 261520
2016-02-22 19:48:27 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2, i32u8imm:$src3),
OpcodeStr,
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
"$src3, $src2, $src1", "$src1, $src2, $src3",
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(_.VT (X86RndScales (_.VT _.RC:$src1),
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
(_.VT (scalar_to_vector (_.ScalarLdFrag addr:$src2))),
(i32 imm:$src3), (i32 FROUND_CURRENT)))>;
}
let Predicates = [HasAVX512] in {
def : Pat<(ffloor _.FRC:$src), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##r) (_.VT (IMPLICIT_DEF)),
(_.VT (COPY_TO_REGCLASS _.FRC:$src, _.RC)), (i32 0x1))), _.FRC)>;
def : Pat<(fceil _.FRC:$src), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##r) (_.VT (IMPLICIT_DEF)),
(_.VT (COPY_TO_REGCLASS _.FRC:$src, _.RC)), (i32 0x2))), _.FRC)>;
def : Pat<(ftrunc _.FRC:$src), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##r) (_.VT (IMPLICIT_DEF)),
(_.VT (COPY_TO_REGCLASS _.FRC:$src, _.RC)), (i32 0x3))), _.FRC)>;
def : Pat<(frint _.FRC:$src), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##r) (_.VT (IMPLICIT_DEF)),
(_.VT (COPY_TO_REGCLASS _.FRC:$src, _.RC)), (i32 0x4))), _.FRC)>;
def : Pat<(fnearbyint _.FRC:$src), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##r) (_.VT (IMPLICIT_DEF)),
(_.VT (COPY_TO_REGCLASS _.FRC:$src, _.RC)), (i32 0xc))), _.FRC)>;
def : Pat<(ffloor (_.ScalarLdFrag addr:$src)), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##m) (_.VT (IMPLICIT_DEF)),
addr:$src, (i32 0x1))), _.FRC)>;
def : Pat<(fceil (_.ScalarLdFrag addr:$src)), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##m) (_.VT (IMPLICIT_DEF)),
addr:$src, (i32 0x2))), _.FRC)>;
def : Pat<(ftrunc (_.ScalarLdFrag addr:$src)), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##m) (_.VT (IMPLICIT_DEF)),
addr:$src, (i32 0x3))), _.FRC)>;
def : Pat<(frint (_.ScalarLdFrag addr:$src)), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##m) (_.VT (IMPLICIT_DEF)),
addr:$src, (i32 0x4))), _.FRC)>;
def : Pat<(fnearbyint (_.ScalarLdFrag addr:$src)), (COPY_TO_REGCLASS
(_.VT (!cast<Instruction>(NAME##m) (_.VT (IMPLICIT_DEF)),
addr:$src, (i32 0xc))), _.FRC)>;
}
Revert "AVX-512: Full implementation for VRNDSCALESS/SD instructions and intrinsics." The instructions were being generated on architectures that don't support avx512. This reverts commit r229837. llvm-svn: 229942
2015-02-20 08:45:28 +08:00
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
defm VRNDSCALESS : avx512_rndscale_scalar<0x0A, "vrndscaless", f32x_info>,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<32, CD8VT1>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
AVX-512: recommitted 229837 + bugfix + test llvm-svn: 230223
2015-02-23 23:12:31 +08:00
defm VRNDSCALESD : avx512_rndscale_scalar<0x0B, "vrndscalesd", f64x_info>, VEX_W,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<64, CD8VT1>;
Revert "AVX-512: Full implementation for VRNDSCALESS/SD instructions and intrinsics." The instructions were being generated on architectures that don't support avx512. This reverts commit r229837. llvm-svn: 229942
2015-02-20 08:45:28 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//-------------------------------------------------
// Integer truncate and extend operations
//-------------------------------------------------
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
multiclass avx512_trunc_common<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo SrcInfo, X86VectorVTInfo DestInfo,
X86MemOperand x86memop> {
defm rr : AVX512_maskable<opc, MRMDestReg, DestInfo, (outs DestInfo.RC:$dst),
(ins SrcInfo.RC:$src1), OpcodeStr ,"$src1", "$src1",
(DestInfo.VT (OpNode (SrcInfo.VT SrcInfo.RC:$src1)))>,
EVEX, T8XS;
// for intrinsic patter match
def : Pat<(DestInfo.VT (X86select DestInfo.KRCWM:$mask,
(DestInfo.VT (OpNode (SrcInfo.VT SrcInfo.RC:$src1))),
undef)),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##rrkz) DestInfo.KRCWM:$mask ,
SrcInfo.RC:$src1)>;
def : Pat<(DestInfo.VT (X86select DestInfo.KRCWM:$mask,
(DestInfo.VT (OpNode (SrcInfo.VT SrcInfo.RC:$src1))),
DestInfo.ImmAllZerosV)),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##rrkz) DestInfo.KRCWM:$mask ,
SrcInfo.RC:$src1)>;
def : Pat<(DestInfo.VT (X86select DestInfo.KRCWM:$mask,
(DestInfo.VT (OpNode (SrcInfo.VT SrcInfo.RC:$src1))),
DestInfo.RC:$src0)),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##rrk) DestInfo.RC:$src0,
DestInfo.KRCWM:$mask ,
SrcInfo.RC:$src1)>;
[AVX512] Add hasSideEffects/mayLoad/mayStore flags to some instructions. llvm-svn: 268174
2016-05-01 09:03:56 +08:00
let mayStore = 1, mayLoad = 1, hasSideEffects = 0 in {
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
def mr : AVX512XS8I<opc, MRMDestMem, (outs),
(ins x86memop:$dst, SrcInfo.RC:$src),
[AVX-512] Remove more superfluous spaces from asm strings. llvm-svn: 257301
2016-01-11 08:44:58 +08:00
OpcodeStr # "\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX;
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
def mrk : AVX512XS8I<opc, MRMDestMem, (outs),
(ins x86memop:$dst, SrcInfo.KRCWM:$mask, SrcInfo.RC:$src),
[AVX-512] Remove more superfluous spaces from asm strings. llvm-svn: 257301
2016-01-11 08:44:58 +08:00
OpcodeStr # "\t{$src, $dst {${mask}}|$dst {${mask}}, $src}",
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
[]>, EVEX, EVEX_K;
[AVX512] Add hasSideEffects/mayLoad/mayStore flags to some instructions. llvm-svn: 268174
2016-05-01 09:03:56 +08:00
}//mayStore = 1, mayLoad = 1, hasSideEffects = 0
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
}
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 242990
2015-07-23 15:39:21 +08:00
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
multiclass avx512_trunc_mr_lowering<X86VectorVTInfo SrcInfo,
X86VectorVTInfo DestInfo,
PatFrag truncFrag, PatFrag mtruncFrag > {
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 242990
2015-07-23 15:39:21 +08:00
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
def : Pat<(truncFrag (SrcInfo.VT SrcInfo.RC:$src), addr:$dst),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##mr)
addr:$dst, SrcInfo.RC:$src)>;
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 242990
2015-07-23 15:39:21 +08:00
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
def : Pat<(mtruncFrag addr:$dst, SrcInfo.KRCWM:$mask,
(SrcInfo.VT SrcInfo.RC:$src)),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##mrk)
addr:$dst, SrcInfo.KRCWM:$mask, SrcInfo.RC:$src)>;
}
multiclass avx512_trunc_sat_mr_lowering<X86VectorVTInfo SrcInfo,
X86VectorVTInfo DestInfo, string sat > {
def: Pat<(!cast<Intrinsic>("int_x86_avx512_mask_pmov"#sat#"_"#SrcInfo.Suffix#
DestInfo.Suffix#"_mem_"#SrcInfo.Size)
addr:$ptr, (SrcInfo.VT SrcInfo.RC:$src), SrcInfo.MRC:$mask),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##mrk) addr:$ptr,
(COPY_TO_REGCLASS SrcInfo.MRC:$mask, SrcInfo.KRCWM),
(SrcInfo.VT SrcInfo.RC:$src))>;
def: Pat<(!cast<Intrinsic>("int_x86_avx512_mask_pmov"#sat#"_"#SrcInfo.Suffix#
DestInfo.Suffix#"_mem_"#SrcInfo.Size)
addr:$ptr, (SrcInfo.VT SrcInfo.RC:$src), -1),
(!cast<Instruction>(NAME#SrcInfo.ZSuffix##mr) addr:$ptr,
(SrcInfo.VT SrcInfo.RC:$src))>;
}
multiclass avx512_trunc<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTSrcInfo, X86VectorVTInfo DestInfoZ128,
X86VectorVTInfo DestInfoZ256, X86VectorVTInfo DestInfoZ,
X86MemOperand x86memopZ128, X86MemOperand x86memopZ256,
X86MemOperand x86memopZ, PatFrag truncFrag, PatFrag mtruncFrag,
Predicate prd = HasAVX512>{
let Predicates = [HasVLX, prd] in {
defm Z128: avx512_trunc_common<opc, OpcodeStr, OpNode, VTSrcInfo.info128,
DestInfoZ128, x86memopZ128>,
avx512_trunc_mr_lowering<VTSrcInfo.info128, DestInfoZ128,
truncFrag, mtruncFrag>, EVEX_V128;
defm Z256: avx512_trunc_common<opc, OpcodeStr, OpNode, VTSrcInfo.info256,
DestInfoZ256, x86memopZ256>,
avx512_trunc_mr_lowering<VTSrcInfo.info256, DestInfoZ256,
truncFrag, mtruncFrag>, EVEX_V256;
}
let Predicates = [prd] in
defm Z: avx512_trunc_common<opc, OpcodeStr, OpNode, VTSrcInfo.info512,
DestInfoZ, x86memopZ>,
avx512_trunc_mr_lowering<VTSrcInfo.info512, DestInfoZ,
truncFrag, mtruncFrag>, EVEX_V512;
}
multiclass avx512_trunc_sat<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTSrcInfo, X86VectorVTInfo DestInfoZ128,
X86VectorVTInfo DestInfoZ256, X86VectorVTInfo DestInfoZ,
X86MemOperand x86memopZ128, X86MemOperand x86memopZ256,
X86MemOperand x86memopZ, string sat, Predicate prd = HasAVX512>{
let Predicates = [HasVLX, prd] in {
defm Z128: avx512_trunc_common<opc, OpcodeStr, OpNode, VTSrcInfo.info128,
DestInfoZ128, x86memopZ128>,
avx512_trunc_sat_mr_lowering<VTSrcInfo.info128, DestInfoZ128,
sat>, EVEX_V128;
defm Z256: avx512_trunc_common<opc, OpcodeStr, OpNode, VTSrcInfo.info256,
DestInfoZ256, x86memopZ256>,
avx512_trunc_sat_mr_lowering<VTSrcInfo.info256, DestInfoZ256,
sat>, EVEX_V256;
}
let Predicates = [prd] in
defm Z: avx512_trunc_common<opc, OpcodeStr, OpNode, VTSrcInfo.info512,
DestInfoZ, x86memopZ>,
avx512_trunc_sat_mr_lowering<VTSrcInfo.info512, DestInfoZ,
sat>, EVEX_V512;
}
multiclass avx512_trunc_qb<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm NAME: avx512_trunc<opc, OpcodeStr, OpNode, avx512vl_i64_info,
v16i8x_info, v16i8x_info, v16i8x_info, i16mem, i32mem, i64mem,
truncstorevi8, masked_truncstorevi8>, EVEX_CD8<8, CD8VO>;
}
multiclass avx512_trunc_sat_qb<bits<8> opc, string sat, SDNode OpNode> {
defm NAME: avx512_trunc_sat<opc, "vpmov"##sat##"qb", OpNode, avx512vl_i64_info,
v16i8x_info, v16i8x_info, v16i8x_info, i16mem, i32mem, i64mem,
sat>, EVEX_CD8<8, CD8VO>;
}
multiclass avx512_trunc_qw<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm NAME: avx512_trunc<opc, OpcodeStr, OpNode, avx512vl_i64_info,
v8i16x_info, v8i16x_info, v8i16x_info, i32mem, i64mem, i128mem,
truncstorevi16, masked_truncstorevi16>, EVEX_CD8<16, CD8VQ>;
}
multiclass avx512_trunc_sat_qw<bits<8> opc, string sat, SDNode OpNode> {
defm NAME: avx512_trunc_sat<opc, "vpmov"##sat##"qw", OpNode, avx512vl_i64_info,
v8i16x_info, v8i16x_info, v8i16x_info, i32mem, i64mem, i128mem,
sat>, EVEX_CD8<16, CD8VQ>;
}
multiclass avx512_trunc_qd<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm NAME: avx512_trunc<opc, OpcodeStr, OpNode, avx512vl_i64_info,
v4i32x_info, v4i32x_info, v8i32x_info, i64mem, i128mem, i256mem,
truncstorevi32, masked_truncstorevi32>, EVEX_CD8<32, CD8VH>;
}
multiclass avx512_trunc_sat_qd<bits<8> opc, string sat, SDNode OpNode> {
defm NAME: avx512_trunc_sat<opc, "vpmov"##sat##"qd", OpNode, avx512vl_i64_info,
v4i32x_info, v4i32x_info, v8i32x_info, i64mem, i128mem, i256mem,
sat>, EVEX_CD8<32, CD8VH>;
}
multiclass avx512_trunc_db<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm NAME: avx512_trunc<opc, OpcodeStr, OpNode, avx512vl_i32_info,
v16i8x_info, v16i8x_info, v16i8x_info, i32mem, i64mem, i128mem,
truncstorevi8, masked_truncstorevi8>, EVEX_CD8<8, CD8VQ>;
}
multiclass avx512_trunc_sat_db<bits<8> opc, string sat, SDNode OpNode> {
defm NAME: avx512_trunc_sat<opc, "vpmov"##sat##"db", OpNode, avx512vl_i32_info,
v16i8x_info, v16i8x_info, v16i8x_info, i32mem, i64mem, i128mem,
sat>, EVEX_CD8<8, CD8VQ>;
}
multiclass avx512_trunc_dw<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm NAME: avx512_trunc<opc, OpcodeStr, OpNode, avx512vl_i32_info,
v8i16x_info, v8i16x_info, v16i16x_info, i64mem, i128mem, i256mem,
truncstorevi16, masked_truncstorevi16>, EVEX_CD8<16, CD8VH>;
}
multiclass avx512_trunc_sat_dw<bits<8> opc, string sat, SDNode OpNode> {
defm NAME: avx512_trunc_sat<opc, "vpmov"##sat##"dw", OpNode, avx512vl_i32_info,
v8i16x_info, v8i16x_info, v16i16x_info, i64mem, i128mem, i256mem,
sat>, EVEX_CD8<16, CD8VH>;
}
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
multiclass avx512_trunc_wb<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm NAME: avx512_trunc<opc, OpcodeStr, OpNode, avx512vl_i16_info,
v16i8x_info, v16i8x_info, v32i8x_info, i64mem, i128mem, i256mem,
truncstorevi8, masked_truncstorevi8,HasBWI>, EVEX_CD8<16, CD8VH>;
}
multiclass avx512_trunc_sat_wb<bits<8> opc, string sat, SDNode OpNode> {
defm NAME: avx512_trunc_sat<opc, "vpmov"##sat##"wb", OpNode, avx512vl_i16_info,
v16i8x_info, v16i8x_info, v32i8x_info, i64mem, i128mem, i256mem,
sat, HasBWI>, EVEX_CD8<16, CD8VH>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Implemented encoding , DAG lowering and intrinsics for Integer Truncate with/without saturation Added tests for DAG lowering ,encoding and intrinsic Differential Revision: http://reviews.llvm.org/D11218 llvm-svn: 243122
2015-07-25 01:24:15 +08:00
defm VPMOVQB : avx512_trunc_qb<0x32, "vpmovqb", X86vtrunc>;
defm VPMOVSQB : avx512_trunc_sat_qb<0x22, "s", X86vtruncs>;
defm VPMOVUSQB : avx512_trunc_sat_qb<0x12, "us", X86vtruncus>;
defm VPMOVQW : avx512_trunc_qw<0x34, "vpmovqw", X86vtrunc>;
defm VPMOVSQW : avx512_trunc_sat_qw<0x24, "s", X86vtruncs>;
defm VPMOVUSQW : avx512_trunc_sat_qw<0x14, "us", X86vtruncus>;
defm VPMOVQD : avx512_trunc_qd<0x35, "vpmovqd", X86vtrunc>;
defm VPMOVSQD : avx512_trunc_sat_qd<0x25, "s", X86vtruncs>;
defm VPMOVUSQD : avx512_trunc_sat_qd<0x15, "us", X86vtruncus>;
defm VPMOVDB : avx512_trunc_db<0x31, "vpmovdb", X86vtrunc>;
defm VPMOVSDB : avx512_trunc_sat_db<0x21, "s", X86vtruncs>;
defm VPMOVUSDB : avx512_trunc_sat_db<0x11, "us", X86vtruncus>;
defm VPMOVDW : avx512_trunc_dw<0x33, "vpmovdw", X86vtrunc>;
defm VPMOVSDW : avx512_trunc_sat_dw<0x23, "s", X86vtruncs>;
defm VPMOVUSDW : avx512_trunc_sat_dw<0x13, "us", X86vtruncus>;
defm VPMOVWB : avx512_trunc_wb<0x30, "vpmovwb", X86vtrunc>;
defm VPMOVSWB : avx512_trunc_sat_wb<0x20, "s", X86vtruncs>;
defm VPMOVUSWB : avx512_trunc_sat_wb<0x10, "us", X86vtruncus>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Optimized SIMD truncate operations for AVX512F set. Optimized <8 x i32> to <8 x i16> <4 x i64> to < 4 x i32> <16 x i16> to <16 x i8> All these oprtrations use now AVX512F set (KNL). Before this change it was implemented with AVX2 set. Differential Revision: http://reviews.llvm.org/D14108 llvm-svn: 251764
2015-11-01 19:45:47 +08:00
let Predicates = [HasAVX512, NoVLX] in {
def: Pat<(v8i16 (X86vtrunc (v8i32 VR256X:$src))),
(v8i16 (EXTRACT_SUBREG
(v16i16 (VPMOVDWZrr (v16i32 (SUBREG_TO_REG (i32 0),
VR256X:$src, sub_ymm)))), sub_xmm))>;
def: Pat<(v4i32 (X86vtrunc (v4i64 VR256X:$src))),
(v4i32 (EXTRACT_SUBREG
(v8i32 (VPMOVQDZrr (v8i64 (SUBREG_TO_REG (i32 0),
VR256X:$src, sub_ymm)))), sub_xmm))>;
}
let Predicates = [HasBWI, NoVLX] in {
def: Pat<(v16i8 (X86vtrunc (v16i16 VR256X:$src))),
(v16i8 (EXTRACT_SUBREG (VPMOVWBZrr (v32i16 (SUBREG_TO_REG (i32 0),
VR256X:$src, sub_ymm))), sub_xmm))>;
}
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
multiclass avx512_extend_common<bits<8> opc, string OpcodeStr,
X86VectorVTInfo DestInfo, X86VectorVTInfo SrcInfo,
X86MemOperand x86memop, PatFrag LdFrag, SDNode OpNode>{
defm rr : AVX512_maskable<opc, MRMSrcReg, DestInfo, (outs DestInfo.RC:$dst),
(ins SrcInfo.RC:$src), OpcodeStr ,"$src", "$src",
(DestInfo.VT (OpNode (SrcInfo.VT SrcInfo.RC:$src)))>,
EVEX;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
let mayLoad = 1 in {
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
defm rm : AVX512_maskable<opc, MRMSrcMem, DestInfo, (outs DestInfo.RC:$dst),
(ins x86memop:$src), OpcodeStr ,"$src", "$src",
(DestInfo.VT (LdFrag addr:$src))>,
EVEX;
}
}
AVX512: Add vpmovzxbw/d/q ,vpmovzxw/d/q ,vpmovzxbdq lowering patterns that support 256bit inputs like AVX patterns ( that are disable in case HasVLX , see SS41I_pmovx_avx2_patterns). Differential Revision: http://reviews.llvm.org/D17504 llvm-svn: 261724
2016-02-24 16:15:20 +08:00
// support full register inputs (like SSE paterns)
multiclass avx512_extend_lowering<SDNode OpNode, X86VectorVTInfo To,
X86VectorVTInfo From, SubRegIndex SubRegIdx> {
def : Pat<(To.VT (OpNode (From.VT From.RC:$src))),
(!cast<Instruction>(NAME#To.ZSuffix#"rr")
(EXTRACT_SUBREG From.RC:$src, SubRegIdx))>;
}
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
multiclass avx512_extend_BW<bits<8> opc, string OpcodeStr, SDNode OpNode,
string ExtTy,PatFrag LdFrag = !cast<PatFrag>(ExtTy#"extloadvi8")> {
let Predicates = [HasVLX, HasBWI] in {
defm Z128: avx512_extend_common<opc, OpcodeStr, v8i16x_info,
v16i8x_info, i64mem, LdFrag, OpNode>,
EVEX_CD8<8, CD8VH>, T8PD, EVEX_V128;
defm Z256: avx512_extend_common<opc, OpcodeStr, v16i16x_info,
v16i8x_info, i128mem, LdFrag, OpNode>,
AVX512: Add vpmovzxbw/d/q ,vpmovzxw/d/q ,vpmovzxbdq lowering patterns that support 256bit inputs like AVX patterns ( that are disable in case HasVLX , see SS41I_pmovx_avx2_patterns). Differential Revision: http://reviews.llvm.org/D17504 llvm-svn: 261724
2016-02-24 16:15:20 +08:00
avx512_extend_lowering<OpNode, v16i16x_info, v32i8x_info, sub_xmm>,
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
EVEX_CD8<8, CD8VH>, T8PD, EVEX_V256;
}
let Predicates = [HasBWI] in {
defm Z : avx512_extend_common<opc, OpcodeStr, v32i16_info,
v32i8x_info, i256mem, LdFrag, OpNode>,
EVEX_CD8<8, CD8VH>, T8PD, EVEX_V512;
}
}
multiclass avx512_extend_BD<bits<8> opc, string OpcodeStr, SDNode OpNode,
string ExtTy,PatFrag LdFrag = !cast<PatFrag>(ExtTy#"extloadvi8")> {
let Predicates = [HasVLX, HasAVX512] in {
defm Z128: avx512_extend_common<opc, OpcodeStr, v4i32x_info,
v16i8x_info, i32mem, LdFrag, OpNode>,
EVEX_CD8<8, CD8VQ>, T8PD, EVEX_V128;
defm Z256: avx512_extend_common<opc, OpcodeStr, v8i32x_info,
v16i8x_info, i64mem, LdFrag, OpNode>,
AVX512: Add vpmovzxbw/d/q ,vpmovzxw/d/q ,vpmovzxbdq lowering patterns that support 256bit inputs like AVX patterns ( that are disable in case HasVLX , see SS41I_pmovx_avx2_patterns). Differential Revision: http://reviews.llvm.org/D17504 llvm-svn: 261724
2016-02-24 16:15:20 +08:00
avx512_extend_lowering<OpNode, v8i32x_info, v32i8x_info, sub_xmm>,
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
EVEX_CD8<8, CD8VQ>, T8PD, EVEX_V256;
}
let Predicates = [HasAVX512] in {
defm Z : avx512_extend_common<opc, OpcodeStr, v16i32_info,
v16i8x_info, i128mem, LdFrag, OpNode>,
EVEX_CD8<8, CD8VQ>, T8PD, EVEX_V512;
}
}
multiclass avx512_extend_BQ<bits<8> opc, string OpcodeStr, SDNode OpNode,
string ExtTy,PatFrag LdFrag = !cast<PatFrag>(ExtTy#"extloadvi8")> {
let Predicates = [HasVLX, HasAVX512] in {
defm Z128: avx512_extend_common<opc, OpcodeStr, v2i64x_info,
v16i8x_info, i16mem, LdFrag, OpNode>,
EVEX_CD8<8, CD8VO>, T8PD, EVEX_V128;
defm Z256: avx512_extend_common<opc, OpcodeStr, v4i64x_info,
v16i8x_info, i32mem, LdFrag, OpNode>,
AVX512: Add vpmovzxbw/d/q ,vpmovzxw/d/q ,vpmovzxbdq lowering patterns that support 256bit inputs like AVX patterns ( that are disable in case HasVLX , see SS41I_pmovx_avx2_patterns). Differential Revision: http://reviews.llvm.org/D17504 llvm-svn: 261724
2016-02-24 16:15:20 +08:00
avx512_extend_lowering<OpNode, v4i64x_info, v32i8x_info, sub_xmm>,
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
EVEX_CD8<8, CD8VO>, T8PD, EVEX_V256;
}
let Predicates = [HasAVX512] in {
defm Z : avx512_extend_common<opc, OpcodeStr, v8i64_info,
v16i8x_info, i64mem, LdFrag, OpNode>,
EVEX_CD8<8, CD8VO>, T8PD, EVEX_V512;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
multiclass avx512_extend_WD<bits<8> opc, string OpcodeStr, SDNode OpNode,
string ExtTy,PatFrag LdFrag = !cast<PatFrag>(ExtTy#"extloadvi16")> {
let Predicates = [HasVLX, HasAVX512] in {
defm Z128: avx512_extend_common<opc, OpcodeStr, v4i32x_info,
v8i16x_info, i64mem, LdFrag, OpNode>,
EVEX_CD8<16, CD8VH>, T8PD, EVEX_V128;
defm Z256: avx512_extend_common<opc, OpcodeStr, v8i32x_info,
v8i16x_info, i128mem, LdFrag, OpNode>,
AVX512: Add vpmovzxbw/d/q ,vpmovzxw/d/q ,vpmovzxbdq lowering patterns that support 256bit inputs like AVX patterns ( that are disable in case HasVLX , see SS41I_pmovx_avx2_patterns). Differential Revision: http://reviews.llvm.org/D17504 llvm-svn: 261724
2016-02-24 16:15:20 +08:00
avx512_extend_lowering<OpNode, v8i32x_info, v16i16x_info, sub_xmm>,
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
EVEX_CD8<16, CD8VH>, T8PD, EVEX_V256;
}
let Predicates = [HasAVX512] in {
defm Z : avx512_extend_common<opc, OpcodeStr, v16i32_info,
v16i16x_info, i256mem, LdFrag, OpNode>,
EVEX_CD8<16, CD8VH>, T8PD, EVEX_V512;
}
}
multiclass avx512_extend_WQ<bits<8> opc, string OpcodeStr, SDNode OpNode,
string ExtTy,PatFrag LdFrag = !cast<PatFrag>(ExtTy#"extloadvi16")> {
let Predicates = [HasVLX, HasAVX512] in {
defm Z128: avx512_extend_common<opc, OpcodeStr, v2i64x_info,
v8i16x_info, i32mem, LdFrag, OpNode>,
EVEX_CD8<16, CD8VQ>, T8PD, EVEX_V128;
defm Z256: avx512_extend_common<opc, OpcodeStr, v4i64x_info,
v8i16x_info, i64mem, LdFrag, OpNode>,
AVX512: Add vpmovzxbw/d/q ,vpmovzxw/d/q ,vpmovzxbdq lowering patterns that support 256bit inputs like AVX patterns ( that are disable in case HasVLX , see SS41I_pmovx_avx2_patterns). Differential Revision: http://reviews.llvm.org/D17504 llvm-svn: 261724
2016-02-24 16:15:20 +08:00
avx512_extend_lowering<OpNode, v4i64x_info, v16i16x_info, sub_xmm>,
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
EVEX_CD8<16, CD8VQ>, T8PD, EVEX_V256;
}
let Predicates = [HasAVX512] in {
defm Z : avx512_extend_common<opc, OpcodeStr, v8i64_info,
v8i16x_info, i128mem, LdFrag, OpNode>,
EVEX_CD8<16, CD8VQ>, T8PD, EVEX_V512;
}
}
multiclass avx512_extend_DQ<bits<8> opc, string OpcodeStr, SDNode OpNode,
string ExtTy,PatFrag LdFrag = !cast<PatFrag>(ExtTy#"extloadvi32")> {
let Predicates = [HasVLX, HasAVX512] in {
defm Z128: avx512_extend_common<opc, OpcodeStr, v2i64x_info,
v4i32x_info, i64mem, LdFrag, OpNode>,
EVEX_CD8<32, CD8VH>, T8PD, EVEX_V128;
defm Z256: avx512_extend_common<opc, OpcodeStr, v4i64x_info,
v4i32x_info, i128mem, LdFrag, OpNode>,
AVX512: Add vpmovzxbw/d/q ,vpmovzxw/d/q ,vpmovzxbdq lowering patterns that support 256bit inputs like AVX patterns ( that are disable in case HasVLX , see SS41I_pmovx_avx2_patterns). Differential Revision: http://reviews.llvm.org/D17504 llvm-svn: 261724
2016-02-24 16:15:20 +08:00
avx512_extend_lowering<OpNode, v4i64x_info, v8i32x_info, sub_xmm>,
AVX-512: Implemented all forms of sign-extend and zero-extend instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238301
2015-05-27 16:15:19 +08:00
EVEX_CD8<32, CD8VH>, T8PD, EVEX_V256;
}
let Predicates = [HasAVX512] in {
defm Z : avx512_extend_common<opc, OpcodeStr, v8i64_info,
v8i32x_info, i256mem, LdFrag, OpNode>,
EVEX_CD8<32, CD8VH>, T8PD, EVEX_V512;
}
}
defm VPMOVZXBW : avx512_extend_BW<0x30, "vpmovzxbw", X86vzext, "z">;
defm VPMOVZXBD : avx512_extend_BD<0x31, "vpmovzxbd", X86vzext, "z">;
defm VPMOVZXBQ : avx512_extend_BQ<0x32, "vpmovzxbq", X86vzext, "z">;
defm VPMOVZXWD : avx512_extend_WD<0x33, "vpmovzxwd", X86vzext, "z">;
defm VPMOVZXWQ : avx512_extend_WQ<0x34, "vpmovzxwq", X86vzext, "z">;
defm VPMOVZXDQ : avx512_extend_DQ<0x35, "vpmovzxdq", X86vzext, "z">;
defm VPMOVSXBW: avx512_extend_BW<0x20, "vpmovsxbw", X86vsext, "s">;
defm VPMOVSXBD: avx512_extend_BD<0x21, "vpmovsxbd", X86vsext, "s">;
defm VPMOVSXBQ: avx512_extend_BQ<0x22, "vpmovsxbq", X86vsext, "s">;
defm VPMOVSXWD: avx512_extend_WD<0x23, "vpmovsxwd", X86vsext, "s">;
defm VPMOVSXWQ: avx512_extend_WQ<0x24, "vpmovsxwq", X86vsext, "s">;
defm VPMOVSXDQ: avx512_extend_DQ<0x25, "vpmovsxdq", X86vsext, "s">;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// GATHER - SCATTER Operations
Masked gather and scatter - added DAGCombine visitors and AVX-512 instruction selection patterns. All other patches, including tests will follow. http://reviews.llvm.org/D7665 llvm-svn: 236211
2015-04-30 16:38:48 +08:00
multiclass avx512_gather<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
X86MemOperand memop, PatFrag GatherNode> {
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
let Constraints = "@earlyclobber $dst, $src1 = $dst, $mask = $mask_wb",
ExeDomain = _.ExeDomain in
Masked gather and scatter - added DAGCombine visitors and AVX-512 instruction selection patterns. All other patches, including tests will follow. http://reviews.llvm.org/D7665 llvm-svn: 236211
2015-04-30 16:38:48 +08:00
def rm : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst, _.KRCWM:$mask_wb),
(ins _.RC:$src1, _.KRCWM:$mask, memop:$src2),
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
!strconcat(OpcodeStr#_.Suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, ${dst} {${mask}}|${dst} {${mask}}, $src2}"),
Masked gather and scatter - added DAGCombine visitors and AVX-512 instruction selection patterns. All other patches, including tests will follow. http://reviews.llvm.org/D7665 llvm-svn: 236211
2015-04-30 16:38:48 +08:00
[(set _.RC:$dst, _.KRCWM:$mask_wb,
(GatherNode (_.VT _.RC:$src1), _.KRCWM:$mask,
vectoraddr:$src2))]>, EVEX, EVEX_K,
EVEX_CD8<_.EltSize, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
multiclass avx512_gather_q_pd<bits<8> dopc, bits<8> qopc,
AVX512VLVectorVTInfo _, string OpcodeStr, string SUFF> {
defm NAME##D##SUFF##Z: avx512_gather<dopc, OpcodeStr##"d", _.info512,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vy512mem, mgatherv8i32>, EVEX_V512, VEX_W;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
defm NAME##Q##SUFF##Z: avx512_gather<qopc, OpcodeStr##"q", _.info512,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vz512mem, mgatherv8i64>, EVEX_V512, VEX_W;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
let Predicates = [HasVLX] in {
defm NAME##D##SUFF##Z256: avx512_gather<dopc, OpcodeStr##"d", _.info256,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vx256xmem, mgatherv4i32>, EVEX_V256, VEX_W;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
defm NAME##Q##SUFF##Z256: avx512_gather<qopc, OpcodeStr##"q", _.info256,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vy256xmem, mgatherv4i64>, EVEX_V256, VEX_W;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
defm NAME##D##SUFF##Z128: avx512_gather<dopc, OpcodeStr##"d", _.info128,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vx128xmem, mgatherv4i32>, EVEX_V128, VEX_W;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
defm NAME##Q##SUFF##Z128: avx512_gather<qopc, OpcodeStr##"q", _.info128,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vx128xmem, mgatherv2i64>, EVEX_V128, VEX_W;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
}
}
multiclass avx512_gather_d_ps<bits<8> dopc, bits<8> qopc,
AVX512VLVectorVTInfo _, string OpcodeStr, string SUFF> {
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
defm NAME##D##SUFF##Z: avx512_gather<dopc, OpcodeStr##"d", _.info512, vz512mem,
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
mgatherv16i32>, EVEX_V512;
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
defm NAME##Q##SUFF##Z: avx512_gather<qopc, OpcodeStr##"q", _.info256, vz512mem,
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
mgatherv8i64>, EVEX_V512;
let Predicates = [HasVLX] in {
defm NAME##D##SUFF##Z256: avx512_gather<dopc, OpcodeStr##"d", _.info256,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vy256xmem, mgatherv8i32>, EVEX_V256;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
defm NAME##Q##SUFF##Z256: avx512_gather<qopc, OpcodeStr##"q", _.info128,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vy128xmem, mgatherv4i64>, EVEX_V256;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
defm NAME##D##SUFF##Z128: avx512_gather<dopc, OpcodeStr##"d", _.info128,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vx128xmem, mgatherv4i32>, EVEX_V128;
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
defm NAME##Q##SUFF##Z128: avx512_gather<qopc, OpcodeStr##"q", _.info128,
vx64xmem, mgatherv2i64>, EVEX_V128;
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
}
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
Masked gather and scatter - added DAGCombine visitors and AVX-512 instruction selection patterns. All other patches, including tests will follow. http://reviews.llvm.org/D7665 llvm-svn: 236211
2015-04-30 16:38:48 +08:00
AVX-512: Added all SKX forms of GATHER instructions. Added intrinsics. Added encoding and tests. llvm-svn: 240905
2015-06-28 18:53:29 +08:00
defm VGATHER : avx512_gather_q_pd<0x92, 0x93, avx512vl_f64_info, "vgather", "PD">,
avx512_gather_d_ps<0x92, 0x93, avx512vl_f32_info, "vgather", "PS">;
defm VPGATHER : avx512_gather_q_pd<0x90, 0x91, avx512vl_i64_info, "vpgather", "Q">,
avx512_gather_d_ps<0x90, 0x91, avx512vl_i32_info, "vpgather", "D">;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Masked gather and scatter - added DAGCombine visitors and AVX-512 instruction selection patterns. All other patches, including tests will follow. http://reviews.llvm.org/D7665 llvm-svn: 236211
2015-04-30 16:38:48 +08:00
multiclass avx512_scatter<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
X86MemOperand memop, PatFrag ScatterNode> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
let mayStore = 1, Constraints = "$mask = $mask_wb", ExeDomain = _.ExeDomain in
Masked gather and scatter - added DAGCombine visitors and AVX-512 instruction selection patterns. All other patches, including tests will follow. http://reviews.llvm.org/D7665 llvm-svn: 236211
2015-04-30 16:38:48 +08:00
def mr : AVX5128I<opc, MRMDestMem, (outs _.KRCWM:$mask_wb),
(ins memop:$dst, _.KRCWM:$mask, _.RC:$src),
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
!strconcat(OpcodeStr#_.Suffix,
Masked gather and scatter - added DAGCombine visitors and AVX-512 instruction selection patterns. All other patches, including tests will follow. http://reviews.llvm.org/D7665 llvm-svn: 236211
2015-04-30 16:38:48 +08:00
"\t{$src, ${dst} {${mask}}|${dst} {${mask}}, $src}"),
[(set _.KRCWM:$mask_wb, (ScatterNode (_.VT _.RC:$src),
_.KRCWM:$mask, vectoraddr:$dst))]>,
EVEX, EVEX_K, EVEX_CD8<_.EltSize, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
multiclass avx512_scatter_q_pd<bits<8> dopc, bits<8> qopc,
AVX512VLVectorVTInfo _, string OpcodeStr, string SUFF> {
defm NAME##D##SUFF##Z: avx512_scatter<dopc, OpcodeStr##"d", _.info512,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vy512mem, mscatterv8i32>, EVEX_V512, VEX_W;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm NAME##Q##SUFF##Z: avx512_scatter<qopc, OpcodeStr##"q", _.info512,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vz512mem, mscatterv8i64>, EVEX_V512, VEX_W;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
let Predicates = [HasVLX] in {
defm NAME##D##SUFF##Z256: avx512_scatter<dopc, OpcodeStr##"d", _.info256,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vx256xmem, mscatterv4i32>, EVEX_V256, VEX_W;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm NAME##Q##SUFF##Z256: avx512_scatter<qopc, OpcodeStr##"q", _.info256,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vy256xmem, mscatterv4i64>, EVEX_V256, VEX_W;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm NAME##D##SUFF##Z128: avx512_scatter<dopc, OpcodeStr##"d", _.info128,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vx128xmem, mscatterv4i32>, EVEX_V128, VEX_W;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm NAME##Q##SUFF##Z128: avx512_scatter<qopc, OpcodeStr##"q", _.info128,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vx128xmem, mscatterv2i64>, EVEX_V128, VEX_W;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
}
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
}
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
multiclass avx512_scatter_d_ps<bits<8> dopc, bits<8> qopc,
AVX512VLVectorVTInfo _, string OpcodeStr, string SUFF> {
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
defm NAME##D##SUFF##Z: avx512_scatter<dopc, OpcodeStr##"d", _.info512, vz512mem,
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
mscatterv16i32>, EVEX_V512;
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
defm NAME##Q##SUFF##Z: avx512_scatter<qopc, OpcodeStr##"q", _.info256, vz512mem,
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
mscatterv8i64>, EVEX_V512;
let Predicates = [HasVLX] in {
defm NAME##D##SUFF##Z256: avx512_scatter<dopc, OpcodeStr##"d", _.info256,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vy256xmem, mscatterv8i32>, EVEX_V256;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm NAME##Q##SUFF##Z256: avx512_scatter<qopc, OpcodeStr##"q", _.info128,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vy128xmem, mscatterv4i64>, EVEX_V256;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm NAME##D##SUFF##Z128: avx512_scatter<dopc, OpcodeStr##"d", _.info128,
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
vx128xmem, mscatterv4i32>, EVEX_V128;
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm NAME##Q##SUFF##Z128: avx512_scatter<qopc, OpcodeStr##"q", _.info128,
vx64xmem, mscatterv2i64>, EVEX_V128;
}
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
}
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm VSCATTER : avx512_scatter_q_pd<0xA2, 0xA3, avx512vl_f64_info, "vscatter", "PD">,
avx512_scatter_d_ps<0xA2, 0xA3, avx512vl_f32_info, "vscatter", "PS">;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: all forms of SCATTER instruction on SKX, encoding, intrinsics and tests. llvm-svn: 240936
2015-06-29 20:14:24 +08:00
defm VPSCATTER : avx512_scatter_q_pd<0xA0, 0xA1, avx512vl_i64_info, "vpscatter", "Q">,
avx512_scatter_d_ps<0xA0, 0xA1, avx512vl_i32_info, "vpscatter", "D">;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
// prefetch
multiclass avx512_gather_scatter_prefetch<bits<8> opc, Format F, string OpcodeStr,
RegisterClass KRC, X86MemOperand memop> {
let Predicates = [HasPFI], hasSideEffects = 1 in
def m : AVX5128I<opc, F, (outs), (ins KRC:$mask, memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src {${mask}}|{${mask}}, $src}"),
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
[]>, EVEX, EVEX_K;
}
defm VGATHERPF0DPS: avx512_gather_scatter_prefetch<0xC6, MRM1m, "vgatherpf0dps",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK16WM, vz512mem>, EVEX_V512, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VGATHERPF0QPS: avx512_gather_scatter_prefetch<0xC7, MRM1m, "vgatherpf0qps",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vz512mem>, EVEX_V512, EVEX_CD8<64, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VGATHERPF0DPD: avx512_gather_scatter_prefetch<0xC6, MRM1m, "vgatherpf0dpd",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vy512mem>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VGATHERPF0QPD: avx512_gather_scatter_prefetch<0xC7, MRM1m, "vgatherpf0qpd",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vz512mem>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VGATHERPF1DPS: avx512_gather_scatter_prefetch<0xC6, MRM2m, "vgatherpf1dps",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK16WM, vz512mem>, EVEX_V512, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VGATHERPF1QPS: avx512_gather_scatter_prefetch<0xC7, MRM2m, "vgatherpf1qps",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vz512mem>, EVEX_V512, EVEX_CD8<64, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VGATHERPF1DPD: avx512_gather_scatter_prefetch<0xC6, MRM2m, "vgatherpf1dpd",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vy512mem>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VGATHERPF1QPD: avx512_gather_scatter_prefetch<0xC7, MRM2m, "vgatherpf1qpd",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vz512mem>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VSCATTERPF0DPS: avx512_gather_scatter_prefetch<0xC6, MRM5m, "vscatterpf0dps",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK16WM, vz512mem>, EVEX_V512, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VSCATTERPF0QPS: avx512_gather_scatter_prefetch<0xC7, MRM5m, "vscatterpf0qps",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vz512mem>, EVEX_V512, EVEX_CD8<64, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VSCATTERPF0DPD: avx512_gather_scatter_prefetch<0xC6, MRM5m, "vscatterpf0dpd",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vy512mem>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VSCATTERPF0QPD: avx512_gather_scatter_prefetch<0xC7, MRM5m, "vscatterpf0qpd",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vz512mem>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VSCATTERPF1DPS: avx512_gather_scatter_prefetch<0xC6, MRM6m, "vscatterpf1dps",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK16WM, vz512mem>, EVEX_V512, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VSCATTERPF1QPS: avx512_gather_scatter_prefetch<0xC7, MRM6m, "vscatterpf1qps",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vz512mem>, EVEX_V512, EVEX_CD8<64, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VSCATTERPF1DPD: avx512_gather_scatter_prefetch<0xC6, MRM6m, "vscatterpf1dpd",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vy512mem>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VSCATTERPF1QPD: avx512_gather_scatter_prefetch<0xC7, MRM6m, "vscatterpf1qpd",
AVX512F: Add GATHER/SCATTER assembler Intel syntax tests for knl/skx/avx . Change memory operand parser handling. Differential Revision: http://reviews.llvm.org/D17564 llvm-svn: 261862
2016-02-25 21:30:17 +08:00
VK8WM, vz512mem>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
// Helper fragments to match sext vXi1 to vXiY.
def v16i1sextv16i32 : PatLeaf<(v16i32 (X86vsrai VR512:$src, (i8 31)))>;
def v8i1sextv8i64 : PatLeaf<(v8i64 (X86vsrai VR512:$src, (i8 63)))>;
AVX-512-SKX: Added instruction VPMOVM2B/W/D/Q. This instruction allows to broadacst mask vector to data vector. llvm-svn: 219083
2014-10-05 22:11:08 +08:00
multiclass cvt_by_vec_width<bits<8> opc, X86VectorVTInfo Vec, string OpcodeStr > {
AVX-512: Added VPMOVx2M instructions for SKX, fixed encoding of VPMOVM2x. llvm-svn: 235385
2015-04-21 22:38:31 +08:00
def rr : AVX512XS8I<opc, MRMSrcReg, (outs Vec.RC:$dst), (ins Vec.KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr##Vec.Suffix, "\t{$src, $dst|$dst, $src}"),
AVX-512-SKX: Added instruction VPMOVM2B/W/D/Q. This instruction allows to broadacst mask vector to data vector. llvm-svn: 219083
2014-10-05 22:11:08 +08:00
[(set Vec.RC:$dst, (Vec.VT (X86vsext Vec.KRC:$src)))]>, EVEX;
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512-SKX: Added instruction VPMOVM2B/W/D/Q. This instruction allows to broadacst mask vector to data vector. llvm-svn: 219083
2014-10-05 22:11:08 +08:00
multiclass cvt_mask_by_elt_width<bits<8> opc, AVX512VLVectorVTInfo VTInfo,
string OpcodeStr, Predicate prd> {
let Predicates = [prd] in
defm Z : cvt_by_vec_width<opc, VTInfo.info512, OpcodeStr>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : cvt_by_vec_width<opc, VTInfo.info256, OpcodeStr>, EVEX_V256;
defm Z128 : cvt_by_vec_width<opc, VTInfo.info128, OpcodeStr>, EVEX_V128;
}
}
multiclass avx512_convert_mask_to_vector<string OpcodeStr> {
defm NAME##B : cvt_mask_by_elt_width<0x28, avx512vl_i8_info, OpcodeStr,
HasBWI>;
defm NAME##W : cvt_mask_by_elt_width<0x28, avx512vl_i16_info, OpcodeStr,
HasBWI>, VEX_W;
defm NAME##D : cvt_mask_by_elt_width<0x38, avx512vl_i32_info, OpcodeStr,
HasDQI>;
defm NAME##Q : cvt_mask_by_elt_width<0x38, avx512vl_i64_info, OpcodeStr,
HasDQI>, VEX_W;
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512-SKX: Added instruction VPMOVM2B/W/D/Q. This instruction allows to broadacst mask vector to data vector. llvm-svn: 219083
2014-10-05 22:11:08 +08:00
defm VPMOVM2 : avx512_convert_mask_to_vector<"vpmovm2">;
AVX-512: Added all forms of COMPRESS instruction + intrinsics + tests llvm-svn: 224019
2014-12-11 23:02:24 +08:00
AVX-512: Added VPMOVx2M instructions for SKX, fixed encoding of VPMOVM2x. llvm-svn: 235385
2015-04-21 22:38:31 +08:00
multiclass convert_vector_to_mask_common<bits<8> opc, X86VectorVTInfo _, string OpcodeStr > {
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
def rr : AVX512XS8I<opc, MRMSrcReg, (outs _.KRC:$dst), (ins _.RC:$src),
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[(set _.KRC:$dst, (X86cvt2mask (_.VT _.RC:$src)))]>, EVEX;
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
// Use 512bit version to implement 128/256 bit in case NoVLX.
multiclass convert_vector_to_mask_lowering<X86VectorVTInfo ExtendInfo,
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
X86VectorVTInfo _> {
def : Pat<(_.KVT (X86cvt2mask (_.VT _.RC:$src))),
(_.KVT (COPY_TO_REGCLASS
(!cast<Instruction>(NAME#"Zrr")
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
(INSERT_SUBREG (ExtendInfo.VT (IMPLICIT_DEF)),
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
_.RC:$src, _.SubRegIdx)),
_.KRC))>;
AVX-512: Added VPMOVx2M instructions for SKX, fixed encoding of VPMOVM2x. llvm-svn: 235385
2015-04-21 22:38:31 +08:00
}
multiclass avx512_convert_vector_to_mask<bits<8> opc, string OpcodeStr,
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : convert_vector_to_mask_common <opc, VTInfo.info512, OpcodeStr>,
EVEX_V512;
AVX-512: Added VPMOVx2M instructions for SKX, fixed encoding of VPMOVM2x. llvm-svn: 235385
2015-04-21 22:38:31 +08:00
let Predicates = [prd, HasVLX] in {
defm Z256 : convert_vector_to_mask_common<opc, VTInfo.info256, OpcodeStr>,
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
EVEX_V256;
AVX-512: Added VPMOVx2M instructions for SKX, fixed encoding of VPMOVM2x. llvm-svn: 235385
2015-04-21 22:38:31 +08:00
defm Z128 : convert_vector_to_mask_common<opc, VTInfo.info128, OpcodeStr>,
AVX512: Fix truncate v32i8 to v32i1 lowering implementation. Enable truncate 128/256bit packed byte/word with AVX512BW but without AVX512VL, use 512bit instructions. Differential Revision: http://reviews.llvm.org/D16531 llvm-svn: 259044
2016-01-28 21:19:25 +08:00
EVEX_V128;
}
let Predicates = [prd, NoVLX] in {
defm Z256_Alt : convert_vector_to_mask_lowering<VTInfo.info512, VTInfo.info256>;
defm Z128_Alt : convert_vector_to_mask_lowering<VTInfo.info512, VTInfo.info128>;
AVX-512: Added VPMOVx2M instructions for SKX, fixed encoding of VPMOVM2x. llvm-svn: 235385
2015-04-21 22:38:31 +08:00
}
}
defm VPMOVB2M : avx512_convert_vector_to_mask<0x29, "vpmovb2m",
avx512vl_i8_info, HasBWI>;
defm VPMOVW2M : avx512_convert_vector_to_mask<0x29, "vpmovw2m",
avx512vl_i16_info, HasBWI>, VEX_W;
defm VPMOVD2M : avx512_convert_vector_to_mask<0x39, "vpmovd2m",
avx512vl_i32_info, HasDQI>;
defm VPMOVQ2M : avx512_convert_vector_to_mask<0x39, "vpmovq2m",
avx512vl_i64_info, HasDQI>, VEX_W;
AVX-512: Added all forms of COMPRESS instruction + intrinsics + tests llvm-svn: 224019
2014-12-11 23:02:24 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - COMPRESS and EXPAND
//
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
AVX-512: Added all forms of COMPRESS instruction + intrinsics + tests llvm-svn: 224019
2014-12-11 23:02:24 +08:00
multiclass compress_by_vec_width<bits<8> opc, X86VectorVTInfo _,
string OpcodeStr> {
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
defm rr : AVX512_maskable<opc, MRMDestReg, _, (outs _.RC:$dst),
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(ins _.RC:$src1), OpcodeStr, "$src1", "$src1",
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
(_.VT (X86compress _.RC:$src1))>, AVX5128IBase;
AVX-512: Added all forms of COMPRESS instruction + intrinsics + tests llvm-svn: 224019
2014-12-11 23:02:24 +08:00
let mayStore = 1 in {
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
def mr : AVX5128I<opc, MRMDestMem, (outs),
(ins _.MemOp:$dst, _.RC:$src),
[AVX-512] Remove more superfluous spaces from asm strings. llvm-svn: 257301
2016-01-11 08:44:58 +08:00
OpcodeStr # "\t{$src, $dst|$dst, $src}",
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
[]>, EVEX_CD8<_.EltSize, CD8VT1>;
AVX-512: Added all forms of COMPRESS instruction + intrinsics + tests llvm-svn: 224019
2014-12-11 23:02:24 +08:00
def mrk : AVX5128I<opc, MRMDestMem, (outs),
(ins _.MemOp:$dst, _.KRCWM:$mask, _.RC:$src),
[AVX-512] Remove more superfluous spaces from asm strings. llvm-svn: 257301
2016-01-11 08:44:58 +08:00
OpcodeStr # "\t{$src, $dst {${mask}}|$dst {${mask}}, $src}",
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
[(store (_.VT (vselect _.KRCWM:$mask,
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
(_.VT (X86compress _.RC:$src)), _.ImmAllZerosV)),
AVX-512: Added all forms of COMPRESS instruction + intrinsics + tests llvm-svn: 224019
2014-12-11 23:02:24 +08:00
addr:$dst)]>,
EVEX_K, EVEX_CD8<_.EltSize, CD8VT1>;
}
}
multiclass compress_by_elt_width<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
defm Z : compress_by_vec_width<opc, VTInfo.info512, OpcodeStr>, EVEX_V512;
let Predicates = [HasVLX] in {
defm Z256 : compress_by_vec_width<opc, VTInfo.info256, OpcodeStr>, EVEX_V256;
defm Z128 : compress_by_vec_width<opc, VTInfo.info128, OpcodeStr>, EVEX_V128;
}
}
defm VPCOMPRESSD : compress_by_elt_width <0x8B, "vpcompressd", avx512vl_i32_info>,
EVEX;
defm VPCOMPRESSQ : compress_by_elt_width <0x8B, "vpcompressq", avx512vl_i64_info>,
EVEX, VEX_W;
defm VCOMPRESSPS : compress_by_elt_width <0x8A, "vcompressps", avx512vl_f32_info>,
EVEX;
defm VCOMPRESSPD : compress_by_elt_width <0x8A, "vcompresspd", avx512vl_f64_info>,
EVEX, VEX_W;
AVX-512: Added EXPAND instructions and intrinsics. llvm-svn: 224241
2014-12-15 18:03:52 +08:00
// expand
multiclass expand_by_vec_width<bits<8> opc, X86VectorVTInfo _,
string OpcodeStr> {
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
Removing tailing whitespaces llvm-svn: 244203
2015-08-06 17:06:20 +08:00
(ins _.RC:$src1), OpcodeStr, "$src1", "$src1",
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
(_.VT (X86expand _.RC:$src1))>, AVX5128IBase;
AVX-512: added all forms of VPSHUFD and VPSHUFHW, VPSHUFLW including encodings. llvm-svn: 238729
2015-06-01 15:17:23 +08:00
AVX-512: Added EXPAND instructions and intrinsics. llvm-svn: 224241
2014-12-15 18:03:52 +08:00
let mayLoad = 1 in
AVX-512: All forms of VCOPMRESS VEXPAND instructions, encoding tests. llvm-svn: 240272
2015-06-22 19:16:30 +08:00
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.MemOp:$src1), OpcodeStr, "$src1", "$src1",
(_.VT (X86expand (_.VT (bitconvert
(_.LdFrag addr:$src1)))))>,
AVX5128IBase, EVEX_CD8<_.EltSize, CD8VT1>;
AVX-512: Added EXPAND instructions and intrinsics. llvm-svn: 224241
2014-12-15 18:03:52 +08:00
}
multiclass expand_by_elt_width<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
defm Z : expand_by_vec_width<opc, VTInfo.info512, OpcodeStr>, EVEX_V512;
let Predicates = [HasVLX] in {
defm Z256 : expand_by_vec_width<opc, VTInfo.info256, OpcodeStr>, EVEX_V256;
defm Z128 : expand_by_vec_width<opc, VTInfo.info128, OpcodeStr>, EVEX_V128;
}
}
defm VPEXPANDD : expand_by_elt_width <0x89, "vpexpandd", avx512vl_i32_info>,
EVEX;
defm VPEXPANDQ : expand_by_elt_width <0x89, "vpexpandq", avx512vl_i64_info>,
EVEX, VEX_W;
defm VEXPANDPS : expand_by_elt_width <0x88, "vexpandps", avx512vl_f32_info>,
EVEX;
defm VEXPANDPD : expand_by_elt_width <0x88, "vexpandpd", avx512vl_f64_info>,
EVEX, VEX_W;
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
//handle instruction reg_vec1 = op(reg_vec,imm)
// op(mem_vec,imm)
// op(broadcast(eltVt),imm)
//all instruction created with FROUND_CURRENT
multiclass avx512_unary_fp_packed_imm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _>{
defm rri : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, i32u8imm:$src2),
AVX512F: Add assembler Intel syntax tests for knl, fix minor bugs. Differential Revision: http://reviews.llvm.org/D17498 llvm-svn: 261521
2016-02-22 20:37:41 +08:00
OpcodeStr##_.Suffix, "$src2, $src1", "$src1, $src2",
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(OpNode (_.VT _.RC:$src1),
(i32 imm:$src2),
(i32 FROUND_CURRENT))>;
let mayLoad = 1 in {
defm rmi : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.MemOp:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix, "$src2, $src1", "$src1, $src2",
(OpNode (_.VT (bitconvert (_.LdFrag addr:$src1))),
(i32 imm:$src2),
(i32 FROUND_CURRENT))>;
defm rmbi : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.ScalarMemOp:$src1, i32u8imm:$src2),
OpcodeStr##_.Suffix, "$src2, ${src1}"##_.BroadcastStr,
"${src1}"##_.BroadcastStr##", $src2",
(OpNode (_.VT (X86VBroadcast(_.ScalarLdFrag addr:$src1))),
(i32 imm:$src2),
(i32 FROUND_CURRENT))>, EVEX_B;
}
}
//handle instruction reg_vec1 = op(reg_vec2,reg_vec3,imm),{sae}
multiclass avx512_unary_fp_sae_packed_imm<bits<8> opc, string OpcodeStr,
SDNode OpNode, X86VectorVTInfo _>{
defm rrib : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, i32u8imm:$src2),
[AVX-512] Make spacing between comma and {sae} operand consistent in asm strings. llvm-svn: 257299
2016-01-11 08:44:52 +08:00
OpcodeStr##_.Suffix, "$src2, {sae}, $src1",
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
"$src1, {sae}, $src2",
(OpNode (_.VT _.RC:$src1),
(i32 imm:$src2),
(i32 FROUND_NO_EXC))>, EVEX_B;
}
multiclass avx512_common_unary_fp_sae_packed_imm<string OpcodeStr,
AVX512VLVectorVTInfo _, bits<8> opc, SDNode OpNode, Predicate prd>{
let Predicates = [prd] in {
defm Z : avx512_unary_fp_packed_imm<opc, OpcodeStr, OpNode, _.info512>,
avx512_unary_fp_sae_packed_imm<opc, OpcodeStr, OpNode, _.info512>,
EVEX_V512;
}
let Predicates = [prd, HasVLX] in {
defm Z128 : avx512_unary_fp_packed_imm<opc, OpcodeStr, OpNode, _.info128>,
EVEX_V128;
defm Z256 : avx512_unary_fp_packed_imm<opc, OpcodeStr, OpNode, _.info256>,
EVEX_V256;
}
}
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
//handle instruction reg_vec1 = op(reg_vec2,reg_vec3,imm)
// op(reg_vec2,mem_vec,imm)
// op(reg_vec2,broadcast(eltVt),imm)
//all instruction created with FROUND_CURRENT
multiclass avx512_fp_packed_imm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _>{
defm rri : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(ins _.RC:$src1, _.RC:$src2, i32u8imm:$src3),
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
OpcodeStr, "$src3, $src2, $src1", "$src1, $src2, $src3",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(i32 imm:$src3),
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
(i32 FROUND_CURRENT))>;
let mayLoad = 1 in {
defm rmi : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(ins _.RC:$src1, _.MemOp:$src2, i32u8imm:$src3),
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
OpcodeStr, "$src3, $src2, $src1", "$src1, $src2, $src3",
(OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2))),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(i32 imm:$src3),
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
(i32 FROUND_CURRENT))>;
defm rmbi : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2, i32u8imm:$src3),
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
OpcodeStr, "$src3, ${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr##", $src3",
(OpNode (_.VT _.RC:$src1),
(_.VT (X86VBroadcast(_.ScalarLdFrag addr:$src2))),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(i32 imm:$src3),
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
(i32 FROUND_CURRENT))>, EVEX_B;
}
}
AVX512: Implemented encoding and intrinsics for vpalignr Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12270 llvm-svn: 246428
2015-08-31 19:14:02 +08:00
//handle instruction reg_vec1 = op(reg_vec2,reg_vec3,imm)
// op(reg_vec2,mem_vec,imm)
multiclass avx512_3Op_rm_imm8<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo DestInfo, X86VectorVTInfo SrcInfo>{
defm rri : AVX512_maskable<opc, MRMSrcReg, DestInfo, (outs DestInfo.RC:$dst),
(ins SrcInfo.RC:$src1, SrcInfo.RC:$src2, u8imm:$src3),
OpcodeStr, "$src3, $src2, $src1", "$src1, $src2, $src3",
(DestInfo.VT (OpNode (SrcInfo.VT SrcInfo.RC:$src1),
(SrcInfo.VT SrcInfo.RC:$src2),
(i8 imm:$src3)))>;
let mayLoad = 1 in
defm rmi : AVX512_maskable<opc, MRMSrcMem, DestInfo, (outs DestInfo.RC:$dst),
(ins SrcInfo.RC:$src1, SrcInfo.MemOp:$src2, u8imm:$src3),
OpcodeStr, "$src3, $src2, $src1", "$src1, $src2, $src3",
(DestInfo.VT (OpNode (SrcInfo.VT SrcInfo.RC:$src1),
(SrcInfo.VT (bitconvert
(SrcInfo.LdFrag addr:$src2))),
(i8 imm:$src3)))>;
}
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
//handle instruction reg_vec1 = op(reg_vec2,reg_vec3,imm)
// op(reg_vec2,mem_vec,imm)
// op(reg_vec2,broadcast(eltVt),imm)
multiclass avx512_3Op_imm8<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512: Implemented encoding and intrinsics for vpalignr Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12270 llvm-svn: 246428
2015-08-31 19:14:02 +08:00
X86VectorVTInfo _>:
avx512_3Op_rm_imm8<opc, OpcodeStr, OpNode, _, _>{
let mayLoad = 1 in
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
defm rmbi : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2, u8imm:$src3),
OpcodeStr, "$src3, ${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr##", $src3",
(OpNode (_.VT _.RC:$src1),
(_.VT (X86VBroadcast(_.ScalarLdFrag addr:$src2))),
(i8 imm:$src3))>, EVEX_B;
}
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
//handle scalar instruction reg_vec1 = op(reg_vec2,reg_vec3,imm)
// op(reg_vec2,mem_scalar,imm)
//all instruction created with FROUND_CURRENT
multiclass avx512_fp_scalar_imm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm rri : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(ins _.RC:$src1, _.RC:$src2, i32u8imm:$src3),
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
OpcodeStr, "$src3, $src2, $src1", "$src1, $src2, $src3",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(i32 imm:$src3),
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
(i32 FROUND_CURRENT))>;
let mayLoad = 1 in {
defm rmi : AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(ins _.RC:$src1, _.MemOp:$src2, i32u8imm:$src3),
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
OpcodeStr, "$src3, $src2, $src1", "$src1, $src2, $src3",
(OpNode (_.VT _.RC:$src1),
(_.VT (scalar_to_vector
(_.ScalarLdFrag addr:$src2))),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(i32 imm:$src3),
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
(i32 FROUND_CURRENT))>;
let isAsmParserOnly = 1 in {
defm rmi_alt :AVX512_maskable_in_asm<opc, MRMSrcMem, _, (outs _.FRC:$dst),
(ins _.FRC:$src1, _.ScalarMemOp:$src2, u8imm:$src3),
OpcodeStr, "$src3, $src2, $src1", "$src1, $src2, $src3",
[]>;
}
}
}
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
//handle instruction reg_vec1 = op(reg_vec2,reg_vec3,imm),{sae}
multiclass avx512_fp_sae_packed_imm<bits<8> opc, string OpcodeStr,
SDNode OpNode, X86VectorVTInfo _>{
defm rrib : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(ins _.RC:$src1, _.RC:$src2, i32u8imm:$src3),
[AVX-512] Make spacing between comma and {sae} operand consistent in asm strings. llvm-svn: 257299
2016-01-11 08:44:52 +08:00
OpcodeStr, "$src3, {sae}, $src2, $src1",
"$src1, $src2, {sae}, $src3",
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(i32 imm:$src3),
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
(i32 FROUND_NO_EXC))>, EVEX_B;
}
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
//handle scalar instruction reg_vec1 = op(reg_vec2,reg_vec3,imm),{sae}
multiclass avx512_fp_sae_scalar_imm<bits<8> opc, string OpcodeStr,
SDNode OpNode, X86VectorVTInfo _> {
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
defm NAME#rrib : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2, i32u8imm:$src3),
[AVX-512] Make spacing between comma and {sae} operand consistent in asm strings. llvm-svn: 257299
2016-01-11 08:44:52 +08:00
OpcodeStr, "$src3, {sae}, $src2, $src1",
"$src1, $src2, {sae}, $src3",
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(i32 imm:$src3),
(i32 FROUND_NO_EXC))>, EVEX_B;
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
}
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
AVX-512: Implemented VRANGEPD and VRANGEPD instructions for SKX. Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238738
2015-06-01 19:05:34 +08:00
multiclass avx512_common_fp_sae_packed_imm<string OpcodeStr,
AVX512VLVectorVTInfo _, bits<8> opc, SDNode OpNode, Predicate prd>{
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
let Predicates = [prd] in {
defm Z : avx512_fp_packed_imm<opc, OpcodeStr, OpNode, _.info512>,
AVX-512: Implemented 256/128bit VALIGND/Q instructions for SKX and KNL Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. Differential Revision: http://reviews.llvm.org/D10310 llvm-svn: 239300
2015-06-08 22:03:17 +08:00
avx512_fp_sae_packed_imm<opc, OpcodeStr, OpNode, _.info512>,
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
EVEX_V512;
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
}
let Predicates = [prd, HasVLX] in {
defm Z128 : avx512_fp_packed_imm<opc, OpcodeStr, OpNode, _.info128>,
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
EVEX_V128;
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
defm Z256 : avx512_fp_packed_imm<opc, OpcodeStr, OpNode, _.info256>,
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
EVEX_V256;
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
}
AVX-512: Implemented VFIXUPIMMPD and VFIXUPIMMPS instructions for KNL and SKX Implemented DAG lowering for all these forms. Added tests for encoding. by Igor Breger (igor.breger@intel.com) llvm-svn: 238728
2015-06-01 14:50:49 +08:00
}
AVX512: Implemented encoding and intrinsics for vpalignr Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12270 llvm-svn: 246428
2015-08-31 19:14:02 +08:00
multiclass avx512_common_3Op_rm_imm8<bits<8> opc, SDNode OpNode, string OpStr,
AVX512VLVectorVTInfo DestInfo, AVX512VLVectorVTInfo SrcInfo>{
let Predicates = [HasBWI] in {
defm Z : avx512_3Op_rm_imm8<opc, OpStr, OpNode, DestInfo.info512,
SrcInfo.info512>, EVEX_V512, AVX512AIi8Base, EVEX_4V;
}
let Predicates = [HasBWI, HasVLX] in {
defm Z128 : avx512_3Op_rm_imm8<opc, OpStr, OpNode, DestInfo.info128,
SrcInfo.info128>, EVEX_V128, AVX512AIi8Base, EVEX_4V;
defm Z256 : avx512_3Op_rm_imm8<opc, OpStr, OpNode, DestInfo.info256,
SrcInfo.info256>, EVEX_V256, AVX512AIi8Base, EVEX_4V;
}
}
AVX-512: Implemented 256/128bit VALIGND/Q instructions for SKX and KNL Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. Differential Revision: http://reviews.llvm.org/D10310 llvm-svn: 239300
2015-06-08 22:03:17 +08:00
multiclass avx512_common_3Op_imm8<string OpcodeStr, AVX512VLVectorVTInfo _,
bits<8> opc, SDNode OpNode>{
let Predicates = [HasAVX512] in {
defm Z : avx512_3Op_imm8<opc, OpcodeStr, OpNode, _.info512>, EVEX_V512;
}
let Predicates = [HasAVX512, HasVLX] in {
defm Z128 : avx512_3Op_imm8<opc, OpcodeStr, OpNode, _.info128>, EVEX_V128;
defm Z256 : avx512_3Op_imm8<opc, OpcodeStr, OpNode, _.info256>, EVEX_V256;
}
}
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
multiclass avx512_common_fp_sae_scalar_imm<string OpcodeStr,
X86VectorVTInfo _, bits<8> opc, SDNode OpNode, Predicate prd>{
let Predicates = [prd] in {
defm Z128 : avx512_fp_scalar_imm<opc, OpcodeStr, OpNode, _>,
avx512_fp_sae_scalar_imm<opc, OpcodeStr, OpNode, _>;
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
}
AVX-512: Implemented VFIXUPIMMSD and VFIXUPIMMSS instructions for KNL Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238811
2015-06-02 16:28:57 +08:00
}
AVX512: Implemented encoding and intrinsics for VGETMANTPD/S , VGETMANTSD/S instructions Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11593 llvm-svn: 246642
2015-09-02 19:18:55 +08:00
multiclass avx512_common_unary_fp_sae_packed_imm_all<string OpcodeStr,
bits<8> opcPs, bits<8> opcPd, SDNode OpNode, Predicate prd>{
defm PS : avx512_common_unary_fp_sae_packed_imm<OpcodeStr, avx512vl_f32_info,
opcPs, OpNode, prd>, EVEX_CD8<32, CD8VF>;
defm PD : avx512_common_unary_fp_sae_packed_imm<OpcodeStr, avx512vl_f64_info,
opcPd, OpNode, prd>, EVEX_CD8<64, CD8VF>, VEX_W;
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
}
AVX-512: Implemented VRANGEPD and VRANGEPD instructions for SKX. Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238738
2015-06-01 19:05:34 +08:00
AVX512: Implemented encoding and intrinsics for VGETMANTPD/S , VGETMANTSD/S instructions Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11593 llvm-svn: 246642
2015-09-02 19:18:55 +08:00
defm VREDUCE : avx512_common_unary_fp_sae_packed_imm_all<"vreduce", 0x56, 0x56,
X86VReduce, HasDQI>, AVX512AIi8Base, EVEX;
defm VRNDSCALE : avx512_common_unary_fp_sae_packed_imm_all<"vrndscale", 0x08, 0x09,
X86VRndScale, HasAVX512>, AVX512AIi8Base, EVEX;
defm VGETMANT : avx512_common_unary_fp_sae_packed_imm_all<"vgetmant", 0x26, 0x26,
X86VGetMant, HasAVX512>, AVX512AIi8Base, EVEX;
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
AVX-512: Implemented VRANGEPD and VRANGEPD instructions for SKX. Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238738
2015-06-01 19:05:34 +08:00
defm VRANGEPD : avx512_common_fp_sae_packed_imm<"vrangepd", avx512vl_f64_info,
0x50, X86VRange, HasDQI>,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<64, CD8VF>, VEX_W;
defm VRANGEPS : avx512_common_fp_sae_packed_imm<"vrangeps", avx512vl_f32_info,
0x50, X86VRange, HasDQI>,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<32, CD8VF>;
AVX-512: Implemented VRANGESD and VRANGESS instructions for SKX Implemented DAG lowering for all these forms. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238834
2015-06-02 22:12:54 +08:00
defm VRANGESD: avx512_common_fp_sae_scalar_imm<"vrangesd", f64x_info,
0x51, X86VRange, HasDQI>,
AVX512AIi8Base, VEX_LIG, EVEX_4V, EVEX_CD8<64, CD8VT1>, VEX_W;
defm VRANGESS: avx512_common_fp_sae_scalar_imm<"vrangess", f32x_info,
0x51, X86VRange, HasDQI>,
AVX512AIi8Base, VEX_LIG, EVEX_4V, EVEX_CD8<32, CD8VT1>;
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
defm VREDUCESD: avx512_common_fp_sae_scalar_imm<"vreducesd", f64x_info,
0x57, X86Reduces, HasDQI>,
AVX512AIi8Base, VEX_LIG, EVEX_4V, EVEX_CD8<64, CD8VT1>, VEX_W;
defm VREDUCESS: avx512_common_fp_sae_scalar_imm<"vreducess", f32x_info,
0x57, X86Reduces, HasDQI>,
AVX512AIi8Base, VEX_LIG, EVEX_4V, EVEX_CD8<32, CD8VT1>;
AVX512: Implemented encoding and intrinsics for VGETMANTPD/S , VGETMANTSD/S instructions Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11593 llvm-svn: 246642
2015-09-02 19:18:55 +08:00
defm VGETMANTSD: avx512_common_fp_sae_scalar_imm<"vgetmantsd", f64x_info,
0x27, X86GetMants, HasAVX512>,
AVX512AIi8Base, VEX_LIG, EVEX_4V, EVEX_CD8<64, CD8VT1>, VEX_W;
defm VGETMANTSS: avx512_common_fp_sae_scalar_imm<"vgetmantss", f32x_info,
0x27, X86GetMants, HasAVX512>,
AVX512AIi8Base, VEX_LIG, EVEX_4V, EVEX_CD8<32, CD8VT1>;
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
multiclass avx512_shuff_packed_128<string OpcodeStr, AVX512VLVectorVTInfo _,
bits<8> opc, SDNode OpNode = X86Shuf128>{
let Predicates = [HasAVX512] in {
defm Z : avx512_3Op_imm8<opc, OpcodeStr, OpNode, _.info512>, EVEX_V512;
}
let Predicates = [HasAVX512, HasVLX] in {
defm Z256 : avx512_3Op_imm8<opc, OpcodeStr, OpNode, _.info256>, EVEX_V256;
}
}
[X86][AVX512] add reduce/range/scalef/rndScale include encoding and intrinsics Differential Revision: http://reviews.llvm.org/D11222 llvm-svn: 242896
2015-07-22 20:00:43 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(v16f32 (ffloor VR512:$src)),
(VRNDSCALEPSZrri VR512:$src, (i32 0x1))>;
def : Pat<(v16f32 (fnearbyint VR512:$src)),
(VRNDSCALEPSZrri VR512:$src, (i32 0xC))>;
def : Pat<(v16f32 (fceil VR512:$src)),
(VRNDSCALEPSZrri VR512:$src, (i32 0x2))>;
def : Pat<(v16f32 (frint VR512:$src)),
(VRNDSCALEPSZrri VR512:$src, (i32 0x4))>;
def : Pat<(v16f32 (ftrunc VR512:$src)),
(VRNDSCALEPSZrri VR512:$src, (i32 0x3))>;
def : Pat<(v8f64 (ffloor VR512:$src)),
(VRNDSCALEPDZrri VR512:$src, (i32 0x1))>;
def : Pat<(v8f64 (fnearbyint VR512:$src)),
(VRNDSCALEPDZrri VR512:$src, (i32 0xC))>;
def : Pat<(v8f64 (fceil VR512:$src)),
(VRNDSCALEPDZrri VR512:$src, (i32 0x2))>;
def : Pat<(v8f64 (frint VR512:$src)),
(VRNDSCALEPDZrri VR512:$src, (i32 0x4))>;
def : Pat<(v8f64 (ftrunc VR512:$src)),
(VRNDSCALEPDZrri VR512:$src, (i32 0x3))>;
}
AVX-512: Implemented SHUFF32x4/SHUFF64x2/SHUFI32x4/SHUFI64x2 instructions for SKX and KNL. Added tests for encoding. By Igor Breger (igor.breger@intel.com) llvm-svn: 238917
2015-06-03 18:56:40 +08:00
defm VSHUFF32X4 : avx512_shuff_packed_128<"vshuff32x4",avx512vl_f32_info, 0x23>,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<32, CD8VF>;
defm VSHUFF64X2 : avx512_shuff_packed_128<"vshuff64x2",avx512vl_f64_info, 0x23>,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<64, CD8VF>, VEX_W;
defm VSHUFI32X4 : avx512_shuff_packed_128<"vshufi32x4",avx512vl_i32_info, 0x43>,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<32, CD8VF>;
defm VSHUFI64X2 : avx512_shuff_packed_128<"vshufi64x2",avx512vl_i64_info, 0x43>,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<64, CD8VF>, VEX_W;
AVX-512: Implemented 256/128bit VALIGND/Q instructions for SKX and KNL Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. Differential Revision: http://reviews.llvm.org/D10310 llvm-svn: 239300
2015-06-08 22:03:17 +08:00
[AVX512] Remove alternate data type versions of VALIGND, VALIGNQ, VMOVSHDUP and VMOVSLDUP. They don't have any tests and I don't think they can be selected. If they are truly needed they should be implemented with patterns against the normal instructions and not separate instructions. llvm-svn: 256475
2015-12-28 03:45:21 +08:00
multiclass avx512_valign<string OpcodeStr, AVX512VLVectorVTInfo VTInfo_I> {
AVX-512: Implemented 256/128bit VALIGND/Q instructions for SKX and KNL Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. Differential Revision: http://reviews.llvm.org/D10310 llvm-svn: 239300
2015-06-08 22:03:17 +08:00
defm NAME: avx512_common_3Op_imm8<OpcodeStr, VTInfo_I, 0x03, X86VAlign>,
AVX512AIi8Base, EVEX_4V;
}
[AVX512] Remove alternate data type versions of VALIGND, VALIGNQ, VMOVSHDUP and VMOVSLDUP. They don't have any tests and I don't think they can be selected. If they are truly needed they should be implemented with patterns against the normal instructions and not separate instructions. llvm-svn: 256475
2015-12-28 03:45:21 +08:00
defm VALIGND: avx512_valign<"valignd", avx512vl_i32_info>,
AVX-512: Implemented 256/128bit VALIGND/Q instructions for SKX and KNL Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. Differential Revision: http://reviews.llvm.org/D10310 llvm-svn: 239300
2015-06-08 22:03:17 +08:00
EVEX_CD8<32, CD8VF>;
[AVX512] Remove alternate data type versions of VALIGND, VALIGNQ, VMOVSHDUP and VMOVSLDUP. They don't have any tests and I don't think they can be selected. If they are truly needed they should be implemented with patterns against the normal instructions and not separate instructions. llvm-svn: 256475
2015-12-28 03:45:21 +08:00
defm VALIGNQ: avx512_valign<"valignq", avx512vl_i64_info>,
AVX-512: Implemented 256/128bit VALIGND/Q instructions for SKX and KNL Implemented DAG lowering for all these forms. Added tests for DAG lowering and encoding. Differential Revision: http://reviews.llvm.org/D10310 llvm-svn: 239300
2015-06-08 22:03:17 +08:00
EVEX_CD8<64, CD8VF>, VEX_W;
AVX-512: Added all forms of VPABS instruction Added all intrinsics, tests for encoding, tests for intrinsics. llvm-svn: 240386
2015-06-23 16:19:46 +08:00
AVX512: Implemented encoding and intrinsics for vpalignr Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12270 llvm-svn: 246428
2015-08-31 19:14:02 +08:00
multiclass avx512_vpalign_lowering<X86VectorVTInfo _ , list<Predicate> p>{
let Predicates = p in
def NAME#_.VTName#rri:
Pat<(_.VT (X86PAlignr _.RC:$src1, _.RC:$src2, (i8 imm:$imm))),
(!cast<Instruction>(NAME#_.ZSuffix#rri)
_.RC:$src1, _.RC:$src2, imm:$imm)>;
}
multiclass avx512_vpalign_lowering_common<AVX512VLVectorVTInfo _>:
avx512_vpalign_lowering<_.info512, [HasBWI]>,
avx512_vpalign_lowering<_.info128, [HasBWI, HasVLX]>,
avx512_vpalign_lowering<_.info256, [HasBWI, HasVLX]>;
defm VPALIGN: avx512_common_3Op_rm_imm8<0x0F, X86PAlignr, "vpalignr" ,
avx512vl_i8_info, avx512vl_i8_info>,
avx512_vpalign_lowering_common<avx512vl_i16_info>,
avx512_vpalign_lowering_common<avx512vl_i32_info>,
avx512_vpalign_lowering_common<avx512vl_f32_info>,
avx512_vpalign_lowering_common<avx512vl_i64_info>,
avx512_vpalign_lowering_common<avx512vl_f64_info>,
EVEX_CD8<8, CD8VF>;
AVX512: Implemented encoding and intrinsics for vdbpsadbw Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D12491 llvm-svn: 246436
2015-08-31 21:09:30 +08:00
defm VDBPSADBW: avx512_common_3Op_rm_imm8<0x42, X86dbpsadbw, "vdbpsadbw" ,
avx512vl_i16_info, avx512vl_i8_info>, EVEX_CD8<8, CD8VF>;
AVX-512: Added all forms of VPABS instruction Added all intrinsics, tests for encoding, tests for intrinsics. llvm-svn: 240386
2015-06-23 16:19:46 +08:00
multiclass avx512_unary_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
AVX512: Implemented encoding and intrinsics for VMOVSHDUP/VMOVSLDUP instructions. Differential Revision: http://reviews.llvm.org/D14322 llvm-svn: 253185
2015-11-16 15:22:00 +08:00
(ins _.RC:$src1), OpcodeStr,
AVX-512: Added all forms of VPABS instruction Added all intrinsics, tests for encoding, tests for intrinsics. llvm-svn: 240386
2015-06-23 16:19:46 +08:00
"$src1", "$src1",
(_.VT (OpNode _.RC:$src1))>, EVEX, AVX5128IBase;
let mayLoad = 1 in
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Implemented encoding and intrinsics for VMOVSHDUP/VMOVSLDUP instructions. Differential Revision: http://reviews.llvm.org/D14322 llvm-svn: 253185
2015-11-16 15:22:00 +08:00
(ins _.MemOp:$src1), OpcodeStr,
AVX-512: Added all forms of VPABS instruction Added all intrinsics, tests for encoding, tests for intrinsics. llvm-svn: 240386
2015-06-23 16:19:46 +08:00
"$src1", "$src1",
(_.VT (OpNode (bitconvert (_.LdFrag addr:$src1))))>,
EVEX, AVX5128IBase, EVEX_CD8<_.EltSize, CD8VF>;
}
multiclass avx512_unary_rmb<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> :
avx512_unary_rm<opc, OpcodeStr, OpNode, _> {
let mayLoad = 1 in
defm rmb : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
AVX512: Implemented encoding and intrinsics for VMOVSHDUP/VMOVSLDUP instructions. Differential Revision: http://reviews.llvm.org/D14322 llvm-svn: 253185
2015-11-16 15:22:00 +08:00
(ins _.ScalarMemOp:$src1), OpcodeStr,
AVX-512: Added all forms of VPABS instruction Added all intrinsics, tests for encoding, tests for intrinsics. llvm-svn: 240386
2015-06-23 16:19:46 +08:00
"${src1}"##_.BroadcastStr,
"${src1}"##_.BroadcastStr,
(_.VT (OpNode (X86VBroadcast
(_.ScalarLdFrag addr:$src1))))>,
EVEX, AVX5128IBase, EVEX_B, EVEX_CD8<_.EltSize, CD8VF>;
}
multiclass avx512_unary_rm_vl<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_unary_rm<opc, OpcodeStr, OpNode, VTInfo.info512>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_unary_rm<opc, OpcodeStr, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_unary_rm<opc, OpcodeStr, OpNode, VTInfo.info128>,
EVEX_V128;
}
}
multiclass avx512_unary_rmb_vl<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_unary_rmb<opc, OpcodeStr, OpNode, VTInfo.info512>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_unary_rmb<opc, OpcodeStr, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_unary_rmb<opc, OpcodeStr, OpNode, VTInfo.info128>,
EVEX_V128;
}
}
multiclass avx512_unary_rm_vl_dq<bits<8> opc_d, bits<8> opc_q, string OpcodeStr,
SDNode OpNode, Predicate prd> {
AVX512: Implemented encoding and intrinsics for VMOVSHDUP/VMOVSLDUP instructions. Differential Revision: http://reviews.llvm.org/D14322 llvm-svn: 253185
2015-11-16 15:22:00 +08:00
defm Q : avx512_unary_rmb_vl<opc_q, OpcodeStr#"q", OpNode, avx512vl_i64_info,
AVX-512: Added all forms of VPABS instruction Added all intrinsics, tests for encoding, tests for intrinsics. llvm-svn: 240386
2015-06-23 16:19:46 +08:00
prd>, VEX_W;
AVX512: Implemented encoding and intrinsics for VMOVSHDUP/VMOVSLDUP instructions. Differential Revision: http://reviews.llvm.org/D14322 llvm-svn: 253185
2015-11-16 15:22:00 +08:00
defm D : avx512_unary_rmb_vl<opc_d, OpcodeStr#"d", OpNode, avx512vl_i32_info,
prd>;
AVX-512: Added all forms of VPABS instruction Added all intrinsics, tests for encoding, tests for intrinsics. llvm-svn: 240386
2015-06-23 16:19:46 +08:00
}
multiclass avx512_unary_rm_vl_bw<bits<8> opc_b, bits<8> opc_w, string OpcodeStr,
SDNode OpNode, Predicate prd> {
AVX512: Implemented encoding and intrinsics for VMOVSHDUP/VMOVSLDUP instructions. Differential Revision: http://reviews.llvm.org/D14322 llvm-svn: 253185
2015-11-16 15:22:00 +08:00
defm W : avx512_unary_rm_vl<opc_w, OpcodeStr#"w", OpNode, avx512vl_i16_info, prd>;
defm B : avx512_unary_rm_vl<opc_b, OpcodeStr#"b", OpNode, avx512vl_i8_info, prd>;
AVX-512: Added all forms of VPABS instruction Added all intrinsics, tests for encoding, tests for intrinsics. llvm-svn: 240386
2015-06-23 16:19:46 +08:00
}
multiclass avx512_unary_rm_vl_all<bits<8> opc_b, bits<8> opc_w,
bits<8> opc_d, bits<8> opc_q,
string OpcodeStr, SDNode OpNode> {
defm NAME : avx512_unary_rm_vl_dq<opc_d, opc_q, OpcodeStr, OpNode,
HasAVX512>,
avx512_unary_rm_vl_bw<opc_b, opc_w, OpcodeStr, OpNode,
HasBWI>;
}
defm VPABS : avx512_unary_rm_vl_all<0x1C, 0x1D, 0x1E, 0x1F, "vpabs", X86Abs>;
def : Pat<(xor
(bc_v16i32 (v16i1sextv16i32)),
(bc_v16i32 (add (v16i32 VR512:$src), (v16i1sextv16i32)))),
(VPABSDZrr VR512:$src)>;
def : Pat<(xor
(bc_v8i64 (v8i1sextv8i64)),
(bc_v8i64 (add (v8i64 VR512:$src), (v8i1sextv8i64)))),
(VPABSQZrr VR512:$src)>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
AVX512: Implemented encoding and intrinsics for vplzcntq, vplzcntd, vpconflictq, vpconflictd Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11931 llvm-svn: 246750
2015-09-03 17:05:31 +08:00
multiclass avx512_ctlz<bits<8> opc, string OpcodeStr, Predicate prd>{
defm NAME : avx512_unary_rm_vl_dq<opc, opc, OpcodeStr, ctlz, prd>;
}
defm VPLZCNT : avx512_ctlz<0x44, "vplzcnt", HasCDI>;
defm VPCONFLICT : avx512_unary_rm_vl_dq<0xC4, 0xC4, "vpconflict", X86Conflict, HasCDI>;
AVX512: Implemented encoding and intrinsics for VMOVSHDUP/VMOVSLDUP instructions. Differential Revision: http://reviews.llvm.org/D14322 llvm-svn: 253185
2015-11-16 15:22:00 +08:00
//===---------------------------------------------------------------------===//
// Replicate Single FP - MOVSHDUP and MOVSLDUP
//===---------------------------------------------------------------------===//
multiclass avx512_replicate<bits<8> opc, string OpcodeStr, SDNode OpNode>{
defm NAME: avx512_unary_rm_vl<opc, OpcodeStr, OpNode, avx512vl_f32_info,
HasAVX512>, XS;
}
defm VMOVSHDUP : avx512_replicate<0x16, "vmovshdup", X86Movshdup>;
defm VMOVSLDUP : avx512_replicate<0x12, "vmovsldup", X86Movsldup>;
AVX512: Implemented encoding, intrinsics and DAG lowering for VMOVDDUP instructions. Differential Revision: http://reviews.llvm.org/D14702 llvm-svn: 253548
2015-11-19 16:26:56 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - MOVDDUP
//===----------------------------------------------------------------------===//
multiclass avx512_movddup_128<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src), OpcodeStr, "$src", "$src",
(_.VT (OpNode (_.VT _.RC:$src)))>, EVEX;
let mayLoad = 1 in
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.ScalarMemOp:$src), OpcodeStr, "$src", "$src",
(_.VT (OpNode (_.VT (scalar_to_vector
(_.ScalarLdFrag addr:$src)))))>,
EVEX, EVEX_CD8<_.EltSize, CD8VH>;
}
multiclass avx512_movddup_common<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo> {
defm Z : avx512_unary_rm<opc, OpcodeStr, OpNode, VTInfo.info512>, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in {
defm Z256 : avx512_unary_rm<opc, OpcodeStr, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_movddup_128<opc, OpcodeStr, OpNode, VTInfo.info128>,
EVEX_V128;
}
}
multiclass avx512_movddup<bits<8> opc, string OpcodeStr, SDNode OpNode>{
defm NAME: avx512_movddup_common<opc, OpcodeStr, OpNode,
avx512vl_f64_info>, XD, VEX_W;
}
defm VMOVDDUP : avx512_movddup<0x12, "vmovddup", X86Movddup>;
def : Pat<(X86Movddup (loadv2f64 addr:$src)),
(VMOVDDUPZ128rm addr:$src)>, Requires<[HasAVX512, HasVLX]>;
def : Pat<(v2f64 (X86VBroadcast (loadf64 addr:$src))),
(VMOVDDUPZ128rm addr:$src)>, Requires<[HasAVX512, HasVLX]>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Unpack Instructions
//===----------------------------------------------------------------------===//
[AVX512] Make sure 128/256-bit DQI versions of VAND/VANDN/VOR/VXOR are also marked as requiring VLX. llvm-svn: 268186
2016-05-01 13:57:06 +08:00
defm VUNPCKH : avx512_fp_binop_p<0x15, "vunpckh", X86Unpckh, HasAVX512>;
defm VUNPCKL : avx512_fp_binop_p<0x14, "vunpckl", X86Unpckl, HasAVX512>;
Implemented encoding and intrinsics of the following instructions vunpckhps/pd, vunpcklps/pd, vpunpcklbw, vpunpckhbw, vpunpcklwd, vpunpckhwd, vpunpckldq, vpunpckhdq, vpunpcklqdq, vpunpckhqdq Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11509 llvm-svn: 243246
2015-07-26 22:41:44 +08:00
defm VPUNPCKLBW : avx512_binop_rm_vl_b<0x60, "vpunpcklbw", X86Unpckl,
SSE_INTALU_ITINS_P, HasBWI>;
defm VPUNPCKHBW : avx512_binop_rm_vl_b<0x68, "vpunpckhbw", X86Unpckh,
SSE_INTALU_ITINS_P, HasBWI>;
defm VPUNPCKLWD : avx512_binop_rm_vl_w<0x61, "vpunpcklwd", X86Unpckl,
SSE_INTALU_ITINS_P, HasBWI>;
defm VPUNPCKHWD : avx512_binop_rm_vl_w<0x69, "vpunpckhwd", X86Unpckh,
SSE_INTALU_ITINS_P, HasBWI>;
defm VPUNPCKLDQ : avx512_binop_rm_vl_d<0x62, "vpunpckldq", X86Unpckl,
SSE_INTALU_ITINS_P, HasAVX512>;
defm VPUNPCKHDQ : avx512_binop_rm_vl_d<0x6A, "vpunpckhdq", X86Unpckh,
SSE_INTALU_ITINS_P, HasAVX512>;
defm VPUNPCKLQDQ : avx512_binop_rm_vl_q<0x6C, "vpunpcklqdq", X86Unpckl,
SSE_INTALU_ITINS_P, HasAVX512>;
defm VPUNPCKHQDQ : avx512_binop_rm_vl_q<0x6D, "vpunpckhqdq", X86Unpckh,
SSE_INTALU_ITINS_P, HasAVX512>;
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Extract & Insert Integer Instructions
//===----------------------------------------------------------------------===//
multiclass avx512_extract_elt_bw_m<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
let mayStore = 1 in
def mr : AVX512Ii8<opc, MRMDestMem, (outs),
(ins _.ScalarMemOp:$dst, _.RC:$src1, u8imm:$src2),
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[(store (_.EltVT (trunc (assertzext (OpNode (_.VT _.RC:$src1),
imm:$src2)))),
addr:$dst)]>,
EVEX, EVEX_CD8<_.EltSize, CD8VT1>;
}
multiclass avx512_extract_elt_b<string OpcodeStr, X86VectorVTInfo _> {
let Predicates = [HasBWI] in {
def rr : AVX512Ii8<0x14, MRMDestReg, (outs GR32orGR64:$dst),
(ins _.RC:$src1, u8imm:$src2),
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[(set GR32orGR64:$dst,
(X86pextrb (_.VT _.RC:$src1), imm:$src2))]>,
EVEX, TAPD;
defm NAME : avx512_extract_elt_bw_m<0x14, OpcodeStr, X86pextrb, _>, TAPD;
}
}
multiclass avx512_extract_elt_w<string OpcodeStr, X86VectorVTInfo _> {
let Predicates = [HasBWI] in {
def rr : AVX512Ii8<0xC5, MRMSrcReg, (outs GR32orGR64:$dst),
(ins _.RC:$src1, u8imm:$src2),
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[(set GR32orGR64:$dst,
(X86pextrw (_.VT _.RC:$src1), imm:$src2))]>,
EVEX, PD;
[AVX512] Add hasSideEffects/mayLoad/mayStore flags to some instructions. llvm-svn: 268174
2016-05-01 09:03:56 +08:00
let hasSideEffects = 0 in
AVX512: Implemented encoding for vpextrw.s instruction. Differential Revision: http://reviews.llvm.org/D14766 llvm-svn: 253447
2015-11-18 16:46:16 +08:00
def rr_REV : AVX512Ii8<0x15, MRMDestReg, (outs GR32orGR64:$dst),
(ins _.RC:$src1, u8imm:$src2),
OpcodeStr#".s\t{$src2, $src1, $dst|$dst, $src1, $src2}", []>,
EVEX, TAPD;
AVX512: vpextrb/w/d/q and vpinsrb/w/d/q implementation. This instructions doesn't have intrincis. Added tests for lowering and encoding. Differential Revision: http://reviews.llvm.org/D12317 llvm-svn: 249688
2015-10-08 20:55:01 +08:00
defm NAME : avx512_extract_elt_bw_m<0x15, OpcodeStr, X86pextrw, _>, TAPD;
}
}
multiclass avx512_extract_elt_dq<string OpcodeStr, X86VectorVTInfo _,
RegisterClass GRC> {
let Predicates = [HasDQI] in {
def rr : AVX512Ii8<0x16, MRMDestReg, (outs GRC:$dst),
(ins _.RC:$src1, u8imm:$src2),
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[(set GRC:$dst,
(extractelt (_.VT _.RC:$src1), imm:$src2))]>,
EVEX, TAPD;
let mayStore = 1 in
def mr : AVX512Ii8<0x16, MRMDestMem, (outs),
(ins _.ScalarMemOp:$dst, _.RC:$src1, u8imm:$src2),
OpcodeStr#"\t{$src2, $src1, $dst|$dst, $src1, $src2}",
[(store (extractelt (_.VT _.RC:$src1),
imm:$src2),addr:$dst)]>,
EVEX, EVEX_CD8<_.EltSize, CD8VT1>, TAPD;
}
}
defm VPEXTRBZ : avx512_extract_elt_b<"vpextrb", v16i8x_info>;
defm VPEXTRWZ : avx512_extract_elt_w<"vpextrw", v8i16x_info>;
defm VPEXTRDZ : avx512_extract_elt_dq<"vpextrd", v4i32x_info, GR32>;
defm VPEXTRQZ : avx512_extract_elt_dq<"vpextrq", v2i64x_info, GR64>, VEX_W;
multiclass avx512_insert_elt_m<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _, PatFrag LdFrag> {
def rm : AVX512Ii8<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2, u8imm:$src3),
OpcodeStr#"\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}",
[(set _.RC:$dst,
(_.VT (OpNode _.RC:$src1, (LdFrag addr:$src2), imm:$src3)))]>,
EVEX_4V, EVEX_CD8<_.EltSize, CD8VT1>;
}
multiclass avx512_insert_elt_bw<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _, PatFrag LdFrag> {
let Predicates = [HasBWI] in {
def rr : AVX512Ii8<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.RC:$src1, GR32orGR64:$src2, u8imm:$src3),
OpcodeStr#"\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}",
[(set _.RC:$dst,
(OpNode _.RC:$src1, GR32orGR64:$src2, imm:$src3))]>, EVEX_4V;
defm NAME : avx512_insert_elt_m<opc, OpcodeStr, OpNode, _, LdFrag>;
}
}
multiclass avx512_insert_elt_dq<bits<8> opc, string OpcodeStr,
X86VectorVTInfo _, RegisterClass GRC> {
let Predicates = [HasDQI] in {
def rr : AVX512Ii8<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.RC:$src1, GRC:$src2, u8imm:$src3),
OpcodeStr#"\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}",
[(set _.RC:$dst,
(_.VT (insertelt _.RC:$src1, GRC:$src2, imm:$src3)))]>,
EVEX_4V, TAPD;
defm NAME : avx512_insert_elt_m<opc, OpcodeStr, insertelt, _,
_.ScalarLdFrag>, TAPD;
}
}
defm VPINSRBZ : avx512_insert_elt_bw<0x20, "vpinsrb", X86pinsrb, v16i8x_info,
extloadi8>, TAPD;
defm VPINSRWZ : avx512_insert_elt_bw<0xC4, "vpinsrw", X86pinsrw, v8i16x_info,
extloadi16>, PD;
defm VPINSRDZ : avx512_insert_elt_dq<0x22, "vpinsrd", v4i32x_info, GR32>;
defm VPINSRQZ : avx512_insert_elt_dq<0x22, "vpinsrq", v2i64x_info, GR64>, VEX_W;
AVX512: Implemented encoding and intrinsics for vshufps/d. Added tests for intrinsics and encoding. Differential Revision: http://reviews.llvm.org/D11709 llvm-svn: 246640
2015-09-02 18:50:58 +08:00
//===----------------------------------------------------------------------===//
// VSHUFPS - VSHUFPD Operations
//===----------------------------------------------------------------------===//
multiclass avx512_shufp<string OpcodeStr, AVX512VLVectorVTInfo VTInfo_I,
AVX512VLVectorVTInfo VTInfo_FP>{
defm NAME: avx512_common_3Op_imm8<OpcodeStr, VTInfo_FP, 0xC6, X86Shufp>,
EVEX_CD8<VTInfo_FP.info512.EltSize, CD8VF>,
AVX512AIi8Base, EVEX_4V;
}
defm VSHUFPS: avx512_shufp<"vshufps", avx512vl_i32_info, avx512vl_f32_info>, PS;
defm VSHUFPD: avx512_shufp<"vshufpd", avx512vl_i64_info, avx512vl_f64_info>, PD, VEX_W;
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Byte shift Left/Right
//===----------------------------------------------------------------------===//
multiclass avx512_shift_packed<bits<8> opc, SDNode OpNode, Format MRMr,
Format MRMm, string OpcodeStr, X86VectorVTInfo _>{
def rr : AVX512<opc, MRMr,
(outs _.RC:$dst), (ins _.RC:$src1, u8imm:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set _.RC:$dst,(_.VT (OpNode _.RC:$src1, (i8 imm:$src2))))]>;
let mayLoad = 1 in
def rm : AVX512<opc, MRMm,
(outs _.RC:$dst), (ins _.MemOp:$src1, u8imm:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
[(set _.RC:$dst,(_.VT (OpNode
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
(_.LdFrag addr:$src1), (i8 imm:$src2))))]>;
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_shift_packed_all<bits<8> opc, SDNode OpNode, Format MRMr,
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
Format MRMm, string OpcodeStr, Predicate prd>{
let Predicates = [prd] in
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm Z512 : avx512_shift_packed<opc, OpNode, MRMr, MRMm,
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
OpcodeStr, v8i64_info>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm Z256 : avx512_shift_packed<opc, OpNode, MRMr, MRMm,
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
OpcodeStr, v4i64x_info>, EVEX_V256;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm Z128 : avx512_shift_packed<opc, OpNode, MRMr, MRMm,
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
OpcodeStr, v2i64x_info>, EVEX_V128;
}
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VPSLLDQ : avx512_shift_packed_all<0x73, X86vshldq, MRM7r, MRM7m, "vpslldq",
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
HasBWI>, AVX512PDIi8Base, EVEX_4V;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VPSRLDQ : avx512_shift_packed_all<0x73, X86vshrdq, MRM3r, MRM3m, "vpsrldq",
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
HasBWI>, AVX512PDIi8Base, EVEX_4V;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_psadbw_packed<bits<8> opc, SDNode OpNode,
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
string OpcodeStr, X86VectorVTInfo _dst,
X86VectorVTInfo _src>{
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
def rr : AVX512BI<opc, MRMSrcReg,
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
(outs _dst.RC:$dst), (ins _src.RC:$src1, _src.RC:$src2),
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
[(set _dst.RC:$dst,(_dst.VT
(OpNode (_src.VT _src.RC:$src1),
(_src.VT _src.RC:$src2))))]>;
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
let mayLoad = 1 in
def rm : AVX512BI<opc, MRMSrcMem,
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
(outs _dst.RC:$dst), (ins _src.RC:$src1, _src.MemOp:$src2),
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
[(set _dst.RC:$dst,(_dst.VT
(OpNode (_src.VT _src.RC:$src1),
(_src.VT (bitconvert
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
(_src.LdFrag addr:$src2))))))]>;
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
multiclass avx512_psadbw_packed_all<bits<8> opc, SDNode OpNode,
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
string OpcodeStr, Predicate prd> {
let Predicates = [prd] in
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
defm Z512 : avx512_psadbw_packed<opc, OpNode, OpcodeStr, v8i64_info,
v64i8_info>, EVEX_V512;
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
let Predicates = [prd, HasVLX] in {
[X86] Fix several issues related to X86's psadbw instruction. This patch fixes the following issues: 1. Fix the return type of X86psadbw: it should not be the same type of inputs. For vNi8 inputs the output should be vMi64, where M = N/8. 2. Fix the return type of int_x86_avx512_psad_bw_512 accordingly. 3. Fix the definiton of PSADBW, VPSADBW, and VPSADBWY accordingly. 4. Adjust the return type when building a DAG node of X86ISD::PSADBW type. 5. Update related tests. Differential revision: http://reviews.llvm.org/D14897 llvm-svn: 254010
2015-11-25 03:51:26 +08:00
defm Z256 : avx512_psadbw_packed<opc, OpNode, OpcodeStr, v4i64x_info,
v32i8x_info>, EVEX_V256;
defm Z128 : avx512_psadbw_packed<opc, OpNode, OpcodeStr, v2i64x_info,
v16i8x_info>, EVEX_V128;
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
}
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VPSADBW : avx512_psadbw_packed_all<0xf6, X86psadbw, "vpsadbw",
[X86][AVX512VLBW] add support in byte shift and SAD add byte shift left/right add SAD - compute sum of absolute differences Differential Revision: http://reviews.llvm.org/D12479 llvm-svn: 246654
2015-09-02 22:21:54 +08:00
HasBWI>, EVEX_4V;
AVX512: Implemented encoding and intrinsics for vpternlogd/q. Differential Revision: http://reviews.llvm.org/D13768 llvm-svn: 250396
2015-10-15 20:33:24 +08:00
multiclass avx512_ternlog<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _>{
let Constraints = "$src1 = $dst" in {
defm rri : AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, u8imm:$src4),
AVX512F: Add assembler Intel syntax tests for knl, fix minor bugs. Differential Revision: http://reviews.llvm.org/D17498 llvm-svn: 261521
2016-02-22 20:37:41 +08:00
OpcodeStr, "$src4, $src3, $src2", "$src2, $src3, $src4",
AVX512: Implemented encoding and intrinsics for vpternlogd/q. Differential Revision: http://reviews.llvm.org/D13768 llvm-svn: 250396
2015-10-15 20:33:24 +08:00
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_.VT _.RC:$src3),
(i8 imm:$src4))>, AVX512AIi8Base, EVEX_4V;
let mayLoad = 1 in {
defm rmi : AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.MemOp:$src3, u8imm:$src4),
AVX512F: Add assembler Intel syntax tests for knl, fix minor bugs. Differential Revision: http://reviews.llvm.org/D17498 llvm-svn: 261521
2016-02-22 20:37:41 +08:00
OpcodeStr, "$src4, $src3, $src2", "$src2, $src3, $src4",
AVX512: Implemented encoding and intrinsics for vpternlogd/q. Differential Revision: http://reviews.llvm.org/D13768 llvm-svn: 250396
2015-10-15 20:33:24 +08:00
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_.VT (bitconvert (_.LdFrag addr:$src3))),
(i8 imm:$src4))>,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<_.EltSize, CD8VF>;
defm rmbi : AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.ScalarMemOp:$src3, u8imm:$src4),
OpcodeStr, "$src4, ${src3}"##_.BroadcastStr##", $src2",
"$src2, ${src3}"##_.BroadcastStr##", $src4",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_.VT (X86VBroadcast(_.ScalarLdFrag addr:$src3))),
(i8 imm:$src4))>, EVEX_B,
AVX512AIi8Base, EVEX_4V, EVEX_CD8<_.EltSize, CD8VF>;
}
}// Constraints = "$src1 = $dst"
}
multiclass avx512_common_ternlog<string OpcodeStr, AVX512VLVectorVTInfo _>{
let Predicates = [HasAVX512] in
defm Z : avx512_ternlog<0x25, OpcodeStr, X86vpternlog, _.info512>, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in {
defm Z128 : avx512_ternlog<0x25, OpcodeStr, X86vpternlog, _.info128>, EVEX_V128;
defm Z256 : avx512_ternlog<0x25, OpcodeStr, X86vpternlog, _.info256>, EVEX_V256;
}
}
defm VPTERNLOGD : avx512_common_ternlog<"vpternlogd", avx512vl_i32_info>;
defm VPTERNLOGQ : avx512_common_ternlog<"vpternlogq", avx512vl_i64_info>, VEX_W;
[X86][AVX512]fix dag & add intrinsics for fixupimm cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics Differential Revision: http://reviews.llvm.org/D16313 llvm-svn: 258124
2016-01-19 22:21:39 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - FixupImm
//===----------------------------------------------------------------------===//
multiclass avx512_fixupimm_packed<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _>{
let Constraints = "$src1 = $dst" in {
defm rri : AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, i32u8imm:$src4),
OpcodeStr##_.Suffix, "$src4, $src3, $src2", "$src2, $src3, $src4",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_.IntVT _.RC:$src3),
(i32 imm:$src4),
(i32 FROUND_CURRENT))>;
let mayLoad = 1 in {
defm rmi : AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.MemOp:$src3, i32u8imm:$src4),
AVX512F: Add assembler Intel syntax tests for knl, fix minor bugs. Differential Revision: http://reviews.llvm.org/D17498 llvm-svn: 261521
2016-02-22 20:37:41 +08:00
OpcodeStr##_.Suffix, "$src4, $src3, $src2", "$src2, $src3, $src4",
[X86][AVX512]fix dag & add intrinsics for fixupimm cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics Differential Revision: http://reviews.llvm.org/D16313 llvm-svn: 258124
2016-01-19 22:21:39 +08:00
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_.IntVT (bitconvert (_.LdFrag addr:$src3))),
(i32 imm:$src4),
(i32 FROUND_CURRENT))>;
defm rmbi : AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.ScalarMemOp:$src3, i32u8imm:$src4),
OpcodeStr##_.Suffix, "$src4, ${src3}"##_.BroadcastStr##", $src2",
"$src2, ${src3}"##_.BroadcastStr##", $src4",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_.IntVT (X86VBroadcast(_.ScalarLdFrag addr:$src3))),
(i32 imm:$src4),
(i32 FROUND_CURRENT))>, EVEX_B;
}
} // Constraints = "$src1 = $dst"
}
multiclass avx512_fixupimm_packed_sae<bits<8> opc, string OpcodeStr,
SDNode OpNode, X86VectorVTInfo _>{
let Constraints = "$src1 = $dst" in {
defm rrib : AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, i32u8imm:$src4),
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
OpcodeStr##_.Suffix, "$src4, {sae}, $src3, $src2",
[X86][AVX512]fix dag & add intrinsics for fixupimm cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics Differential Revision: http://reviews.llvm.org/D16313 llvm-svn: 258124
2016-01-19 22:21:39 +08:00
"$src2, $src3, {sae}, $src4",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_.IntVT _.RC:$src3),
(i32 imm:$src4),
(i32 FROUND_NO_EXC))>, EVEX_B;
}
}
multiclass avx512_fixupimm_scalar<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _, X86VectorVTInfo _src3VT> {
let Constraints = "$src1 = $dst" , Predicates = [HasAVX512] in {
defm rri : AVX512_maskable_3src_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, i32u8imm:$src4),
OpcodeStr##_.Suffix, "$src4, $src3, $src2", "$src2, $src3, $src4",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_src3VT.VT _src3VT.RC:$src3),
(i32 imm:$src4),
(i32 FROUND_CURRENT))>;
defm rrib : AVX512_maskable_3src_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, i32u8imm:$src4),
OpcodeStr##_.Suffix, "$src4, {sae}, $src3, $src2",
"$src2, $src3, {sae}, $src4",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_src3VT.VT _src3VT.RC:$src3),
(i32 imm:$src4),
(i32 FROUND_NO_EXC))>, EVEX_B;
let mayLoad = 1 in
defm rmi : AVX512_maskable_3src_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.ScalarMemOp:$src3, i32u8imm:$src4),
OpcodeStr##_.Suffix, "$src4, $src3, $src2", "$src2, $src3, $src4",
(OpNode (_.VT _.RC:$src1),
(_.VT _.RC:$src2),
(_src3VT.VT (scalar_to_vector
(_src3VT.ScalarLdFrag addr:$src3))),
(i32 imm:$src4),
(i32 FROUND_CURRENT))>;
}
}
multiclass avx512_fixupimm_packed_all<AVX512VLVectorVTInfo _Vec>{
let Predicates = [HasAVX512] in
defm Z : avx512_fixupimm_packed<0x54, "vfixupimm", X86VFixupimm, _Vec.info512>,
avx512_fixupimm_packed_sae<0x54, "vfixupimm", X86VFixupimm, _Vec.info512>,
AVX512AIi8Base, EVEX_4V, EVEX_V512;
let Predicates = [HasAVX512, HasVLX] in {
defm Z128 : avx512_fixupimm_packed<0x54, "vfixupimm", X86VFixupimm, _Vec.info128>,
AVX512AIi8Base, EVEX_4V, EVEX_V128;
defm Z256 : avx512_fixupimm_packed<0x54, "vfixupimm", X86VFixupimm, _Vec.info256>,
AVX512AIi8Base, EVEX_4V, EVEX_V256;
}
}
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VFIXUPIMMSS : avx512_fixupimm_scalar<0x55, "vfixupimm", X86VFixupimmScalar,
f32x_info, v4i32x_info>,
[X86][AVX512]fix dag & add intrinsics for fixupimm cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics Differential Revision: http://reviews.llvm.org/D16313 llvm-svn: 258124
2016-01-19 22:21:39 +08:00
AVX512AIi8Base, VEX_LIG, EVEX_4V, EVEX_CD8<32, CD8VT1>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VFIXUPIMMSD : avx512_fixupimm_scalar<0x55, "vfixupimm", X86VFixupimmScalar,
f64x_info, v2i64x_info>,
[X86][AVX512]fix dag & add intrinsics for fixupimm cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics Differential Revision: http://reviews.llvm.org/D16313 llvm-svn: 258124
2016-01-19 22:21:39 +08:00
AVX512AIi8Base, VEX_LIG, EVEX_4V, EVEX_CD8<64, CD8VT1>, VEX_W;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VFIXUPIMMPS : avx512_fixupimm_packed_all<avx512vl_f32_info>,
[X86][AVX512]fix dag & add intrinsics for fixupimm cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics Differential Revision: http://reviews.llvm.org/D16313 llvm-svn: 258124
2016-01-19 22:21:39 +08:00
EVEX_CD8<32, CD8VF>;
[X86][AVX] Add support for 64-bit VZEXT_LOAD of 256/512-bit vectors to EltsFromConsecutiveLoads Follow up to D16217 and D16729 This change uncovered an odd pattern where VZEXT_LOAD v4i64 was being lowered to a load of the lower v2i64 (so the 2nd i64 destination element wasn't being zeroed), I can't find any use/reason for this and have removed the pattern and replaced it so only the 1st i64 element is loaded and the upper bits all zeroed. This matches the description for X86ISD::VZEXT_LOAD Differential Revision: http://reviews.llvm.org/D16768 llvm-svn: 259635
2016-02-03 17:41:59 +08:00
defm VFIXUPIMMPD : avx512_fixupimm_packed_all<avx512vl_f64_info>,
[X86][AVX512]fix dag & add intrinsics for fixupimm cover all width and types (pd/ps/sd/ss) of fixupimm instruction and inrtinsics Differential Revision: http://reviews.llvm.org/D16313 llvm-svn: 258124
2016-01-19 22:21:39 +08:00
EVEX_CD8<64, CD8VF>, VEX_W;