maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/Target/X86/X86InstrAVX512.td

5397 lines
264 KiB
TableGen
Raw Normal View History

[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// Group template arguments that can be derived from the vector type (EltNum x
// EltVT). These are things like the register class for the writemask, etc.
// The idea is to pass one of these as the template argument rather than the
// individual arguments.
AVX-512: Added some comments to ERI scalar intrinsics. No functional change. llvm-svn: 223761
2014-12-09 15:06:32 +08:00
// The template is also used for scalar types, in this case numelts is 1.
[AVX512] Minor fix in lowering pattern for broadcast intrustions. No functional change. llvm-svn: 224122
2014-12-12 22:21:30 +08:00
class X86VectorVTInfo<int numelts, ValueType eltvt, RegisterClass rc,
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
string suffix = ""> {
RegisterClass RC = rc;
[AVX512] Minor fix in lowering pattern for broadcast intrustions. No functional change. llvm-svn: 224122
2014-12-12 22:21:30 +08:00
ValueType EltVT = eltvt;
[AVX512] Two new attributes in X86VectorVTInfo for subvector insert The new attributes are NumElts and the CD8TupleForm. This prepares the code to enable x8 and x2 inserts. NFC, no change in X86.td.expanded except for the new attributes. llvm-svn: 219871
2014-10-16 07:42:09 +08:00
int NumElts = numelts;
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// Corresponding mask register class.
RegisterClass KRC = !cast<RegisterClass>("VK" # NumElts);
// Corresponding write-mask register class.
RegisterClass KRCWM = !cast<RegisterClass>("VK" # NumElts # "WM");
// The GPR register class that can hold the write mask. Use GR8 for fewer
// than 8 elements. Use shift-right and equal to work around the lack of
// !lt in tablegen.
RegisterClass MRC =
!cast<RegisterClass>("GR" #
!if (!eq (!srl(NumElts, 3), 0), 8, NumElts));
// Suffix used in the instruction mnemonic.
string Suffix = suffix;
AVX-512: Added some comments to ERI scalar intrinsics. No functional change. llvm-svn: 223761
2014-12-09 15:06:32 +08:00
// VTName is a string name for vector VT. For vector types it will be
// v # NumElts # EltVT, so for vector of 8 elements of i32 it will be v8i32
// It is a little bit complex for scalar types, where NumElts = 1.
// In this case we build v4f32 or v2f64
string VTName = "v" # !if (!eq (NumElts, 1),
!if (!eq (EltVT.Size, 32), 4,
!if (!eq (EltVT.Size, 64), 2, NumElts)), NumElts) # EltVT;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// The vector VT.
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
ValueType VT = !cast<ValueType>(VTName);
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
string EltTypeName = !cast<string>(EltVT);
// Size of the element type in bits, e.g. 32 for v16i32.
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
string EltSizeName = !subst("i", "", !subst("f", "", EltTypeName));
int EltSize = EltVT.Size;
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// "i" for integer types and "f" for floating-point types
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
string TypeVariantName = !subst(EltSizeName, "", EltTypeName);
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// Size of RC in bits, e.g. 512 for VR512.
int Size = VT.Size;
// The corresponding memory operand, e.g. i512mem for VR512.
X86MemOperand MemOp = !cast<X86MemOperand>(TypeVariantName # Size # "mem");
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
X86MemOperand ScalarMemOp = !cast<X86MemOperand>(EltVT # "mem");
// Load patterns
// Note: For 128/256-bit integer VT we choose loadv2i64/loadv4i64
// due to load promotion during legalization
PatFrag LdFrag = !cast<PatFrag>("load" #
!if (!eq (TypeVariantName, "i"),
!if (!eq (Size, 128), "v2i64",
!if (!eq (Size, 256), "v4i64",
VTName)), VTName));
PatFrag ScalarLdFrag = !cast<PatFrag>("load" # EltVT);
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// The corresponding float type, e.g. v16f32 for v16i32
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
// Note: For EltSize < 32, FloatVT is illegal and TableGen
// fails to compile, so we choose FloatVT = VT
ValueType FloatVT = !cast<ValueType>(
!if (!eq (!srl(EltSize,5),0),
VTName,
!if (!eq(TypeVariantName, "i"),
"v" # NumElts # "f" # EltSize,
VTName)));
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
// The string to specify embedded broadcast in assembly.
string BroadcastStr = "{1to" # NumElts # "}";
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
[AVX512] Two new attributes in X86VectorVTInfo for subvector insert The new attributes are NumElts and the CD8TupleForm. This prepares the code to enable x8 and x2 inserts. NFC, no change in X86.td.expanded except for the new attributes. llvm-svn: 219871
2014-10-16 07:42:09 +08:00
// 8-bit compressed displacement tuple/subvector format. This is only
// defined for NumElts <= 8.
CD8VForm CD8TupleForm = !if (!eq (!srl(NumElts, 4), 0),
!cast<CD8VForm>("CD8VT" # NumElts), ?);
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
SubRegIndex SubRegIdx = !if (!eq (Size, 128), sub_xmm,
!if (!eq (Size, 256), sub_ymm, ?));
Domain ExeDomain = !if (!eq (EltTypeName, "f32"), SSEPackedSingle,
!if (!eq (EltTypeName, "f64"), SSEPackedDouble,
SSEPackedInt));
[AVX512] Move DAG for all-zero node to X86VectorVTInfo No functional change. No change in X86.td.expanded except for the appearance of the new attributes. The new attributes will be used in the subsequent patch. llvm-svn: 219360
2014-10-09 07:25:31 +08:00
// A vector type of the same width with element type i32. This is used to
// create the canonical constant zero node ImmAllZerosV.
ValueType i32VT = !cast<ValueType>("v" # !srl(Size, 5) # "i32");
dag ImmAllZerosV = (VT (bitconvert (i32VT immAllZerosV)));
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
}
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
def v64i8_info : X86VectorVTInfo<64, i8, VR512, "b">;
def v32i16_info : X86VectorVTInfo<32, i16, VR512, "w">;
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
def v16i32_info : X86VectorVTInfo<16, i32, VR512, "d">;
def v8i64_info : X86VectorVTInfo<8, i64, VR512, "q">;
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
def v16f32_info : X86VectorVTInfo<16, f32, VR512, "ps">;
def v8f64_info : X86VectorVTInfo<8, f64, VR512, "pd">;
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
// "x" in v32i8x_info means RC = VR256X
def v32i8x_info : X86VectorVTInfo<32, i8, VR256X, "b">;
def v16i16x_info : X86VectorVTInfo<16, i16, VR256X, "w">;
def v8i32x_info : X86VectorVTInfo<8, i32, VR256X, "d">;
def v4i64x_info : X86VectorVTInfo<4, i64, VR256X, "q">;
[AVX-512] Expanded rsqrt/rcp instructions to VL subset. Refactored multiclass through AVX512_maskable llvm-svn: 220783
2014-10-29 00:37:13 +08:00
def v8f32x_info : X86VectorVTInfo<8, f32, VR256X, "ps">;
def v4f64x_info : X86VectorVTInfo<4, f64, VR256X, "pd">;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
def v16i8x_info : X86VectorVTInfo<16, i8, VR128X, "b">;
def v8i16x_info : X86VectorVTInfo<8, i16, VR128X, "w">;
def v4i32x_info : X86VectorVTInfo<4, i32, VR128X, "d">;
def v2i64x_info : X86VectorVTInfo<2, i64, VR128X, "q">;
[AVX-512] Expanded rsqrt/rcp instructions to VL subset. Refactored multiclass through AVX512_maskable llvm-svn: 220783
2014-10-29 00:37:13 +08:00
def v4f32x_info : X86VectorVTInfo<4, f32, VR128X, "ps">;
def v2f64x_info : X86VectorVTInfo<2, f64, VR128X, "pd">;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
AVX-512: Added some comments to ERI scalar intrinsics. No functional change. llvm-svn: 223761
2014-12-09 15:06:32 +08:00
// We map scalar types to the smallest (128-bit) vector type
// with the appropriate element type. This allows to use the same masking logic.
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
def f32x_info : X86VectorVTInfo<1, f32, VR128X, "ss">;
def f64x_info : X86VectorVTInfo<1, f64, VR128X, "sd">;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
class AVX512VLVectorVTInfo<X86VectorVTInfo i512, X86VectorVTInfo i256,
X86VectorVTInfo i128> {
X86VectorVTInfo info512 = i512;
X86VectorVTInfo info256 = i256;
X86VectorVTInfo info128 = i128;
}
def avx512vl_i8_info : AVX512VLVectorVTInfo<v64i8_info, v32i8x_info,
v16i8x_info>;
def avx512vl_i16_info : AVX512VLVectorVTInfo<v32i16_info, v16i16x_info,
v8i16x_info>;
def avx512vl_i32_info : AVX512VLVectorVTInfo<v16i32_info, v8i32x_info,
v4i32x_info>;
def avx512vl_i64_info : AVX512VLVectorVTInfo<v8i64_info, v4i64x_info,
v2i64x_info>;
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
def avx512vl_f32_info : AVX512VLVectorVTInfo<v16f32_info, v8f32x_info,
v4f32x_info>;
def avx512vl_f64_info : AVX512VLVectorVTInfo<v8f64_info, v4f64x_info,
v2f64x_info>;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
// This multiclass generates the masking variants from the non-masking
// variant. It only provides the assembly pieces for the masking variants.
// It assumes custom ISel patterns for masking which can be provided as
// template arguments.
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
multiclass AVX512_maskable_custom<bits<8> O, Format F,
dag Outs,
dag Ins, dag MaskingIns, dag ZeroMaskingIns,
string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
list<dag> Pattern,
list<dag> MaskingPattern,
list<dag> ZeroMaskingPattern,
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
string Round = "",
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
string MaskingConstraint = "",
InstrItinClass itin = NoItinerary,
bit IsCommutable = 0> {
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
let isCommutable = IsCommutable in
def NAME: AVX512<O, F, Outs, Ins,
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
OpcodeStr#"\t{"#AttSrcAsm#", $dst "#Round#"|"#
"$dst "#Round#", "#IntelSrcAsm#"}",
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
Pattern, itin>;
[AVX512] Generate masking instruction variants with tablegen After adding the masking variants to several instructions, I have decided to experiment with generating these from the non-masking/unconditional variant. This will hopefully reduce the amount repetition that we currently have in order to define an instruction with all its variants (for a reg/mem instruction this would be 6 instruction defs and 2 Pat<> for the intrinsic). The patch is the first cut that is currently only applied to valignd/q to make the patch small. A few notes on the approach: * In order to stitch together the dag for both the conditional and the unconditional patterns I pass the RHS of the set rather than the full pattern (set dest, RHS). * Rather than subclassing each instruction base class (e.g. AVX512AIi8), with a masking variant which wouldn't scale, I derived the masking instructions from a new base class AVX512 (this is just I<> with Requires<HasAVX512>). The instructions derive from this now, plus a new set of classes that add the format bits and everything else that instruction base class provided (i.e. AVX512AIi8 vs. AVX512AIi8Base). I hope we can go incrementally from here. I expect that: * We will need different variants of the masking class. One example is instructions requiring three vector sources. In this case we tie one of the source operands to dest rather than a new implicit source operand ($src0) * Add the zero-masking variant * Add more AVX512*Base classes as new uses are added I've looked at X86.td.expanded before and after to make sure that nothing got lost for valignd/q. llvm-svn: 215125
2014-08-08 01:53:55 +08:00
[AVX512] Add codegen test for the masking variant of valign The AddedComplexity is needed just like in avx512_perm_3src. There may be a bug in the complexity computation... llvm-svn: 215168
2014-08-08 07:18:18 +08:00
// Prefer over VMOV*rrk Pat<>
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
let AddedComplexity = 20 in
def NAME#k: AVX512<O, F, Outs, MaskingIns,
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
OpcodeStr#"\t{"#AttSrcAsm#", $dst {${mask}}"#Round#"|"#
"$dst {${mask}}"#Round#", "#IntelSrcAsm#"}",
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
MaskingPattern, itin>,
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
EVEX_K {
// In case of the 3src subclass this is overridden with a let.
string Constraints = MaskingConstraint;
}
[AVX512] Add zero-masking variant to AVX512_masking multiclass This completes one item from the todo-list of r215125 "Generate masking instruction variants with tablegen". The AddedComplexity is needed just like for the k variant. Added a codegen test based on valignq. llvm-svn: 215173
2014-08-08 07:53:38 +08:00
let AddedComplexity = 30 in // Prefer over VMOV*rrkz Pat<>
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
def NAME#kz: AVX512<O, F, Outs, ZeroMaskingIns,
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
OpcodeStr#"\t{"#AttSrcAsm#", $dst {${mask}} {z}"#Round#"|"#
"$dst {${mask}} {z}"#Round#", "#IntelSrcAsm#"}",
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
ZeroMaskingPattern,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
itin>,
[AVX512] Add zero-masking variant to AVX512_masking multiclass This completes one item from the todo-list of r215125 "Generate masking instruction variants with tablegen". The AddedComplexity is needed just like for the k variant. Added a codegen test based on valignq. llvm-svn: 215173
2014-08-08 07:53:38 +08:00
EVEX_KZ;
[AVX512] Generate masking instruction variants with tablegen After adding the masking variants to several instructions, I have decided to experiment with generating these from the non-masking/unconditional variant. This will hopefully reduce the amount repetition that we currently have in order to define an instruction with all its variants (for a reg/mem instruction this would be 6 instruction defs and 2 Pat<> for the intrinsic). The patch is the first cut that is currently only applied to valignd/q to make the patch small. A few notes on the approach: * In order to stitch together the dag for both the conditional and the unconditional patterns I pass the RHS of the set rather than the full pattern (set dest, RHS). * Rather than subclassing each instruction base class (e.g. AVX512AIi8), with a masking variant which wouldn't scale, I derived the masking instructions from a new base class AVX512 (this is just I<> with Requires<HasAVX512>). The instructions derive from this now, plus a new set of classes that add the format bits and everything else that instruction base class provided (i.e. AVX512AIi8 vs. AVX512AIi8Base). I hope we can go incrementally from here. I expect that: * We will need different variants of the masking class. One example is instructions requiring three vector sources. In this case we tie one of the source operands to dest rather than a new implicit source operand ($src0) * Add the zero-masking variant * Add more AVX512*Base classes as new uses are added I've looked at X86.td.expanded before and after to make sure that nothing got lost for valignd/q. llvm-svn: 215125
2014-08-08 01:53:55 +08:00
}
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
// Common base class of AVX512_maskable and AVX512_maskable_3src.
multiclass AVX512_maskable_common<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs,
dag Ins, dag MaskingIns, dag ZeroMaskingIns,
string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
dag RHS, dag MaskingRHS,
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
SDNode Select = vselect, string Round = "",
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
string MaskingConstraint = "",
InstrItinClass itin = NoItinerary,
bit IsCommutable = 0> :
AVX512_maskable_custom<O, F, Outs, Ins, MaskingIns, ZeroMaskingIns, OpcodeStr,
AttSrcAsm, IntelSrcAsm,
[(set _.RC:$dst, RHS)],
[(set _.RC:$dst, MaskingRHS)],
[(set _.RC:$dst,
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
(Select _.KRCWM:$mask, RHS, _.ImmAllZerosV))],
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
Round, MaskingConstraint, NoItinerary, IsCommutable>;
[AVX512] Peel off an asm-only class from AVX512_masking_common. No functional change. This enables the generation of masking instructions that don't provide a ISel pattern. llvm-svn: 219358
2014-10-09 07:25:23 +08:00
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
// This multiclass generates the unconditional/non-masking, the masking and
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
// the zero-masking variant of the vector instruction. In the masking case, the
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
// perserved vector elements come from a new dummy input operand tied to $dst.
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
multiclass AVX512_maskable<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag Ins, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
dag RHS, string Round = "",
InstrItinClass itin = NoItinerary,
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
bit IsCommutable = 0> :
AVX512_maskable_common<O, F, _, Outs, Ins,
!con((ins _.RC:$src0, _.KRCWM:$mask), Ins),
!con((ins _.KRCWM:$mask), Ins),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
(vselect _.KRCWM:$mask, RHS, _.RC:$src0), vselect,
Round, "$src0 = $dst", itin, IsCommutable>;
// This multiclass generates the unconditional/non-masking, the masking and
// the zero-masking variant of the scalar instruction.
multiclass AVX512_maskable_scalar<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag Ins, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
dag RHS, string Round = "",
InstrItinClass itin = NoItinerary,
bit IsCommutable = 0> :
AVX512_maskable_common<O, F, _, Outs, Ins,
!con((ins _.RC:$src0, _.KRCWM:$mask), Ins),
!con((ins _.KRCWM:$mask), Ins),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
(X86select _.KRCWM:$mask, RHS, _.RC:$src0), X86select,
Round, "$src0 = $dst", itin, IsCommutable>;
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
// Similar to AVX512_maskable but in this case one of the source operands
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
// ($src1) is already tied to $dst so we just use that for the preserved
// vector elements. NOTE that the NonTiedIns (the ins dag) should exclude
// $src1.
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
multiclass AVX512_maskable_3src<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag NonTiedIns, string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
dag RHS> :
AVX512_maskable_common<O, F, _, Outs,
!con((ins _.RC:$src1), NonTiedIns),
!con((ins _.RC:$src1, _.KRCWM:$mask), NonTiedIns),
!con((ins _.RC:$src1, _.KRCWM:$mask), NonTiedIns),
OpcodeStr, AttSrcAsm, IntelSrcAsm, RHS,
(vselect _.KRCWM:$mask, RHS, _.RC:$src1)>;
multiclass AVX512_maskable_in_asm<bits<8> O, Format F, X86VectorVTInfo _,
dag Outs, dag Ins,
string OpcodeStr,
string AttSrcAsm, string IntelSrcAsm,
list<dag> Pattern> :
AVX512_maskable_custom<O, F, Outs, Ins,
!con((ins _.RC:$src0, _.KRCWM:$mask), Ins),
!con((ins _.KRCWM:$mask), Ins),
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
OpcodeStr, AttSrcAsm, IntelSrcAsm, Pattern, [], [], "",
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
"$src0 = $dst">;
[AVX512] Add asm-only support for vextract*x4 masking variants These derive from the new asm-only masking definitions. Unfortunately I wasn't able to find a ISel pattern that we could legally generate for the masking variants. The problem is that since the destination is v4* we would need VK4 register classes and v4i1 value types to express the masking. These are however not legal types/classes in AVX512f but only in VL, so things get complicated pretty quickly. We can revisit this question later if we have a more pressing need to express something like this. So the ISel patterns are empty for the masking instructions and the next patch will add Pat<>s instead to match the intrinsics calls with instructions. llvm-svn: 219361
2014-10-09 07:25:33 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Bitcasts between 512-bit vector types. Return the original type since
// no instruction is needed for the conversion
let Predicates = [HasAVX512] in {
def : Pat<(v8f64 (bitconvert (v8i64 VR512:$src))), (v8f64 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v8f64 (bitconvert (v16i32 VR512:$src))), (v8f64 VR512:$src)>;
def : Pat<(v8f64 (bitconvert (v32i16 VR512:$src))), (v8f64 VR512:$src)>;
def : Pat<(v8f64 (bitconvert (v64i8 VR512:$src))), (v8f64 VR512:$src)>;
def : Pat<(v8f64 (bitconvert (v16f32 VR512:$src))), (v8f64 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (bitconvert (v8i64 VR512:$src))), (v16f32 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v16f32 (bitconvert (v16i32 VR512:$src))), (v16f32 VR512:$src)>;
def : Pat<(v16f32 (bitconvert (v32i16 VR512:$src))), (v16f32 VR512:$src)>;
def : Pat<(v16f32 (bitconvert (v64i8 VR512:$src))), (v16f32 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (bitconvert (v8f64 VR512:$src))), (v16f32 VR512:$src)>;
def : Pat<(v8i64 (bitconvert (v16i32 VR512:$src))), (v8i64 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v8i64 (bitconvert (v32i16 VR512:$src))), (v8i64 VR512:$src)>;
def : Pat<(v8i64 (bitconvert (v64i8 VR512:$src))), (v8i64 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i64 (bitconvert (v8f64 VR512:$src))), (v8i64 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v8i64 (bitconvert (v16f32 VR512:$src))), (v8i64 VR512:$src)>;
def : Pat<(v16i32 (bitconvert (v8i64 VR512:$src))), (v16i32 VR512:$src)>;
AVX-512: added a missing bitcast from v16f32 to v16i32 llvm-svn: 215351
2014-08-11 17:59:08 +08:00
def : Pat<(v16i32 (bitconvert (v16f32 VR512:$src))), (v16i32 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v16i32 (bitconvert (v32i16 VR512:$src))), (v16i32 VR512:$src)>;
def : Pat<(v16i32 (bitconvert (v64i8 VR512:$src))), (v16i32 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16i32 (bitconvert (v8f64 VR512:$src))), (v16i32 VR512:$src)>;
[SKX] Enabling SKX target and AVX512BW, AVX512DQ, AVX512VL features. Enabling HasAVX512{DQ,BW,VL} predicates. Adding VK2, VK4, VK32, VK64 masked register classes. Adding new types (v64i8, v32i16) to VR512. Extending calling conventions for new types (v64i8, v32i16) Patch by Zinovy Nis <zinovy.y.nis@intel.com> Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213545
2014-07-21 22:54:21 +08:00
def : Pat<(v32i16 (bitconvert (v8i64 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v16i32 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v64i8 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v8f64 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v16f32 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v32i16 (bitconvert (v16f32 VR512:$src))), (v32i16 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v8i64 VR512:$src))), (v64i8 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v16i32 VR512:$src))), (v64i8 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v32i16 VR512:$src))), (v64i8 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v8f64 VR512:$src))), (v64i8 VR512:$src)>;
def : Pat<(v64i8 (bitconvert (v16f32 VR512:$src))), (v64i8 VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v2i64 (bitconvert (v4i32 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v2i64 (bitconvert (v8i16 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v2i64 (bitconvert (v16i8 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v2i64 (bitconvert (v2f64 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v2i64 (bitconvert (v4f32 VR128X:$src))), (v2i64 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v2i64 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v8i16 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v16i8 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v2f64 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v4i32 (bitconvert (v4f32 VR128X:$src))), (v4i32 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v2i64 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v4i32 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v16i8 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v2f64 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v8i16 (bitconvert (v4f32 VR128X:$src))), (v8i16 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v2i64 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v4i32 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v8i16 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v2f64 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v16i8 (bitconvert (v4f32 VR128X:$src))), (v16i8 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v2i64 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v4i32 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v8i16 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v16i8 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v4f32 (bitconvert (v2f64 VR128X:$src))), (v4f32 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v2i64 VR128X:$src))), (v2f64 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v4i32 VR128X:$src))), (v2f64 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v8i16 VR128X:$src))), (v2f64 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v16i8 VR128X:$src))), (v2f64 VR128X:$src)>;
def : Pat<(v2f64 (bitconvert (v4f32 VR128X:$src))), (v2f64 VR128X:$src)>;
// Bitcasts between 256-bit vector types. Return the original type since
// no instruction is needed for the conversion
def : Pat<(v4f64 (bitconvert (v8f32 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v4f64 (bitconvert (v8i32 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v4f64 (bitconvert (v4i64 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v4f64 (bitconvert (v16i16 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v4f64 (bitconvert (v32i8 VR256X:$src))), (v4f64 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v8i32 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v4i64 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v4f64 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v32i8 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v8f32 (bitconvert (v16i16 VR256X:$src))), (v8f32 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v8f32 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v8i32 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v4f64 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v32i8 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v4i64 (bitconvert (v16i16 VR256X:$src))), (v4i64 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v4f64 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v4i64 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v8f32 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v8i32 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v32i8 (bitconvert (v16i16 VR256X:$src))), (v32i8 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v32i8 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v16i16 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v8f32 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v4i64 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v8i32 (bitconvert (v4f64 VR256X:$src))), (v8i32 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v8f32 VR256X:$src))), (v16i16 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v8i32 VR256X:$src))), (v16i16 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v4i64 VR256X:$src))), (v16i16 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v4f64 VR256X:$src))), (v16i16 VR256X:$src)>;
def : Pat<(v16i16 (bitconvert (v32i8 VR256X:$src))), (v16i16 VR256X:$src)>;
}
//
// AVX-512: VPXOR instruction writes zero to its upper part, it's safe build zeros.
//
let isReMaterializable = 1, isAsCheapAsAMove = 1, canFoldAsLoad = 1,
isPseudo = 1, Predicates = [HasAVX512] in {
def AVX512_512_SET0 : I<0, Pseudo, (outs VR512:$dst), (ins), "",
[(set VR512:$dst, (v16f32 immAllZerosV))]>;
}
Remove duplicate pattern and add predicate checks on other patterns. llvm-svn: 200455
2014-01-30 14:03:19 +08:00
let Predicates = [HasAVX512] in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i64 immAllZerosV), (AVX512_512_SET0)>;
def : Pat<(v16i32 immAllZerosV), (AVX512_512_SET0)>;
def : Pat<(v8f64 immAllZerosV), (AVX512_512_SET0)>;
Remove duplicate pattern and add predicate checks on other patterns. llvm-svn: 200455
2014-01-30 14:03:19 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - VECTOR INSERT
//
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
multiclass vinsert_for_size_no_alt<int Opcode,
X86VectorVTInfo From, X86VectorVTInfo To,
PatFrag vinsert_insert,
SDNodeXForm INSERT_get_vinsert_imm> {
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
let hasSideEffects = 0, ExeDomain = To.ExeDomain in {
def rr : AVX512AIi8<Opcode, MRMSrcReg, (outs VR512:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins VR512:$src1, From.RC:$src2, u8imm:$src3),
[AVX512] Two new attributes in X86VectorVTInfo for subvector insert The new attributes are NumElts and the CD8TupleForm. This prepares the code to enable x8 and x2 inserts. NFC, no change in X86.td.expanded except for the new attributes. llvm-svn: 219871
2014-10-16 07:42:09 +08:00
"vinsert" # From.EltTypeName # "x" # From.NumElts #
"\t{$src3, $src2, $src1, $dst|"
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
"$dst, $src1, $src2, $src3}",
[AVX512] Pull pattern for subvector insert into the instruction definition No functional change intended. Very similar to the change I made for subvector extract in r218480. test/CodeGen/X86/avx512-insert-extract.ll covers this. llvm-svn: 218928
2014-10-03 07:18:30 +08:00
[(set To.RC:$dst, (vinsert_insert:$src3 (To.VT VR512:$src1),
(From.VT From.RC:$src2),
(iPTR imm)))]>,
EVEX_4V, EVEX_V512;
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
let mayLoad = 1 in
def rm : AVX512AIi8<Opcode, MRMSrcMem, (outs VR512:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins VR512:$src1, From.MemOp:$src2, u8imm:$src3),
[AVX512] Two new attributes in X86VectorVTInfo for subvector insert The new attributes are NumElts and the CD8TupleForm. This prepares the code to enable x8 and x2 inserts. NFC, no change in X86.td.expanded except for the new attributes. llvm-svn: 219871
2014-10-16 07:42:09 +08:00
"vinsert" # From.EltTypeName # "x" # From.NumElts #
"\t{$src3, $src2, $src1, $dst|"
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
"$dst, $src1, $src2, $src3}",
[AVX512] Two new attributes in X86VectorVTInfo for subvector insert The new attributes are NumElts and the CD8TupleForm. This prepares the code to enable x8 and x2 inserts. NFC, no change in X86.td.expanded except for the new attributes. llvm-svn: 219871
2014-10-16 07:42:09 +08:00
[]>,
EVEX_4V, EVEX_V512, EVEX_CD8<From.EltSize, From.CD8TupleForm>;
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
}
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
}
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
multiclass vinsert_for_size<int Opcode,
X86VectorVTInfo From, X86VectorVTInfo To,
X86VectorVTInfo AltFrom, X86VectorVTInfo AltTo,
PatFrag vinsert_insert,
SDNodeXForm INSERT_get_vinsert_imm> :
vinsert_for_size_no_alt<Opcode, From, To,
vinsert_insert, INSERT_get_vinsert_imm> {
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
// Codegen pattern with the alternative types, e.g. v2i64 -> v8i64 for
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
// vinserti32x4. Only add this if 64x2 and friends are not supported
// natively via AVX512DQ.
let Predicates = [NoDQI] in
def : Pat<(vinsert_insert:$ins
(AltTo.VT VR512:$src1), (AltFrom.VT From.RC:$src2), (iPTR imm)),
(AltTo.VT (!cast<Instruction>(NAME # From.EltSize # "x4rr")
VR512:$src1, From.RC:$src2,
(INSERT_get_vinsert_imm VR512:$ins)))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Rename arg from Opcode32/64 to Opcode128/256 in vinsert_for_size It's the W bit that selects between 32 or 64 elt type and not the opcode. The opcode selects between the width of the insert (128 or 256). llvm-svn: 219870
2014-10-16 07:42:04 +08:00
multiclass vinsert_for_type<ValueType EltVT32, int Opcode128,
ValueType EltVT64, int Opcode256> {
defm NAME # "32x4" : vinsert_for_size<Opcode128,
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
X86VectorVTInfo< 4, EltVT32, VR128X>,
X86VectorVTInfo<16, EltVT32, VR512>,
X86VectorVTInfo< 2, EltVT64, VR128X>,
X86VectorVTInfo< 8, EltVT64, VR512>,
vinsert128_insert,
INSERT_get_vinsert128_imm>;
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
let Predicates = [HasDQI] in
defm NAME # "64x2" : vinsert_for_size_no_alt<Opcode128,
X86VectorVTInfo< 2, EltVT64, VR128X>,
X86VectorVTInfo< 8, EltVT64, VR512>,
vinsert128_insert,
INSERT_get_vinsert128_imm>, VEX_W;
[AVX512] Rename arg from Opcode32/64 to Opcode128/256 in vinsert_for_size It's the W bit that selects between 32 or 64 elt type and not the opcode. The opcode selects between the width of the insert (128 or 256). llvm-svn: 219870
2014-10-16 07:42:04 +08:00
defm NAME # "64x4" : vinsert_for_size<Opcode256,
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
X86VectorVTInfo< 4, EltVT64, VR256X>,
X86VectorVTInfo< 8, EltVT64, VR512>,
X86VectorVTInfo< 8, EltVT32, VR256>,
X86VectorVTInfo<16, EltVT32, VR512>,
vinsert256_insert,
INSERT_get_vinsert256_imm>, VEX_W;
[AVX512] Add DQ subvector inserts In AVX512f we support 64x2 and 32x8 inserts via matching them to 32x4 and 64x4 respectively. These are matched by "Alt" Pat<>'s (Alt stands for alternative VTs). Since DQ has native support for these intructions, I peeled off the non-"Alt" part of the baseclass into vinsert_for_size_no_alt. The DQ instructions are derived from this multiclass. The "Alt" Pat<>'s are disabled with DQ. Fixes <rdar://problem/18426089> llvm-svn: 219874
2014-10-16 07:42:17 +08:00
let Predicates = [HasDQI] in
defm NAME # "32x8" : vinsert_for_size_no_alt<Opcode256,
X86VectorVTInfo< 8, EltVT32, VR256X>,
X86VectorVTInfo<16, EltVT32, VR512>,
vinsert256_insert,
INSERT_get_vinsert256_imm>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Refactor subvector inserts No functional change. Very similar to the extract refactoring I did in r218478. Compared X86.td.expanded before and after. llvm-svn: 218927
2014-10-03 07:18:28 +08:00
defm VINSERTF : vinsert_for_type<f32, 0x18, f64, 0x1a>;
defm VINSERTI : vinsert_for_type<i32, 0x38, i64, 0x3a>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// vinsertps - insert f32 to XMM
def VINSERTPSzrr : AVX512AIi8<0x21, MRMSrcReg, (outs VR128X:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins VR128X:$src1, VR128X:$src2, u8imm:$src3),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vinsertps\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}",
Rename X86insrtps to the proper instruction name. Summary: The INSERTPS pattern fragment was called insrtps (mising 'e'), which would make it harder to grep for the patterns related to this instruction. Renaming it to use the proper instruction name. Reviewers: nadav CC: llvm-commits Differential Revision: http://reviews.llvm.org/D3443 llvm-svn: 206779
2014-04-22 04:07:29 +08:00
[(set VR128X:$dst, (X86insertps VR128X:$src1, VR128X:$src2, imm:$src3))]>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_4V;
def VINSERTPSzrm: AVX512AIi8<0x21, MRMSrcMem, (outs VR128X:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins VR128X:$src1, f32mem:$src2, u8imm:$src3),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vinsertps\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}",
Rename X86insrtps to the proper instruction name. Summary: The INSERTPS pattern fragment was called insrtps (mising 'e'), which would make it harder to grep for the patterns related to this instruction. Renaming it to use the proper instruction name. Reviewers: nadav CC: llvm-commits Differential Revision: http://reviews.llvm.org/D3443 llvm-svn: 206779
2014-04-22 04:07:29 +08:00
[(set VR128X:$dst, (X86insertps VR128X:$src1,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v4f32 (scalar_to_vector (loadf32 addr:$src2))),
imm:$src3))]>, EVEX_4V, EVEX_CD8<32, CD8VT1>;
//===----------------------------------------------------------------------===//
// AVX-512 VECTOR EXTRACT
//---
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
multiclass vextract_for_size<int Opcode,
X86VectorVTInfo From, X86VectorVTInfo To,
X86VectorVTInfo AltFrom, X86VectorVTInfo AltTo,
PatFrag vextract_extract,
SDNodeXForm EXTRACT_get_vextract_imm> {
let hasSideEffects = 0, ExeDomain = To.ExeDomain in {
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rr : AVX512_maskable_in_asm<Opcode, MRMDestReg, To, (outs To.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins VR512:$src1, u8imm:$idx),
[AVX512] Add asm-only support for vextract*x4 masking variants These derive from the new asm-only masking definitions. Unfortunately I wasn't able to find a ISel pattern that we could legally generate for the masking variants. The problem is that since the destination is v4* we would need VK4 register classes and v4i1 value types to express the masking. These are however not legal types/classes in AVX512f but only in VL, so things get complicated pretty quickly. We can revisit this question later if we have a more pressing need to express something like this. So the ISel patterns are empty for the masking instructions and the next patch will add Pat<>s instead to match the intrinsics calls with instructions. llvm-svn: 219361
2014-10-09 07:25:33 +08:00
"vextract" # To.EltTypeName # "x4",
"$idx, $src1", "$src1, $idx",
[(set To.RC:$dst, (vextract_extract:$idx (From.VT VR512:$src1),
(iPTR imm)))]>,
AVX512AIi8Base, EVEX, EVEX_V512;
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
let mayStore = 1 in
def rm : AVX512AIi8<Opcode, MRMDestMem, (outs),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins To.MemOp:$dst, VR512:$src1, u8imm:$src2),
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
"vextract" # To.EltTypeName # "x4\t{$src2, $src1, $dst|"
"$dst, $src1, $src2}",
[]>, EVEX, EVEX_V512, EVEX_CD8<To.EltSize, CD8VT4>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
// Codegen pattern with the alternative types, e.g. v8i64 -> v2i64 for
// vextracti32x4
def : Pat<(vextract_extract:$ext (AltFrom.VT VR512:$src1), (iPTR imm)),
(AltTo.VT (!cast<Instruction>(NAME # To.EltSize # "x4rr")
VR512:$src1,
(EXTRACT_get_vextract_imm To.RC:$ext)))>;
// A 128/256-bit subvector extract from the first 512-bit vector position is
// a subregister copy that needs no instruction.
def : Pat<(To.VT (extract_subvector (From.VT VR512:$src), (iPTR 0))),
(To.VT
(EXTRACT_SUBREG (From.VT VR512:$src), To.SubRegIdx))>;
// And for the alternative types.
def : Pat<(AltTo.VT (extract_subvector (AltFrom.VT VR512:$src), (iPTR 0))),
(AltTo.VT
(EXTRACT_SUBREG (AltFrom.VT VR512:$src), AltTo.SubRegIdx))>;
[AVX512] Intrinsics for vextract*x4 This adds the Pat<>'s for the intrinsics. These are necessary because we don't lower these intrinsics to SDNodes but match them directly. See the rational in the previous commit. llvm-svn: 219362
2014-10-09 07:25:37 +08:00
// Intrinsic call with masking.
def : Pat<(!cast<Intrinsic>("int_x86_avx512_mask_vextract" # To.EltTypeName #
"x4_512")
VR512:$src1, (iPTR imm:$idx), To.RC:$src0, GR8:$mask),
(!cast<Instruction>(NAME # To.EltSize # "x4rrk") To.RC:$src0,
(v4i1 (COPY_TO_REGCLASS GR8:$mask, VK4WM)),
VR512:$src1, imm:$idx)>;
// Intrinsic call with zero-masking.
def : Pat<(!cast<Intrinsic>("int_x86_avx512_mask_vextract" # To.EltTypeName #
"x4_512")
VR512:$src1, (iPTR imm:$idx), To.ImmAllZerosV, GR8:$mask),
(!cast<Instruction>(NAME # To.EltSize # "x4rrkz")
(v4i1 (COPY_TO_REGCLASS GR8:$mask, VK4WM)),
VR512:$src1, imm:$idx)>;
// Intrinsic call without masking.
def : Pat<(!cast<Intrinsic>("int_x86_avx512_mask_vextract" # To.EltTypeName #
"x4_512")
VR512:$src1, (iPTR imm:$idx), To.ImmAllZerosV, (i8 -1)),
(!cast<Instruction>(NAME # To.EltSize # "x4rr")
VR512:$src1, imm:$idx)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
multiclass vextract_for_type<ValueType EltVT32, int Opcode32,
ValueType EltVT64, int Opcode64> {
defm NAME # "32x4" : vextract_for_size<Opcode32,
X86VectorVTInfo<16, EltVT32, VR512>,
X86VectorVTInfo< 4, EltVT32, VR128X>,
X86VectorVTInfo< 8, EltVT64, VR512>,
X86VectorVTInfo< 2, EltVT64, VR128X>,
vextract128_extract,
EXTRACT_get_vextract128_imm>;
defm NAME # "64x4" : vextract_for_size<Opcode64,
X86VectorVTInfo< 8, EltVT64, VR512>,
X86VectorVTInfo< 4, EltVT64, VR256X>,
X86VectorVTInfo<16, EltVT32, VR512>,
X86VectorVTInfo< 8, EltVT32, VR256>,
vextract256_extract,
EXTRACT_get_vextract256_imm>, VEX_W;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Refactor subvector extracts No functional change. These are now implemented as two levels of multiclasses heavily relying on the new X86VectorVTInfo class. The multiclass at the first level that is called with float or int provides the 128 or 256 bit subvector extracts. The second level provides the register and memory variants and some more Pat<>s. I've compared the td.expanded files before and after. One change is that ExeDomain for 64x4 is SSEPackedDouble now. I think this is correct, i.e. a bugfix. (BTW, this is the change that was blocked on the recent tablegen fix. The class-instance values X86VectorVTInfo inside vextract_for_type weren't properly evaluated.) Part of <rdar://problem/17688758> llvm-svn: 218478
2014-09-26 07:48:45 +08:00
defm VEXTRACTF : vextract_for_type<f32, 0x19, f64, 0x1b>;
defm VEXTRACTI : vextract_for_type<i32, 0x39, i64, 0x3b>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// A 128-bit subvector insert to the first 512-bit vector position
// is a subregister copy that needs no instruction.
def : Pat<(insert_subvector undef, (v2i64 VR128X:$src), (iPTR 0)),
(INSERT_SUBREG (v8i64 (IMPLICIT_DEF)),
(INSERT_SUBREG (v4i64 (IMPLICIT_DEF)), VR128X:$src, sub_xmm),
sub_ymm)>;
def : Pat<(insert_subvector undef, (v2f64 VR128X:$src), (iPTR 0)),
(INSERT_SUBREG (v8f64 (IMPLICIT_DEF)),
(INSERT_SUBREG (v4f64 (IMPLICIT_DEF)), VR128X:$src, sub_xmm),
sub_ymm)>;
def : Pat<(insert_subvector undef, (v4i32 VR128X:$src), (iPTR 0)),
(INSERT_SUBREG (v16i32 (IMPLICIT_DEF)),
(INSERT_SUBREG (v8i32 (IMPLICIT_DEF)), VR128X:$src, sub_xmm),
sub_ymm)>;
def : Pat<(insert_subvector undef, (v4f32 VR128X:$src), (iPTR 0)),
(INSERT_SUBREG (v16f32 (IMPLICIT_DEF)),
(INSERT_SUBREG (v8f32 (IMPLICIT_DEF)), VR128X:$src, sub_xmm),
sub_ymm)>;
def : Pat<(insert_subvector undef, (v4i64 VR256X:$src), (iPTR 0)),
(INSERT_SUBREG (v8i64 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
def : Pat<(insert_subvector undef, (v4f64 VR256X:$src), (iPTR 0)),
(INSERT_SUBREG (v8f64 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
def : Pat<(insert_subvector undef, (v8i32 VR256X:$src), (iPTR 0)),
(INSERT_SUBREG (v16i32 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
def : Pat<(insert_subvector undef, (v8f32 VR256X:$src), (iPTR 0)),
(INSERT_SUBREG (v16f32 (IMPLICIT_DEF)), VR256X:$src, sub_ymm)>;
// vextractps - extract 32 bits from XMM
def VEXTRACTPSzrr : AVX512AIi8<0x17, MRMDestReg, (outs GR32:$dst),
[X86] Use u8imm in several places that used i32i8imm that don't require an i32 type. llvm-svn: 227031
2015-01-25 10:21:13 +08:00
(ins VR128X:$src1, u8imm:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vextractps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set GR32:$dst, (extractelt (bc_v4i32 (v4f32 VR128X:$src1)), imm:$src2))]>,
EVEX;
def VEXTRACTPSzmr : AVX512AIi8<0x17, MRMDestMem, (outs),
[X86] Use u8imm in several places that used i32i8imm that don't require an i32 type. llvm-svn: 227031
2015-01-25 10:21:13 +08:00
(ins f32mem:$dst, VR128X:$src1, u8imm:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vextractps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(store (extractelt (bc_v4i32 (v4f32 VR128X:$src1)), imm:$src2),
AVX-512: Optimized BUILD_VECTOR pattern; fixed encoding of VEXTRACTPS instruction. llvm-svn: 201134
2014-02-11 15:25:59 +08:00
addr:$dst)]>, EVEX, EVEX_CD8<32, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===---------------------------------------------------------------------===//
// AVX-512 BROADCAST
//---
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
multiclass avx512_fp_broadcast<bits<8> opc, SDNode OpNode, RegisterClass SrcRC,
ValueType svt, X86VectorVTInfo _> {
defm r : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins SrcRC:$src), "vbroadcast"## !subst("p", "s", _.Suffix),
"$src", "$src", (_.VT (OpNode (svt SrcRC:$src)))>,
T8PD, EVEX;
let mayLoad = 1 in {
defm m : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.ScalarMemOp:$src),
"vbroadcast"##!subst("p", "s", _.Suffix), "$src", "$src",
(_.VT (OpNode (_.ScalarLdFrag addr:$src)))>,
T8PD, EVEX;
}
}
multiclass avx512_fp_broadcast_vl<bits<8> opc, SDNode OpNode,
AVX512VLVectorVTInfo _> {
defm Z : avx512_fp_broadcast<opc, OpNode, VR128X, _.info128.VT, _.info512>,
EVEX_V512;
let Predicates = [HasVLX] in {
defm Z256 : avx512_fp_broadcast<opc, OpNode, VR128X, _.info128.VT, _.info256>,
EVEX_V256;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let ExeDomain = SSEPackedSingle in {
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
defm VBROADCASTSS : avx512_fp_broadcast_vl<0x18, X86VBroadcast,
avx512vl_f32_info>, EVEX_CD8<32, CD8VT1>;
let Predicates = [HasVLX] in {
defm VBROADCASTSSZ128 : avx512_fp_broadcast<0x18, X86VBroadcast, VR128X,
v4f32, v4f32x_info>, EVEX_V128,
EVEX_CD8<32, CD8VT1>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
let ExeDomain = SSEPackedDouble in {
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
defm VBROADCASTSD : avx512_fp_broadcast_vl<0x19, X86VBroadcast,
avx512vl_f64_info>, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Add a comment for avx512_broadcast_pat multiclass llvm-svn: 224341
2014-12-17 00:12:11 +08:00
// avx512_broadcast_pat introduces patterns for broadcast with a scalar argument.
// Later, we can canonize broadcast instructions before ISel phase and
// eliminate additional patterns on ISel.
[AVX512] Added lowering for VBROADCASTSS/SD instructions. Lowering patterns were written through avx512_broadcast_pat multiclass as pattern generates VBROADCAST and COPY_TO_REGCLASS nodes. Added lowering tests. llvm-svn: 223804
2014-12-10 02:45:30 +08:00
// SrcRC_v and SrcRC_s are RegisterClasses for vector and scalar
// representations of source
multiclass avx512_broadcast_pat<string InstName, SDNode OpNode,
X86VectorVTInfo _, RegisterClass SrcRC_v,
RegisterClass SrcRC_s> {
[AVX512] Minor fix in lowering pattern for broadcast intrustions. No functional change. llvm-svn: 224122
2014-12-12 22:21:30 +08:00
def : Pat<(_.VT (OpNode (_.EltVT SrcRC_s:$src))),
[AVX512] Added lowering for VBROADCASTSS/SD instructions. Lowering patterns were written through avx512_broadcast_pat multiclass as pattern generates VBROADCAST and COPY_TO_REGCLASS nodes. Added lowering tests. llvm-svn: 223804
2014-12-10 02:45:30 +08:00
(!cast<Instruction>(InstName##"r")
(COPY_TO_REGCLASS SrcRC_s:$src, SrcRC_v))>;
let AddedComplexity = 30 in {
def : Pat<(_.VT (vselect _.KRCWM:$mask,
[AVX512] Minor fix in lowering pattern for broadcast intrustions. No functional change. llvm-svn: 224122
2014-12-12 22:21:30 +08:00
(OpNode (_.EltVT SrcRC_s:$src)), _.RC:$src0)),
[AVX512] Added lowering for VBROADCASTSS/SD instructions. Lowering patterns were written through avx512_broadcast_pat multiclass as pattern generates VBROADCAST and COPY_TO_REGCLASS nodes. Added lowering tests. llvm-svn: 223804
2014-12-10 02:45:30 +08:00
(!cast<Instruction>(InstName##"rk") _.RC:$src0, _.KRCWM:$mask,
(COPY_TO_REGCLASS SrcRC_s:$src, SrcRC_v))>;
def : Pat<(_.VT(vselect _.KRCWM:$mask,
[AVX512] Minor fix in lowering pattern for broadcast intrustions. No functional change. llvm-svn: 224122
2014-12-12 22:21:30 +08:00
(OpNode (_.EltVT SrcRC_s:$src)), _.ImmAllZerosV)),
[AVX512] Added lowering for VBROADCASTSS/SD instructions. Lowering patterns were written through avx512_broadcast_pat multiclass as pattern generates VBROADCAST and COPY_TO_REGCLASS nodes. Added lowering tests. llvm-svn: 223804
2014-12-10 02:45:30 +08:00
(!cast<Instruction>(InstName##"rkz") _.KRCWM:$mask,
(COPY_TO_REGCLASS SrcRC_s:$src, SrcRC_v))>;
}
}
defm : avx512_broadcast_pat<"VBROADCASTSSZ", X86VBroadcast, v16f32_info,
VR128X, FR32X>;
defm : avx512_broadcast_pat<"VBROADCASTSDZ", X86VBroadcast, v8f64_info,
VR128X, FR64X>;
let Predicates = [HasVLX] in {
defm : avx512_broadcast_pat<"VBROADCASTSSZ256", X86VBroadcast,
v8f32x_info, VR128X, FR32X>;
defm : avx512_broadcast_pat<"VBROADCASTSSZ128", X86VBroadcast,
v4f32x_info, VR128X, FR32X>;
defm : avx512_broadcast_pat<"VBROADCASTSDZ256", X86VBroadcast,
v4f64x_info, VR128X, FR64X>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (X86VBroadcast (loadf32 addr:$src))),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSSZm addr:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (X86VBroadcast (loadf64 addr:$src))),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSDZm addr:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[X86][AVX512] Add patterns that match the AVX512 floating point vbroadcast intrinsics. Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193421
2013-10-26 01:47:18 +08:00
def : Pat<(int_x86_avx512_vbroadcast_ss_512 addr:$src),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSSZm addr:$src)>;
[X86][AVX512] Add patterns that match the AVX512 floating point vbroadcast intrinsics. Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193421
2013-10-26 01:47:18 +08:00
def : Pat<(int_x86_avx512_vbroadcast_sd_512 addr:$src),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSDZm addr:$src)>;
[X86][AVX512] Add patterns that match the AVX512 floating point vbroadcast intrinsics. Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193421
2013-10-26 01:47:18 +08:00
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
multiclass avx512_int_broadcast_reg<bits<8> opc, X86VectorVTInfo _,
RegisterClass SrcRC> {
defm r : AVX512_maskable_in_asm<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins SrcRC:$src), "vpbroadcast"##_.Suffix,
"$src", "$src", []>, T8PD, EVEX;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
multiclass avx512_int_broadcast_reg_vl<bits<8> opc, AVX512VLVectorVTInfo _,
RegisterClass SrcRC, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_int_broadcast_reg<opc, _.info512, SrcRC>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_int_broadcast_reg<opc, _.info256, SrcRC>, EVEX_V256;
defm Z128 : avx512_int_broadcast_reg<opc, _.info128, SrcRC>, EVEX_V128;
}
}
defm VPBROADCASTBr : avx512_int_broadcast_reg_vl<0x7A, avx512vl_i8_info, GR32,
HasBWI>;
defm VPBROADCASTWr : avx512_int_broadcast_reg_vl<0x7B, avx512vl_i16_info, GR32,
HasBWI>;
defm VPBROADCASTDr : avx512_int_broadcast_reg_vl<0x7C, avx512vl_i32_info, GR32,
HasAVX512>;
defm VPBROADCASTQr : avx512_int_broadcast_reg_vl<0x7C, avx512vl_i64_info, GR64,
HasAVX512>, VEX_W;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat <(v16i32 (X86vzext VK16WM:$mask)),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTDrZrkz VK16WM:$mask, (i32 (MOV32ri 0x1)))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat <(v8i64 (X86vzext VK8WM:$mask)),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTQrZrkz VK8WM:$mask, (i64 (MOV64ri 0x1)))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16i32 (X86VBroadcast (i32 GR32:$src))),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTDrZr GR32:$src)>;
Add AVX512 patterns for v16i32 broadcast and v2i64 zero extend load. Patch by Aleksey Bader. llvm-svn: 196435
2013-12-05 08:11:25 +08:00
def : Pat<(v16i32 (X86VBroadcastm VK16WM:$mask, (i32 GR32:$src))),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTDrZrkz VK16WM:$mask, GR32:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i64 (X86VBroadcast (i64 GR64:$src))),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTQrZr GR64:$src)>;
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
def : Pat<(v8i64 (X86VBroadcastm VK8WM:$mask, (i64 GR64:$src))),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTQrZrkz VK8WM:$mask, GR64:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Add AVX512 unmasked integer broadcast intrinsics and support. llvm-svn: 193748
2013-10-31 21:56:31 +08:00
def : Pat<(v16i32 (int_x86_avx512_pbroadcastd_i32_512 (i32 GR32:$src))),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTDrZr GR32:$src)>;
Add AVX512 unmasked integer broadcast intrinsics and support. llvm-svn: 193748
2013-10-31 21:56:31 +08:00
def : Pat<(v8i64 (int_x86_avx512_pbroadcastq_i64_512 (i64 GR64:$src))),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTQrZr GR64:$src)>;
Add AVX512 unmasked integer broadcast intrinsics and support. llvm-svn: 193748
2013-10-31 21:56:31 +08:00
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def : Pat<(v16i32 (int_x86_avx512_mask_pbroadcast_d_gpr_512 (i32 GR32:$src),
(v16i32 immAllZerosV), (i16 GR16:$mask))),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTDrZrkz (COPY_TO_REGCLASS GR16:$mask, VK16WM), GR32:$src)>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def : Pat<(v8i64 (int_x86_avx512_mask_pbroadcast_q_gpr_512 (i64 GR64:$src),
(bc_v8i64 (v16i32 immAllZerosV)), (i8 GR8:$mask))),
[AVX512] Added VPBROADCAST{BWDQ} (Load with Broadcast Integer Data from General Purpose Register) encodings for AVX512-BW/VL subsets Added encoding tests. llvm-svn: 223787
2014-12-10 00:38:41 +08:00
(VPBROADCASTQrZrkz (COPY_TO_REGCLASS GR8:$mask, VK8WM), GR64:$src)>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_int_broadcast_rm<bits<8> opc, string OpcodeStr,
X86MemOperand x86memop, PatFrag ld_frag,
RegisterClass DstRC, ValueType OpVT, ValueType SrcVT,
RegisterClass KRC> {
def rr : AVX5128I<opc, MRMSrcReg, (outs DstRC:$dst), (ins VR128X:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set DstRC:$dst,
(OpVT (X86VBroadcast (SrcVT VR128X:$src))))]>, EVEX;
def krr : AVX5128I<opc, MRMSrcReg, (outs DstRC:$dst), (ins KRC:$mask,
VR128X:$src),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set DstRC:$dst,
(OpVT (X86VBroadcastm KRC:$mask, (SrcVT VR128X:$src))))]>,
EVEX, EVEX_KZ;
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
let mayLoad = 1 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rm : AVX5128I<opc, MRMSrcMem, (outs DstRC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set DstRC:$dst,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(OpVT (X86VBroadcast (ld_frag addr:$src))))]>, EVEX;
def krm : AVX5128I<opc, MRMSrcMem, (outs DstRC:$dst), (ins KRC:$mask,
x86memop:$src),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src}"),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set DstRC:$dst, (OpVT (X86VBroadcastm KRC:$mask,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ld_frag addr:$src))))]>, EVEX, EVEX_KZ;
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm VPBROADCASTDZ : avx512_int_broadcast_rm<0x58, "vpbroadcastd", i32mem,
loadi32, VR512, v16i32, v4i32, VK16WM>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
defm VPBROADCASTQZ : avx512_int_broadcast_rm<0x59, "vpbroadcastq", i64mem,
loadi64, VR512, v8i64, v2i64, VK8WM>, EVEX_V512, VEX_W,
EVEX_CD8<64, CD8VT1>;
[X86] AVX512: Add vbroadcasti* For now I used a separate template for these sub-vector/tuple broadcasts rather than sharing the mem variants with avx512_int_broadcast_rm. <rdar://problem/17402869> llvm-svn: 211828
2014-06-27 08:43:38 +08:00
multiclass avx512_int_subvec_broadcast_rm<bits<8> opc, string OpcodeStr,
X86MemOperand x86memop, PatFrag ld_frag,
RegisterClass KRC> {
let mayLoad = 1 in {
def rm : AVX5128I<opc, MRMSrcMem, (outs VR512:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[X86] AVX512: Add vbroadcasti* For now I used a separate template for these sub-vector/tuple broadcasts rather than sharing the mem variants with avx512_int_broadcast_rm. <rdar://problem/17402869> llvm-svn: 211828
2014-06-27 08:43:38 +08:00
[]>, EVEX;
def krm : AVX5128I<opc, MRMSrcMem, (outs VR512:$dst), (ins KRC:$mask,
x86memop:$src),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src}"),
[X86] AVX512: Add vbroadcasti* For now I used a separate template for these sub-vector/tuple broadcasts rather than sharing the mem variants with avx512_int_broadcast_rm. <rdar://problem/17402869> llvm-svn: 211828
2014-06-27 08:43:38 +08:00
[]>, EVEX, EVEX_KZ;
}
}
defm VBROADCASTI32X4 : avx512_int_subvec_broadcast_rm<0x5a, "vbroadcasti32x4",
i128mem, loadv2i64, VK16WM>,
EVEX_V512, EVEX_CD8<32, CD8VT4>;
defm VBROADCASTI64X4 : avx512_int_subvec_broadcast_rm<0x5b, "vbroadcasti64x4",
i256mem, loadv4i64, VK16WM>, VEX_W,
EVEX_V512, EVEX_CD8<64, CD8VT4>;
Add AVX512 unmasked integer broadcast intrinsics and support. llvm-svn: 193748
2013-10-31 21:56:31 +08:00
def : Pat<(v16i32 (int_x86_avx512_pbroadcastd_512 (v4i32 VR128X:$src))),
(VPBROADCASTDZrr VR128X:$src)>;
def : Pat<(v8i64 (int_x86_avx512_pbroadcastq_512 (v2i64 VR128X:$src))),
(VPBROADCASTQZrr VR128X:$src)>;
[AVX512] Bring back vector-shuffle lowering support through broadcasts Ffter commit at rev219046 512-bit broadcasts lowering become non-optimal. Most of tests on broadcasting and embedded broadcasting were changed and they doesn’t produce efficient code. Example below is from commit changes (it’s the first test from test/CodeGen/X86/avx512-vbroadcast.ll): define <16 x i32> @_inreg16xi32(i32 %a) { ; CHECK-LABEL: _inreg16xi32: ; CHECK: ## BB#0: -; CHECK-NEXT: vpbroadcastd %edi, %zmm0 +; CHECK-NEXT: vmovd %edi, %xmm0 +; CHECK-NEXT: vpbroadcastd %xmm0, %ymm0 +; CHECK-NEXT: vinserti64x4 $1, %ymm0, %zmm0, %zmm0 ; CHECK-NEXT: retq %b = insertelement <16 x i32> undef, i32 %a, i32 0 %c = shufflevector <16 x i32> %b, <16 x i32> undef, <16 x i32> zeroinitializer ret <16 x i32> %c } Here, 256-bit broadcast was generated instead of 512-bit one. In this patch 1) I added vector-shuffle lowering through broadcasts 2) Removed asserts and branches likes because this is incorrect - assert(Subtarget->hasDQI() && "We can only lower v8i64 with AVX-512-DQI"); 3) Fixed lowering tests llvm-svn: 220774
2014-10-28 20:28:51 +08:00
def : Pat<(v16f32 (X86VBroadcast (v16f32 VR512:$src))),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSSZr (EXTRACT_SUBREG (v16f32 VR512:$src), sub_xmm))>;
[AVX512] Bring back vector-shuffle lowering support through broadcasts Ffter commit at rev219046 512-bit broadcasts lowering become non-optimal. Most of tests on broadcasting and embedded broadcasting were changed and they doesn’t produce efficient code. Example below is from commit changes (it’s the first test from test/CodeGen/X86/avx512-vbroadcast.ll): define <16 x i32> @_inreg16xi32(i32 %a) { ; CHECK-LABEL: _inreg16xi32: ; CHECK: ## BB#0: -; CHECK-NEXT: vpbroadcastd %edi, %zmm0 +; CHECK-NEXT: vmovd %edi, %xmm0 +; CHECK-NEXT: vpbroadcastd %xmm0, %ymm0 +; CHECK-NEXT: vinserti64x4 $1, %ymm0, %zmm0, %zmm0 ; CHECK-NEXT: retq %b = insertelement <16 x i32> undef, i32 %a, i32 0 %c = shufflevector <16 x i32> %b, <16 x i32> undef, <16 x i32> zeroinitializer ret <16 x i32> %c } Here, 256-bit broadcast was generated instead of 512-bit one. In this patch 1) I added vector-shuffle lowering through broadcasts 2) Removed asserts and branches likes because this is incorrect - assert(Subtarget->hasDQI() && "We can only lower v8i64 with AVX-512-DQI"); 3) Fixed lowering tests llvm-svn: 220774
2014-10-28 20:28:51 +08:00
def : Pat<(v8f64 (X86VBroadcast (v8f64 VR512:$src))),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSDZr (EXTRACT_SUBREG (v8f64 VR512:$src), sub_xmm))>;
[AVX512] Bring back vector-shuffle lowering support through broadcasts Ffter commit at rev219046 512-bit broadcasts lowering become non-optimal. Most of tests on broadcasting and embedded broadcasting were changed and they doesn’t produce efficient code. Example below is from commit changes (it’s the first test from test/CodeGen/X86/avx512-vbroadcast.ll): define <16 x i32> @_inreg16xi32(i32 %a) { ; CHECK-LABEL: _inreg16xi32: ; CHECK: ## BB#0: -; CHECK-NEXT: vpbroadcastd %edi, %zmm0 +; CHECK-NEXT: vmovd %edi, %xmm0 +; CHECK-NEXT: vpbroadcastd %xmm0, %ymm0 +; CHECK-NEXT: vinserti64x4 $1, %ymm0, %zmm0, %zmm0 ; CHECK-NEXT: retq %b = insertelement <16 x i32> undef, i32 %a, i32 0 %c = shufflevector <16 x i32> %b, <16 x i32> undef, <16 x i32> zeroinitializer ret <16 x i32> %c } Here, 256-bit broadcast was generated instead of 512-bit one. In this patch 1) I added vector-shuffle lowering through broadcasts 2) Removed asserts and branches likes because this is incorrect - assert(Subtarget->hasDQI() && "We can only lower v8i64 with AVX-512-DQI"); 3) Fixed lowering tests llvm-svn: 220774
2014-10-28 20:28:51 +08:00
def : Pat<(v16i32 (X86VBroadcast (v16i32 VR512:$src))),
(VPBROADCASTDZrr (EXTRACT_SUBREG (v16i32 VR512:$src), sub_xmm))>;
def : Pat<(v8i64 (X86VBroadcast (v8i64 VR512:$src))),
(VPBROADCASTQZrr (EXTRACT_SUBREG (v8i64 VR512:$src), sub_xmm))>;
[X86][AVX512] Add patterns that match the AVX512 floating point register vbroadcast intrinsics. Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193422
2013-10-26 02:04:12 +08:00
def : Pat<(v16f32 (int_x86_avx512_vbroadcast_ss_ps_512 (v4f32 VR128X:$src))),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSSZr VR128X:$src)>;
[X86][AVX512] Add patterns that match the AVX512 floating point register vbroadcast intrinsics. Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193422
2013-10-26 02:04:12 +08:00
def : Pat<(v8f64 (int_x86_avx512_vbroadcast_sd_pd_512 (v2f64 VR128X:$src))),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSDZr VR128X:$src)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Provide fallback in case the load node that is used in the patterns above
// is used by additional users, which prevents the pattern selection.
def : Pat<(v16f32 (X86VBroadcast FR32X:$src)),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSSZr (COPY_TO_REGCLASS FR32X:$src, VR128X))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (X86VBroadcast FR64X:$src)),
[AVX512] Added VBROADCAST{SS/SD} encoding for VL subset. Refactored through AVX512_maskable llvm-svn: 220908
2014-10-30 22:21:47 +08:00
(VBROADCASTSDZr (COPY_TO_REGCLASS FR64X:$src, VR128X))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(v8i32 (X86VBroadcastm (v8i1 VK8WM:$mask), (loadi32 addr:$src))),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(EXTRACT_SUBREG
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16i32 (VPBROADCASTDZkrm (COPY_TO_REGCLASS VK8WM:$mask, VK16WM),
addr:$src)), sub_ymm)>;
}
//===----------------------------------------------------------------------===//
// AVX-512 BROADCAST MASK TO VECTOR REGISTER
//---
multiclass avx512_mask_broadcast<bits<8> opc, string OpcodeStr,
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
RegisterClass KRC> {
let Predicates = [HasCDI] in
def Zrr : AVX512XS8I<opc, MRMSrcReg, (outs VR512:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
[]>, EVEX, EVEX_V512;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
let Predicates = [HasCDI, HasVLX] in {
def Z128rr : AVX512XS8I<opc, MRMSrcReg, (outs VR128:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
[]>, EVEX, EVEX_V128;
def Z256rr : AVX512XS8I<opc, MRMSrcReg, (outs VR256:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
[]>, EVEX, EVEX_V256;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
Add HasCDI predicate to AVX512 VPBROADCASTM*. llvm-svn: 210892
2014-06-13 19:40:31 +08:00
let Predicates = [HasCDI] in {
AVX-512: Fixed encoding of VPBROADCASTM and added SKX forms of this instruction llvm-svn: 220638
2014-10-26 17:52:24 +08:00
defm VPBROADCASTMW2D : avx512_mask_broadcast<0x3A, "vpbroadcastmw2d",
VK16>;
defm VPBROADCASTMB2Q : avx512_mask_broadcast<0x2A, "vpbroadcastmb2q",
VK8>, VEX_W;
Add HasCDI predicate to AVX512 VPBROADCASTM*. llvm-svn: 210892
2014-06-13 19:40:31 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - VPERM
//
// -- immediate form --
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
multiclass avx512_perm_imm<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
let ExeDomain = _.ExeDomain in {
def ri : AVX512AIi8<opc, MRMSrcReg, (outs _.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins _.RC:$src1, u8imm:$src2),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
[(set _.RC:$dst,
(_.VT (OpNode _.RC:$src1, (i8 imm:$src2))))]>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX;
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
def mi : AVX512AIi8<opc, MRMSrcMem, (outs _.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins _.MemOp:$src1, u8imm:$src2),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
[(set _.RC:$dst,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(_.VT (OpNode (_.LdFrag addr:$src1),
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
(i8 imm:$src2))))]>,
EVEX, EVEX_CD8<_.EltSize, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
}
[AVX512] Add vpermil variable version This is implemented via a multiclass that derives from the vperm imm multiclass. Fixes <rdar://problem/18426089> llvm-svn: 220737
2014-10-28 07:08:40 +08:00
multiclass avx512_permil<bits<8> OpcImm, bits<8> OpcVar, X86VectorVTInfo _,
X86VectorVTInfo Ctrl> :
avx512_perm_imm<OpcImm, "vpermil" # _.Suffix, X86VPermilpi, _> {
let ExeDomain = _.ExeDomain in {
def rr : AVX5128I<OpcVar, MRMSrcReg, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2),
!strconcat("vpermil" # _.Suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[AVX512] Add vpermil variable version This is implemented via a multiclass that derives from the vperm imm multiclass. Fixes <rdar://problem/18426089> llvm-svn: 220737
2014-10-28 07:08:40 +08:00
[(set _.RC:$dst,
(_.VT (X86VPermilpv _.RC:$src1,
(Ctrl.VT Ctrl.RC:$src2))))]>,
EVEX_4V;
def rm : AVX5128I<OpcVar, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, Ctrl.MemOp:$src2),
!strconcat("vpermil" # _.Suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[AVX512] Add vpermil variable version This is implemented via a multiclass that derives from the vperm imm multiclass. Fixes <rdar://problem/18426089> llvm-svn: 220737
2014-10-28 07:08:40 +08:00
[(set _.RC:$dst,
(_.VT (X86VPermilpv _.RC:$src1,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(Ctrl.VT (Ctrl.LdFrag addr:$src2)))))]>,
[AVX512] Add vpermil variable version This is implemented via a multiclass that derives from the vperm imm multiclass. Fixes <rdar://problem/18426089> llvm-svn: 220737
2014-10-28 07:08:40 +08:00
EVEX_4V;
}
}
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
defm VPERMQZ : avx512_perm_imm<0x00, "vpermq", X86VPermi, v8i64_info>,
EVEX_V512, VEX_W;
defm VPERMPDZ : avx512_perm_imm<0x01, "vpermpd", X86VPermi, v8f64_info>,
EVEX_V512, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Add vpermil variable version This is implemented via a multiclass that derives from the vperm imm multiclass. Fixes <rdar://problem/18426089> llvm-svn: 220737
2014-10-28 07:08:40 +08:00
defm VPERMILPSZ : avx512_permil<0x04, 0x0C, v16f32_info, v16i32_info>,
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
EVEX_V512;
[AVX512] Add vpermil variable version This is implemented via a multiclass that derives from the vperm imm multiclass. Fixes <rdar://problem/18426089> llvm-svn: 220737
2014-10-28 07:08:40 +08:00
defm VPERMILPDZ : avx512_permil<0x05, 0x0D, v8f64_info, v8i64_info>,
[AVX512] Clean up avx512_perm_imm to use X86VectorVTInfo No functionality change. No change in X86.td.expanded except that we only set the CD8 attributes for the memory variants. (This shouldn't be used unless we have a memory operand.) llvm-svn: 220736
2014-10-28 07:08:37 +08:00
EVEX_V512, VEX_W;
[AVX512] Derive vpermil* from avx512_perm_imm This used to derive from avx512_pshuf_imm which is confusing. NFC. Compared X86.td.expanded. llvm-svn: 220735
2014-10-28 07:08:34 +08:00
def : Pat<(v16i32 (X86VPermilpi VR512:$src1, (i8 imm:$imm))),
(VPERMILPSZri VR512:$src1, imm:$imm)>;
def : Pat<(v8i64 (X86VPermilpi VR512:$src1, (i8 imm:$imm))),
(VPERMILPDZri VR512:$src1, imm:$imm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// -- VPERM - register form --
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
multiclass avx512_perm<bits<8> opc, string OpcodeStr, RegisterClass RC,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
PatFrag mem_frag, X86MemOperand x86memop, ValueType OpVT> {
def rr : AVX5128I<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src1, RC:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
(OpVT (X86VPermv RC:$src1, RC:$src2)))]>, EVEX_4V;
def rm : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, x86memop:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
(OpVT (X86VPermv RC:$src1, (mem_frag addr:$src2))))]>,
EVEX_4V;
}
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMDZ : avx512_perm<0x36, "vpermd", VR512, loadv16i32, i512mem,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
v16i32>, EVEX_V512, EVEX_CD8<32, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMQZ : avx512_perm<0x36, "vpermq", VR512, loadv8i64, i512mem,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
v8i64>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
let ExeDomain = SSEPackedSingle in
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMPSZ : avx512_perm<0x16, "vpermps", VR512, loadv16f32, f512mem,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
v16f32>, EVEX_V512, EVEX_CD8<32, CD8VF>;
let ExeDomain = SSEPackedDouble in
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMPDZ : avx512_perm<0x16, "vpermpd", VR512, loadv8f64, f512mem,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
v8f64>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
// -- VPERM2I - 3 source operands form --
multiclass avx512_perm_3src<bits<8> opc, string OpcodeStr, RegisterClass RC,
PatFrag mem_frag, X86MemOperand x86memop,
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
SDNode OpNode, ValueType OpVT, RegisterClass KRC> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Constraints = "$src1 = $dst" in {
def rr : AVX5128I<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src1, RC:$src2, RC:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
AVX-512: added VPERM2D VPERM2Q VPERM2PS VPERM2PD instructions, they give better sequences than VPERMI llvm-svn: 199893
2014-01-23 22:27:26 +08:00
(OpVT (OpNode RC:$src1, RC:$src2, RC:$src3)))]>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_4V;
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
def rrk : AVX5128I<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src1, KRC:$mask, RC:$src2, RC:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst {${mask}}|"
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
"$dst {${mask}}, $src2, $src3}"),
[(set RC:$dst, (OpVT (vselect KRC:$mask,
(OpNode RC:$src1, RC:$src2,
RC:$src3),
RC:$src1)))]>,
EVEX_4V, EVEX_K;
let AddedComplexity = 30 in // Prefer over VMOV*rrkz Pat<>
def rrkz : AVX5128I<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src1, KRC:$mask, RC:$src2, RC:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst {${mask}} {z} |",
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
"$dst {${mask}} {z}, $src2, $src3}"),
[(set RC:$dst, (OpVT (vselect KRC:$mask,
(OpNode RC:$src1, RC:$src2,
RC:$src3),
(OpVT (bitconvert
(v16i32 immAllZerosV))))))]>,
EVEX_4V, EVEX_KZ;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rm : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, RC:$src2, x86memop:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
(OpVT (OpNode RC:$src1, RC:$src2,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(mem_frag addr:$src3))))]>, EVEX_4V;
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
def rmk : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, KRC:$mask, RC:$src2, x86memop:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst {${mask}}|"
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
"$dst {${mask}}, $src2, $src3}"),
[(set RC:$dst,
(OpVT (vselect KRC:$mask,
(OpNode RC:$src1, RC:$src2,
(mem_frag addr:$src3)),
RC:$src1)))]>,
EVEX_4V, EVEX_K;
let AddedComplexity = 10 in // Prefer over the rrkz variant
def rmkz : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, KRC:$mask, RC:$src2, x86memop:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst {${mask}} {z}|"
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
"$dst {${mask}} {z}, $src2, $src3}"),
[(set RC:$dst,
(OpVT (vselect KRC:$mask,
(OpNode RC:$src1, RC:$src2,
(mem_frag addr:$src3)),
(OpVT (bitconvert
(v16i32 immAllZerosV))))))]>,
EVEX_4V, EVEX_KZ;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMI2D : avx512_perm_3src<0x76, "vpermi2d", VR512, loadv16i32,
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
i512mem, X86VPermiv3, v16i32, VK16WM>,
EVEX_V512, EVEX_CD8<32, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMI2Q : avx512_perm_3src<0x76, "vpermi2q", VR512, loadv8i64,
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
i512mem, X86VPermiv3, v8i64, VK8WM>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMI2PS : avx512_perm_3src<0x77, "vpermi2ps", VR512, loadv16f32,
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
i512mem, X86VPermiv3, v16f32, VK16WM>,
EVEX_V512, EVEX_CD8<32, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMI2PD : avx512_perm_3src<0x77, "vpermi2pd", VR512, loadv8f64,
[X86] AVX512: Add writemask variants for vperm*2* This includes assembler and codegen support (see the new tests in avx512-encodings.s and avx512-shuffle.ll). <rdar://problem/17492620> llvm-svn: 212221
2014-07-03 05:25:54 +08:00
i512mem, X86VPermiv3, v8f64, VK8WM>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: added VPERM2D VPERM2Q VPERM2PS VPERM2PD instructions, they give better sequences than VPERMI llvm-svn: 199893
2014-01-23 22:27:26 +08:00
[X86] AVX512: Generate Pat<>'s for the vpermt2* intrinsics via multiclass This new multiclass, avx512_perm_table_3src derives from the current one and provides the Pat<>. The next patch will add another Pat<> that uses the writemask. Note that I dropped the type annotation from the intrinsic call, i.e.: (v16f32 VR512:$src1) -> R512:$src1. I think that this should be fine (at least many intrinsic calls don't provide them) and it greatly reduces the number of template arguments. llvm-svn: 212222
2014-07-03 05:25:58 +08:00
multiclass avx512_perm_table_3src<bits<8> opc, string Suffix, RegisterClass RC,
PatFrag mem_frag, X86MemOperand x86memop,
[X86] AVX512: Allow writemask argument in vpermt* intrinsics llvm-svn: 212223
2014-07-03 05:26:01 +08:00
SDNode OpNode, ValueType OpVT, RegisterClass KRC,
ValueType MaskVT, RegisterClass MRC> :
[X86] AVX512: Generate Pat<>'s for the vpermt2* intrinsics via multiclass This new multiclass, avx512_perm_table_3src derives from the current one and provides the Pat<>. The next patch will add another Pat<> that uses the writemask. Note that I dropped the type annotation from the intrinsic call, i.e.: (v16f32 VR512:$src1) -> R512:$src1. I think that this should be fine (at least many intrinsic calls don't provide them) and it greatly reduces the number of template arguments. llvm-svn: 212222
2014-07-03 05:25:58 +08:00
avx512_perm_3src<opc, "vpermt2"##Suffix, RC, mem_frag, x86memop, OpNode,
OpVT, KRC> {
def : Pat<(OpVT (!cast<Intrinsic>("int_x86_avx512_mask_vpermt_"##Suffix##"_512")
VR512:$idx, VR512:$src1, VR512:$src2, -1)),
(!cast<Instruction>(NAME#rr) VR512:$src1, VR512:$idx, VR512:$src2)>;
[X86] AVX512: Allow writemask argument in vpermt* intrinsics llvm-svn: 212223
2014-07-03 05:26:01 +08:00
def : Pat<(OpVT (!cast<Intrinsic>("int_x86_avx512_mask_vpermt_"##Suffix##"_512")
VR512:$idx, VR512:$src1, VR512:$src2, MRC:$mask)),
(!cast<Instruction>(NAME#rrk) VR512:$src1,
(MaskVT (COPY_TO_REGCLASS MRC:$mask, KRC)), VR512:$idx, VR512:$src2)>;
[X86] AVX512: Generate Pat<>'s for the vpermt2* intrinsics via multiclass This new multiclass, avx512_perm_table_3src derives from the current one and provides the Pat<>. The next patch will add another Pat<> that uses the writemask. Note that I dropped the type annotation from the intrinsic call, i.e.: (v16f32 VR512:$src1) -> R512:$src1. I think that this should be fine (at least many intrinsic calls don't provide them) and it greatly reduces the number of template arguments. llvm-svn: 212222
2014-07-03 05:25:58 +08:00
}
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMT2D : avx512_perm_table_3src<0x7E, "d", VR512, loadv16i32, i512mem,
[X86] AVX512: Allow writemask argument in vpermt* intrinsics llvm-svn: 212223
2014-07-03 05:26:01 +08:00
X86VPermv3, v16i32, VK16WM, v16i1, GR16>,
EVEX_V512, EVEX_CD8<32, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMT2Q : avx512_perm_table_3src<0x7E, "q", VR512, loadv8i64, i512mem,
[X86] AVX512: Allow writemask argument in vpermt* intrinsics llvm-svn: 212223
2014-07-03 05:26:01 +08:00
X86VPermv3, v8i64, VK8WM, v8i1, GR8>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMT2PS : avx512_perm_table_3src<0x7F, "ps", VR512, loadv16f32, i512mem,
[X86] AVX512: Allow writemask argument in vpermt* intrinsics llvm-svn: 212223
2014-07-03 05:26:01 +08:00
X86VPermv3, v16f32, VK16WM, v16i1, GR16>,
EVEX_V512, EVEX_CD8<32, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPERMT2PD : avx512_perm_table_3src<0x7F, "pd", VR512, loadv8f64, i512mem,
[X86] AVX512: Allow writemask argument in vpermt* intrinsics llvm-svn: 212223
2014-07-03 05:26:01 +08:00
X86VPermv3, v8f64, VK8WM, v8i1, GR8>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: optimized a shuffle pattern to VINSERTI64x4. Added intrinsics for VPERMT2PS/PD/D/Q instructions. llvm-svn: 207513
2014-04-29 17:09:15 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - BLEND using mask
//
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
multiclass avx512_blendmask<bits<8> opc, string OpcodeStr, X86VectorVTInfo _> {
let ExeDomain = _.ExeDomain in {
def rr : AVX5128I<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr,
"\t{$src2, $src1, ${dst} |${dst}, $src1, $src2}"),
[]>, EVEX_4V;
def rrk : AVX5128I<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2),
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, ${dst} {${mask}}|${dst} {${mask}}, $src1, $src2}"),
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
[(set _.RC:$dst, (X86select _.KRCWM:$mask, (_.VT _.RC:$src1),
(_.VT _.RC:$src2)))]>, EVEX_4V, EVEX_K;
def rrkz : AVX5128I<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr,
"\t{$src2, $src1, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src1, $src2}"),
[]>, EVEX_4V, EVEX_KZ;
let mayLoad = 1 in {
def rm : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, _.MemOp:$src2),
!strconcat(OpcodeStr,
"\t{$src2, $src1, ${dst} |${dst}, $src1, $src2}"),
[]>, EVEX_4V, EVEX_CD8<_.EltSize, CD8VF>;
def rmk : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2),
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, ${dst} {${mask}}|${dst} {${mask}}, $src1, $src2}"),
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
[(set _.RC:$dst, (X86select _.KRCWM:$mask, (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2)))))]>,
EVEX_4V, EVEX_K, EVEX_CD8<_.EltSize, CD8VF>;
def rmkz : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2),
!strconcat(OpcodeStr,
"\t{$src2, $src1, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src1, $src2}"),
[]>, EVEX_4V, EVEX_KZ, EVEX_CD8<_.EltSize, CD8VF>;
}
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
multiclass avx512_blendmask_rmb<bits<8> opc, string OpcodeStr, X86VectorVTInfo _> {
def rmbk : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src1, _.ScalarMemOp:$src2),
!strconcat(OpcodeStr,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.RC:$dst,(X86select _.KRCWM:$mask, (_.VT _.RC:$src1),
(X86VBroadcast (_.ScalarLdFrag addr:$src2))))]>,
AVX-512: BLENDM - fixed encoding of the broadcast version Added more intrinsics and encoding tests. llvm-svn: 224760
2014-12-23 17:36:28 +08:00
EVEX_4V, EVEX_K, EVEX_B, EVEX_CD8<_.EltSize, CD8VF>;
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
def rmb : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2),
!strconcat(OpcodeStr,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst|",
"$dst, $src1, ${src2}", _.BroadcastStr, "}"),
AVX-512: BLENDM - fixed encoding of the broadcast version Added more intrinsics and encoding tests. llvm-svn: 224760
2014-12-23 17:36:28 +08:00
[]>, EVEX_4V, EVEX_B, EVEX_CD8<_.EltSize, CD8VF>;
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
}
multiclass blendmask_dq <bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
defm Z : avx512_blendmask <opc, OpcodeStr, VTInfo.info512>,
avx512_blendmask_rmb <opc, OpcodeStr, VTInfo.info512>, EVEX_V512;
let Predicates = [HasVLX] in {
defm Z256 : avx512_blendmask<opc, OpcodeStr, VTInfo.info256>,
avx512_blendmask_rmb <opc, OpcodeStr, VTInfo.info256>, EVEX_V256;
defm Z128 : avx512_blendmask<opc, OpcodeStr, VTInfo.info128>,
avx512_blendmask_rmb <opc, OpcodeStr, VTInfo.info128>, EVEX_V128;
}
}
multiclass blendmask_bw <bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
let Predicates = [HasBWI] in
defm Z : avx512_blendmask <opc, OpcodeStr, VTInfo.info512>, EVEX_V512;
let Predicates = [HasBWI, HasVLX] in {
defm Z256 : avx512_blendmask <opc, OpcodeStr, VTInfo.info256>, EVEX_V256;
defm Z128 : avx512_blendmask <opc, OpcodeStr, VTInfo.info128>, EVEX_V128;
}
}
defm VBLENDMPS : blendmask_dq <0x65, "vblendmps", avx512vl_f32_info>;
defm VBLENDMPD : blendmask_dq <0x65, "vblendmpd", avx512vl_f64_info>, VEX_W;
defm VPBLENDMD : blendmask_dq <0x64, "vpblendmd", avx512vl_i32_info>;
defm VPBLENDMQ : blendmask_dq <0x64, "vpblendmq", avx512vl_i64_info>, VEX_W;
defm VPBLENDMB : blendmask_bw <0x66, "vpblendmb", avx512vl_i8_info>;
defm VPBLENDMW : blendmask_bw <0x66, "vpblendmw", avx512vl_i16_info>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(v8f32 (vselect (v8i1 VK8WM:$mask), (v8f32 VR256X:$src1),
(v8f32 VR256X:$src2))),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(EXTRACT_SUBREG
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
(v16f32 (VBLENDMPSZrrk (COPY_TO_REGCLASS VK8WM:$mask, VK16WM),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_ymm)>;
def : Pat<(v8i32 (vselect (v8i1 VK8WM:$mask), (v8i32 VR256X:$src1),
(v8i32 VR256X:$src2))),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(EXTRACT_SUBREG
AVX-512: Added all forms of BLENDM instructions, intrinsics, encoding tests for AVX-512F and skx instructions. llvm-svn: 224707
2014-12-22 21:52:48 +08:00
(v16i32 (VPBLENDMDZrrk (COPY_TO_REGCLASS VK8WM:$mask, VK16WM),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_ymm)>;
}
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
//===----------------------------------------------------------------------===//
// Compare Instructions
//===----------------------------------------------------------------------===//
// avx512_cmp_scalar - AVX512 CMPSS and CMPSD
multiclass avx512_cmp_scalar<RegisterClass RC, X86MemOperand x86memop,
Simplify a multiclass. No functional change. llvm-svn: 227038
2015-01-25 16:49:19 +08:00
SDNode OpNode, ValueType VT,
PatFrag ld_frag, string Suffix> {
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
def rr : AVX512Ii8<0xC2, MRMSrcReg,
Simplify a multiclass. No functional change. llvm-svn: 227038
2015-01-25 16:49:19 +08:00
(outs VK1:$dst), (ins RC:$src1, RC:$src2, AVXCC:$cc),
!strconcat("vcmp${cc}", Suffix,
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
[(set VK1:$dst, (OpNode (VT RC:$src1), RC:$src2, imm:$cc))],
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
IIC_SSE_ALU_F32S_RR>, EVEX_4V;
def rm : AVX512Ii8<0xC2, MRMSrcMem,
Simplify a multiclass. No functional change. llvm-svn: 227038
2015-01-25 16:49:19 +08:00
(outs VK1:$dst), (ins RC:$src1, x86memop:$src2, AVXCC:$cc),
!strconcat("vcmp${cc}", Suffix,
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
[(set VK1:$dst, (OpNode (VT RC:$src1),
(ld_frag addr:$src2), imm:$cc))], IIC_SSE_ALU_F32P_RM>, EVEX_4V;
Mark x86 _alt instructions as AsmParserOnly so they will be omitted from disassembler without string matches. llvm-svn: 198545
2014-01-05 12:55:55 +08:00
let isAsmParserOnly = 1, hasSideEffects = 0 in {
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
def rri_alt : AVX512Ii8<0xC2, MRMSrcReg,
[x86] Add assembly parser bounds checking to the immediate value for cmpss/cmpsd/cmpps/cmppd. llvm-svn: 226642
2015-01-21 14:07:53 +08:00
(outs VK1:$dst), (ins RC:$src1, RC:$src2, u8imm:$cc),
Simplify a multiclass. No functional change. llvm-svn: 227038
2015-01-25 16:49:19 +08:00
!strconcat("vcmp", Suffix,
"\t{$cc, $src2, $src1, $dst|$dst, $src1, $src2, $cc}"),
[], IIC_SSE_ALU_F32S_RR>, EVEX_4V;
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let mayLoad = 1 in
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
def rmi_alt : AVX512Ii8<0xC2, MRMSrcMem,
[x86] Add assembly parser bounds checking to the immediate value for cmpss/cmpsd/cmpps/cmppd. llvm-svn: 226642
2015-01-21 14:07:53 +08:00
(outs VK1:$dst), (ins RC:$src1, x86memop:$src2, u8imm:$cc),
Simplify a multiclass. No functional change. llvm-svn: 227038
2015-01-25 16:49:19 +08:00
!strconcat("vcmp", Suffix,
"\t{$cc, $src2, $src1, $dst|$dst, $src1, $src2, $cc}"),
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
}
}
let Predicates = [HasAVX512] in {
Simplify a multiclass. No functional change. llvm-svn: 227038
2015-01-25 16:49:19 +08:00
defm VCMPSSZ : avx512_cmp_scalar<FR32X, f32mem, X86cmpms, f32, loadf32, "ss">,
XS;
defm VCMPSDZ : avx512_cmp_scalar<FR64X, f64mem, X86cmpms, f64, loadf64, "sd">,
XD, VEX_W;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
multiclass avx512_icmp_packed<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : AVX512BI<opc, MRMSrcReg,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2)))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_ALU_F32P_RR>, EVEX_4V;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
let mayLoad = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rm : AVX512BI<opc, MRMSrcMem,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.MemOp:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2)))))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V;
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
def rrk : AVX512BI<opc, MRMSrcReg,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2))))],
IIC_SSE_ALU_F32P_RR>, EVEX_4V, EVEX_K;
let mayLoad = 1 in
def rmk : AVX512BI<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2),
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert
(_.LdFrag addr:$src2))))))],
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K;
}
multiclass avx512_icmp_packed_rmb<bits<8> opc, string OpcodeStr, SDNode OpNode,
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
X86VectorVTInfo _> :
avx512_icmp_packed<opc, OpcodeStr, OpNode, _> {
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
let mayLoad = 1 in {
def rmb : AVX512BI<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.RC:$src1, _.ScalarMemOp:$src2),
!strconcat(OpcodeStr, "\t{${src2}", _.BroadcastStr, ", $src1, $dst",
"|$dst, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1),
(X86VBroadcast (_.ScalarLdFrag addr:$src2))))],
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_B;
def rmbk : AVX512BI<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1,
_.ScalarMemOp:$src2),
!strconcat(OpcodeStr,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(X86VBroadcast
(_.ScalarLdFrag addr:$src2)))))],
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K, EVEX_B;
}
}
multiclass avx512_icmp_packed_vl<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_packed<opc, OpcodeStr, OpNode, VTInfo.info512>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_packed<opc, OpcodeStr, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_icmp_packed<opc, OpcodeStr, OpNode, VTInfo.info128>,
EVEX_V128;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
multiclass avx512_icmp_packed_rmb_vl<bits<8> opc, string OpcodeStr,
SDNode OpNode, AVX512VLVectorVTInfo VTInfo,
Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_packed_rmb<opc, OpcodeStr, OpNode, VTInfo.info512>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_packed_rmb<opc, OpcodeStr, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_icmp_packed_rmb<opc, OpcodeStr, OpNode, VTInfo.info128>,
EVEX_V128;
}
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
defm VPCMPEQB : avx512_icmp_packed_vl<0x74, "vpcmpeqb", X86pcmpeqm,
avx512vl_i8_info, HasBWI>,
EVEX_CD8<8, CD8VF>;
defm VPCMPEQW : avx512_icmp_packed_vl<0x75, "vpcmpeqw", X86pcmpeqm,
avx512vl_i16_info, HasBWI>,
EVEX_CD8<16, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPEQD : avx512_icmp_packed_rmb_vl<0x76, "vpcmpeqd", X86pcmpeqm,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
avx512vl_i32_info, HasAVX512>,
EVEX_CD8<32, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPEQQ : avx512_icmp_packed_rmb_vl<0x29, "vpcmpeqq", X86pcmpeqm,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
avx512vl_i64_info, HasAVX512>,
T8PD, VEX_W, EVEX_CD8<64, CD8VF>;
defm VPCMPGTB : avx512_icmp_packed_vl<0x64, "vpcmpgtb", X86pcmpgtm,
avx512vl_i8_info, HasBWI>,
EVEX_CD8<8, CD8VF>;
defm VPCMPGTW : avx512_icmp_packed_vl<0x65, "vpcmpgtw", X86pcmpgtm,
avx512vl_i16_info, HasBWI>,
EVEX_CD8<16, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPGTD : avx512_icmp_packed_rmb_vl<0x66, "vpcmpgtd", X86pcmpgtm,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
avx512vl_i32_info, HasAVX512>,
EVEX_CD8<32, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPGTQ : avx512_icmp_packed_rmb_vl<0x37, "vpcmpgtq", X86pcmpgtm,
[SKX] avx512_icmp_packed multiclass extension Extended avx512_icmp_packed multiclass by masking versions. Added avx512_icmp_packed_rmb multiclass for embedded broadcast versions. Added corresponding _vl multiclasses. Added encoding tests for CPCMP{EQ|GT}* instructions. Add more fields for X86VectorVTInfo. Added AVX512VLVectorVTInfo that include X86VectorVTInfo for 512/256/128-bit versions Differential Revision: http://reviews.llvm.org/D5024 llvm-svn: 216383
2014-08-25 22:49:34 +08:00
avx512vl_i64_info, HasAVX512>,
T8PD, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i1 (X86pcmpgtm (v8i32 VR256X:$src1), (v8i32 VR256X:$src2))),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
(COPY_TO_REGCLASS (VPCMPGTDZrr
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm))), VK8)>;
def : Pat<(v8i1 (X86pcmpeqm (v8i32 VR256X:$src1), (v8i32 VR256X:$src2))),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
(COPY_TO_REGCLASS (VPCMPEQDZrr
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm))), VK8)>;
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
multiclass avx512_icmp_cc<bits<8> opc, string Suffix, SDNode OpNode,
X86VectorVTInfo _> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rri : AVX512AIi8<opc, MRMSrcReg,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2, AVX512ICC:$cc),
[X86] AVX512: Factor generating the AsmString into avx512_icmp_cc Adding a writemask variant would require a third asm string to be passed to the template. Generate the AsmString in the template instead. No change in X86.td.expanded. llvm-svn: 212113
2014-07-02 02:03:43 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
imm:$cc))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_ALU_F32P_RR>, EVEX_4V;
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
let mayLoad = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rmi : AVX512AIi8<opc, MRMSrcMem,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.MemOp:$src2, AVX512ICC:$cc),
[X86] AVX512: Factor generating the AsmString into avx512_icmp_cc Adding a writemask variant would require a third asm string to be passed to the template. Generate the AsmString in the template instead. No change in X86.td.expanded. llvm-svn: 212113
2014-07-02 02:03:43 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2))),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V;
def rrik : AVX512AIi8<opc, MRMSrcReg,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
AVX512ICC:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{$src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc)))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RR>, EVEX_4V, EVEX_K;
let mayLoad = 1 in
def rmik : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
AVX512ICC:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{$src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(_.VT (bitconvert (_.LdFrag addr:$src2))),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc)))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Accept explicit immediate argument form instead of comparison code.
Mark x86 _alt instructions as AsmParserOnly so they will be omitted from disassembler without string matches. llvm-svn: 198545
2014-01-05 12:55:55 +08:00
let isAsmParserOnly = 1, hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rri_alt : AVX512AIi8<opc, MRMSrcReg,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.RC:$src2, u8imm:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp", Suffix, "\t{$cc, $src2, $src1, $dst|",
"$dst, $src1, $src2, $cc}"),
[X86] AVX512: Factor generating the AsmString into avx512_icmp_cc Adding a writemask variant would require a third asm string to be passed to the template. Generate the AsmString in the template instead. No change in X86.td.expanded. llvm-svn: 212113
2014-07-02 02:03:43 +08:00
[], IIC_SSE_ALU_F32P_RR>, EVEX_4V;
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let mayLoad = 1 in
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
def rmi_alt : AVX512AIi8<opc, MRMSrcMem,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(outs _.KRC:$dst), (ins _.RC:$src1, _.MemOp:$src2, u8imm:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp", Suffix, "\t{$cc, $src2, $src1, $dst|",
"$dst, $src1, $src2, $cc}"),
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V;
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
def rrik_alt : AVX512AIi8<opc, MRMSrcReg,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.RC:$src2,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
u8imm:$cc),
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
!strconcat("vpcmp", Suffix,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
"\t{$cc, $src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2, $cc}"),
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
[], IIC_SSE_ALU_F32P_RR>, EVEX_4V, EVEX_K;
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let mayLoad = 1 in
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
def rmik_alt : AVX512AIi8<opc, MRMSrcMem,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1, _.MemOp:$src2,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
u8imm:$cc),
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
!strconcat("vpcmp", Suffix,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
"\t{$cc, $src2, $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, $src2, $cc}"),
[X86] AVX512: Allow writemasks with vpcmp For now I only updated the _alt variants. The main variants are used by codegen and that will need a bit more work to trigger. <rdar://problem/17492620> llvm-svn: 212114
2014-07-02 02:03:45 +08:00
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
multiclass avx512_icmp_cc_rmb<bits<8> opc, string Suffix, SDNode OpNode,
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
X86VectorVTInfo _> :
avx512_icmp_cc<opc, Suffix, OpNode, _> {
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
def rmib : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.RC:$src1, _.ScalarMemOp:$src2,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
AVX512ICC:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst|",
"$dst, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.KRC:$dst, (OpNode (_.VT _.RC:$src1),
(X86VBroadcast (_.ScalarLdFrag addr:$src2)),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_B;
def rmibk : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1,
[X86] Teach disassembler to handle illegal immediates on AVX512 integer compare instructions. llvm-svn: 227302
2015-01-28 18:09:56 +08:00
_.ScalarMemOp:$src2, AVX512ICC:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp${cc}", Suffix,
"\t{${src2}", _.BroadcastStr, ", $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, ${src2}", _.BroadcastStr, "}"),
[(set _.KRC:$dst, (and _.KRCWM:$mask,
(OpNode (_.VT _.RC:$src1),
(X86VBroadcast (_.ScalarLdFrag addr:$src2)),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
imm:$cc)))],
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K, EVEX_B;
// Accept explicit immediate argument form instead of comparison code.
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let isAsmParserOnly = 1, hasSideEffects = 0, mayLoad = 1 in {
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
def rmib_alt : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.RC:$src1, _.ScalarMemOp:$src2,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
u8imm:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp", Suffix,
"\t{$cc, ${src2}", _.BroadcastStr, ", $src1, $dst|",
"$dst, $src1, ${src2}", _.BroadcastStr, ", $cc}"),
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_B;
def rmibk_alt : AVX512AIi8<opc, MRMSrcMem,
(outs _.KRC:$dst), (ins _.KRCWM:$mask, _.RC:$src1,
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
_.ScalarMemOp:$src2, u8imm:$cc),
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
!strconcat("vpcmp", Suffix,
"\t{$cc, ${src2}", _.BroadcastStr, ", $src1, $dst {${mask}}|",
"$dst {${mask}}, $src1, ${src2}", _.BroadcastStr, ", $cc}"),
[], IIC_SSE_ALU_F32P_RM>, EVEX_4V, EVEX_K, EVEX_B;
}
}
multiclass avx512_icmp_cc_vl<bits<8> opc, string Suffix, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_cc<opc, Suffix, OpNode, VTInfo.info512>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_cc<opc, Suffix, OpNode, VTInfo.info256>, EVEX_V256;
defm Z128 : avx512_icmp_cc<opc, Suffix, OpNode, VTInfo.info128>, EVEX_V128;
}
}
multiclass avx512_icmp_cc_rmb_vl<bits<8> opc, string Suffix, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_icmp_cc_rmb<opc, Suffix, OpNode, VTInfo.info512>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_icmp_cc_rmb<opc, Suffix, OpNode, VTInfo.info256>,
EVEX_V256;
defm Z128 : avx512_icmp_cc_rmb<opc, Suffix, OpNode, VTInfo.info128>,
EVEX_V128;
}
}
defm VPCMPB : avx512_icmp_cc_vl<0x3F, "b", X86cmpm, avx512vl_i8_info,
HasBWI>, EVEX_CD8<8, CD8VF>;
defm VPCMPUB : avx512_icmp_cc_vl<0x3E, "ub", X86cmpmu, avx512vl_i8_info,
HasBWI>, EVEX_CD8<8, CD8VF>;
defm VPCMPW : avx512_icmp_cc_vl<0x3F, "w", X86cmpm, avx512vl_i16_info,
HasBWI>, VEX_W, EVEX_CD8<16, CD8VF>;
defm VPCMPUW : avx512_icmp_cc_vl<0x3E, "uw", X86cmpmu, avx512vl_i16_info,
HasBWI>, VEX_W, EVEX_CD8<16, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPD : avx512_icmp_cc_rmb_vl<0x1F, "d", X86cmpm, avx512vl_i32_info,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
HasAVX512>, EVEX_CD8<32, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPUD : avx512_icmp_cc_rmb_vl<0x1E, "ud", X86cmpmu, avx512vl_i32_info,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
HasAVX512>, EVEX_CD8<32, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPQ : avx512_icmp_cc_rmb_vl<0x1F, "q", X86cmpm, avx512vl_i64_info,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
HasAVX512>, VEX_W, EVEX_CD8<64, CD8VF>;
[SKX] Deriving rmb multiclasses from general one (avx512_icmp_packed_rmb and avx512_icmp_cc_rmb). Thanks Adam Nemet for notice about this. llvm-svn: 218051
2014-09-18 22:06:55 +08:00
defm VPCMPUQ : avx512_icmp_cc_rmb_vl<0x1E, "uq", X86cmpmu, avx512vl_i64_info,
[SKX] Added new versions of cmp instructions in avx512_icmp_cc multiclass, added VL multiclass. Added encoding tests llvm-svn: 216532
2014-08-27 17:34:37 +08:00
HasAVX512>, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[X86] AVX512: Fix asm syntax for packed vcmp The *_alt defs for vcmp are used by the InstParser (the asm string in the main def is used by the InstPrinter) . The former was accepting vector registers as destination rather than mask registers. llvm-svn: 211750
2014-06-26 08:21:12 +08:00
// avx512_cmp_packed - compare packed instructions
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_cmp_packed<RegisterClass KRC, RegisterClass RC,
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
X86MemOperand x86memop, ValueType vt,
string suffix, Domain d> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rri : AVX512PIi8<0xC2, MRMSrcReg,
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(outs KRC:$dst), (ins RC:$src1, RC:$src2, AVXCC:$cc),
!strconcat("vcmp${cc}", suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[x86] Prevent instruction selection of AVX512 cmp.ps/pd/ss/sd intrinsics with illegal immediates. Correctly this time. I did the wrong patterns the first time. llvm-svn: 224891
2014-12-28 04:08:45 +08:00
[(set KRC:$dst, (X86cmpm (vt RC:$src1), (vt RC:$src2), imm:$cc))], d>;
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let hasSideEffects = 0 in
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def rrib: AVX512PIi8<0xC2, MRMSrcReg,
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
(outs KRC:$dst), (ins RC:$src1, RC:$src2, AVXCC:$cc),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
!strconcat("vcmp${cc}", suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{{sae}, $src2, $src1, $dst|$dst, $src1, $src2, {sae}}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[], d>, EVEX_B;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rmi : AVX512PIi8<0xC2, MRMSrcMem,
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(outs KRC:$dst), (ins RC:$src1, x86memop:$src2, AVXCC:$cc),
AVX-512: added VPERM2D VPERM2Q VPERM2PS VPERM2PD instructions, they give better sequences than VPERMI llvm-svn: 199893
2014-01-23 22:27:26 +08:00
!strconcat("vcmp${cc}", suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2, $cc}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set KRC:$dst,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(X86cmpm (vt RC:$src1), (load addr:$src2), imm:$cc))], d>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Accept explicit immediate argument form instead of comparison code.
Mark x86 _alt instructions as AsmParserOnly so they will be omitted from disassembler without string matches. llvm-svn: 198545
2014-01-05 12:55:55 +08:00
let isAsmParserOnly = 1, hasSideEffects = 0 in {
Use AVX512PIi8 for the alt forms of vcmp instructions. This adds the TB prefix and keeps the mnemonic from starting with an extra 'v' llvm-svn: 192272
2013-10-09 12:24:38 +08:00
def rri_alt : AVX512PIi8<0xC2, MRMSrcReg,
[x86] Add assembly parser bounds checking to the immediate value for cmpss/cmpsd/cmpps/cmppd. llvm-svn: 226642
2015-01-21 14:07:53 +08:00
(outs KRC:$dst), (ins RC:$src1, RC:$src2, u8imm:$cc),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
!strconcat("vcmp", suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$cc, $src2, $src1, $dst|$dst, $src1, $src2, $cc}"), [], d>;
[x86] Add some mayLoad/hasSideEffects flags. Remove one that was already covered by a pattern. llvm-svn: 226562
2015-01-20 20:15:30 +08:00
let mayLoad = 1 in
Use AVX512PIi8 for the alt forms of vcmp instructions. This adds the TB prefix and keeps the mnemonic from starting with an extra 'v' llvm-svn: 192272
2013-10-09 12:24:38 +08:00
def rmi_alt : AVX512PIi8<0xC2, MRMSrcMem,
[x86] Add assembly parser bounds checking to the immediate value for cmpss/cmpsd/cmpps/cmppd. llvm-svn: 226642
2015-01-21 14:07:53 +08:00
(outs KRC:$dst), (ins RC:$src1, x86memop:$src2, u8imm:$cc),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
!strconcat("vcmp", suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$cc, $src2, $src1, $dst|$dst, $src1, $src2, $cc}"), [], d>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
defm VCMPPSZ : avx512_cmp_packed<VK16, VR512, f512mem, v16f32,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
"ps", SSEPackedSingle>, PS, EVEX_4V, EVEX_V512,
Simplify some x86 format classes and remove some ambiguities in their application. llvm-svn: 200608
2014-02-01 16:17:56 +08:00
EVEX_CD8<32, CD8VF>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
defm VCMPPDZ : avx512_cmp_packed<VK8, VR512, f512mem, v8f64,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
"pd", SSEPackedDouble>, PD, EVEX_4V, VEX_W, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>;
def : Pat<(v8i1 (X86cmpm (v8f32 VR256X:$src1), (v8f32 VR256X:$src2), imm:$cc)),
(COPY_TO_REGCLASS (VCMPPSZrri
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
imm:$cc), VK8)>;
def : Pat<(v8i1 (X86cmpm (v8i32 VR256X:$src1), (v8i32 VR256X:$src2), imm:$cc)),
(COPY_TO_REGCLASS (VPCMPDZrri
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
imm:$cc), VK8)>;
def : Pat<(v8i1 (X86cmpmu (v8i32 VR256X:$src1), (v8i32 VR256X:$src2), imm:$cc)),
(COPY_TO_REGCLASS (VPCMPUDZrri
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)),
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src2, sub_ymm)),
imm:$cc), VK8)>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def : Pat<(i16 (int_x86_avx512_mask_cmp_ps_512 (v16f32 VR512:$src1),
[x86] Change AVX512 intrinsics to take a 8-bit immediate for the comparision kind instead of a 32-bit immediate. This better aligns with the emitted instruction. It also matches SSE and AVX1 equivalents. Also add auto upgrade support. llvm-svn: 226430
2015-01-19 14:07:27 +08:00
(v16f32 VR512:$src2), i8immZExt5:$cc, (i16 -1),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
FROUND_NO_EXC)),
(COPY_TO_REGCLASS (VCMPPSZrrib VR512:$src1, VR512:$src2,
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
(I8Imm imm:$cc)), GR16)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def : Pat<(i8 (int_x86_avx512_mask_cmp_pd_512 (v8f64 VR512:$src1),
[x86] Change AVX512 intrinsics to take a 8-bit immediate for the comparision kind instead of a 32-bit immediate. This better aligns with the emitted instruction. It also matches SSE and AVX1 equivalents. Also add auto upgrade support. llvm-svn: 226430
2015-01-19 14:07:27 +08:00
(v8f64 VR512:$src2), i8immZExt5:$cc, (i8 -1),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
FROUND_NO_EXC)),
(COPY_TO_REGCLASS (VCMPPDZrrib VR512:$src1, VR512:$src2,
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
(I8Imm imm:$cc)), GR8)>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def : Pat<(i16 (int_x86_avx512_mask_cmp_ps_512 (v16f32 VR512:$src1),
[x86] Change AVX512 intrinsics to take a 8-bit immediate for the comparision kind instead of a 32-bit immediate. This better aligns with the emitted instruction. It also matches SSE and AVX1 equivalents. Also add auto upgrade support. llvm-svn: 226430
2015-01-19 14:07:27 +08:00
(v16f32 VR512:$src2), i8immZExt5:$cc, (i16 -1),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
FROUND_CURRENT)),
(COPY_TO_REGCLASS (VCMPPSZrri VR512:$src1, VR512:$src2,
(I8Imm imm:$cc)), GR16)>;
def : Pat<(i8 (int_x86_avx512_mask_cmp_pd_512 (v8f64 VR512:$src1),
[x86] Change AVX512 intrinsics to take a 8-bit immediate for the comparision kind instead of a 32-bit immediate. This better aligns with the emitted instruction. It also matches SSE and AVX1 equivalents. Also add auto upgrade support. llvm-svn: 226430
2015-01-19 14:07:27 +08:00
(v8f64 VR512:$src2), i8immZExt5:$cc, (i8 -1),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
FROUND_CURRENT)),
(COPY_TO_REGCLASS (VCMPPDZrri VR512:$src1, VR512:$src2,
(I8Imm imm:$cc)), GR8)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask register copy, including
// - copy between mask registers
// - load/store mask registers
// - copy from GPR to mask register and vice versa
//
multiclass avx512_mask_mov<bits<8> opc_kk, bits<8> opc_km, bits<8> opc_mk,
string OpcodeStr, RegisterClass KRC,
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
ValueType vvt, ValueType ivt, X86MemOperand x86memop> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def kk : I<opc_kk, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), []>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in
def km : I<opc_km, MRMSrcMem, (outs KRC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
[(set KRC:$dst, (vvt (bitconvert (ivt (load addr:$src)))))]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayStore = 1 in
def mk : I<opc_mk, MRMDestMem, (outs), (ins x86memop:$dst, KRC:$src),
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[(store KRC:$src, addr:$dst)]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
multiclass avx512_mask_mov_gpr<bits<8> opc_kr, bits<8> opc_rk,
string OpcodeStr,
RegisterClass KRC, RegisterClass GRC> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def kr : I<opc_kr, MRMSrcReg, (outs KRC:$dst), (ins GRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), []>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rk : I<opc_rk, MRMSrcReg, (outs GRC:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), []>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
}
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasDQI] in
defm KMOVB : avx512_mask_mov<0x90, 0x90, 0x91, "kmovb", VK8, v8i1, i8,
i8mem>,
avx512_mask_mov_gpr<0x92, 0x93, "kmovb", VK8, GR32>,
VEX, PD;
let Predicates = [HasAVX512] in
defm KMOVW : avx512_mask_mov<0x90, 0x90, 0x91, "kmovw", VK16, v16i1, i16,
i16mem>,
avx512_mask_mov_gpr<0x92, 0x93, "kmovw", VK16, GR32>,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
VEX, PS;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasBWI] in {
defm KMOVD : avx512_mask_mov<0x90, 0x90, 0x91, "kmovd", VK32, v32i1, i32,
i32mem>, VEX, PD, VEX_W;
defm KMOVD : avx512_mask_mov_gpr<0x92, 0x93, "kmovd", VK32, GR32>,
VEX, XD;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasBWI] in {
defm KMOVQ : avx512_mask_mov<0x90, 0x90, 0x91, "kmovq", VK64, v64i1, i64,
i64mem>, VEX, PS, VEX_W;
defm KMOVQ : avx512_mask_mov_gpr<0x92, 0x93, "kmovq", VK64, GR64>,
VEX, XD, VEX_W;
}
// GR from/to mask register
let Predicates = [HasDQI] in {
def : Pat<(v8i1 (bitconvert (i8 GR8:$src))),
(KMOVBkr (SUBREG_TO_REG (i32 0), GR8:$src, sub_8bit))>;
def : Pat<(i8 (bitconvert (v8i1 VK8:$src))),
(EXTRACT_SUBREG (KMOVBrk VK8:$src), sub_8bit)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(v16i1 (bitconvert (i16 GR16:$src))),
(KMOVWkr (SUBREG_TO_REG (i32 0), GR16:$src, sub_16bit))>;
def : Pat<(i16 (bitconvert (v16i1 VK16:$src))),
(EXTRACT_SUBREG (KMOVWrk VK16:$src), sub_16bit)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
let Predicates = [HasBWI] in {
def : Pat<(v32i1 (bitconvert (i32 GR32:$src))), (KMOVDkr GR32:$src)>;
def : Pat<(i32 (bitconvert (v32i1 VK32:$src))), (KMOVDrk VK32:$src)>;
}
let Predicates = [HasBWI] in {
def : Pat<(v64i1 (bitconvert (i64 GR64:$src))), (KMOVQkr GR64:$src)>;
def : Pat<(i64 (bitconvert (v64i1 VK64:$src))), (KMOVQrk VK64:$src)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
// Load/store kreg
let Predicates = [HasDQI] in {
def : Pat<(store (i8 (bitconvert (v8i1 VK8:$src))), addr:$dst),
(KMOVBmk addr:$dst, VK8:$src)>;
}
let Predicates = [HasAVX512] in {
def : Pat<(store (i16 (bitconvert (v16i1 VK16:$src))), addr:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(KMOVWmk addr:$dst, VK16:$src)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
def : Pat<(store (i8 (bitconvert (v8i1 VK8:$src))), addr:$dst),
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
(KMOVWmk addr:$dst, (COPY_TO_REGCLASS VK8:$src, VK16))>;
def : Pat<(i1 (load addr:$src)),
(COPY_TO_REGCLASS (KMOVWkm addr:$src), VK1)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
def : Pat<(v8i1 (bitconvert (i8 (load addr:$src)))),
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
(COPY_TO_REGCLASS (KMOVWkm addr:$src), VK8)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
let Predicates = [HasBWI] in {
def : Pat<(store (i32 (bitconvert (v32i1 VK32:$src))), addr:$dst),
(KMOVDmk addr:$dst, VK32:$src)>;
}
let Predicates = [HasBWI] in {
def : Pat<(store (i64 (bitconvert (v64i1 VK64:$src))), addr:$dst),
(KMOVQmk addr:$dst, VK64:$src)>;
}
AVX-512: Added implementation of CONCAT_VECTORS for v8i1 vectors (by Alexey Bader). Added implementation of "truncate" from integer type (i64/i32/i16/i8) to i1. llvm-svn: 197482
2013-12-17 16:33:15 +08:00
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasAVX512] in {
AVX-512: Fixed a bug in emitting compare for MVT:i1 type. Added a test. llvm-svn: 215889
2014-08-18 19:59:06 +08:00
def : Pat<(i1 (trunc (i64 GR64:$src))),
(COPY_TO_REGCLASS (KMOVWkr (AND32ri (EXTRACT_SUBREG $src, sub_32bit),
(i32 1))), VK1)>;
AVX-512: fixed some patterns for MVT::i1 llvm-svn: 197981
2013-12-24 22:24:07 +08:00
def : Pat<(i1 (trunc (i32 GR32:$src))),
AVX-512: Assembly parsing of broadcast semantic in AVX-512; imlemented by Nis Zinovy (zinovy.y.nis@intel.com) Fixed truncate i32 to i1; a test will be provided in the next commit. llvm-svn: 201757
2014-02-20 14:34:39 +08:00
(COPY_TO_REGCLASS (KMOVWkr (AND32ri $src, (i32 1))), VK1)>;
AVX-512: fixed some patterns for MVT::i1 llvm-svn: 197981
2013-12-24 22:24:07 +08:00
def : Pat<(i1 (trunc (i8 GR8:$src))),
AVX-512: Assembly parsing of broadcast semantic in AVX-512; imlemented by Nis Zinovy (zinovy.y.nis@intel.com) Fixed truncate i32 to i1; a test will be provided in the next commit. llvm-svn: 201757
2014-02-20 14:34:39 +08:00
(COPY_TO_REGCLASS
(KMOVWkr (AND32ri (SUBREG_TO_REG (i32 0), GR8:$src, sub_8bit), (i32 1))),
VK1)>;
def : Pat<(i1 (trunc (i16 GR16:$src))),
(COPY_TO_REGCLASS
(KMOVWkr (AND32ri (SUBREG_TO_REG (i32 0), $src, sub_16bit), (i32 1))),
VK1)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
def : Pat<(i32 (zext VK1:$src)),
(AND32ri (KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)), (i32 1))>;
AVX-512: fixed some patterns for MVT::i1 llvm-svn: 197981
2013-12-24 22:24:07 +08:00
def : Pat<(i8 (zext VK1:$src)),
(EXTRACT_SUBREG
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
(AND32ri (KMOVWrk
(COPY_TO_REGCLASS VK1:$src, VK16)), (i32 1)), sub_8bit)>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def : Pat<(i64 (zext VK1:$src)),
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
(AND64ri8 (SUBREG_TO_REG (i64 0),
(KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)), sub_32bit), (i64 1))>;
AVX-512: implemented zext fron i1 to i16 llvm-svn: 201502
2014-02-17 15:29:33 +08:00
def : Pat<(i16 (zext VK1:$src)),
(EXTRACT_SUBREG
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
(AND32ri (KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)), (i32 1)),
sub_16bit)>;
AVX-512: insert element to mask vector; store i1 data Implemented INSERT_VECTOR_ELT operation for v16i1 and v8i1 vectors; Implemented "store" for i1 type llvm-svn: 205850
2014-04-09 20:37:50 +08:00
def : Pat<(v16i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK16)>;
def : Pat<(v8i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK8)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasBWI] in {
def : Pat<(v32i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK32)>;
def : Pat<(v64i1 (scalar_to_vector VK1:$src)),
(COPY_TO_REGCLASS VK1:$src, VK64)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// With AVX-512 only, 8-bit mask is promoted to 16-bit mask.
let Predicates = [HasAVX512] in {
// GR from/to 8-bit mask without native support
def : Pat<(v8i1 (bitconvert (i8 GR8:$src))),
(COPY_TO_REGCLASS
(KMOVWkr (SUBREG_TO_REG (i32 0), GR8:$src, sub_8bit)),
VK8)>;
def : Pat<(i8 (bitconvert (v8i1 VK8:$src))),
(EXTRACT_SUBREG
(KMOVWrk (COPY_TO_REGCLASS VK8:$src, VK16)),
sub_8bit)>;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
AVX-512: Fixed extract_vector_elt for v16i1 and v8i1 vectors. llvm-svn: 201066
2014-02-10 15:02:39 +08:00
def : Pat<(i1 (X86Vextract VK16:$src, (iPTR 0))),
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
(COPY_TO_REGCLASS VK16:$src, VK1)>;
AVX-512: Fixed extract_vector_elt for v16i1 and v8i1 vectors. llvm-svn: 201066
2014-02-10 15:02:39 +08:00
def : Pat<(i1 (X86Vextract VK8:$src, (iPTR 0))),
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
(COPY_TO_REGCLASS VK8:$src, VK1)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
let Predicates = [HasBWI] in {
def : Pat<(i1 (X86Vextract VK32:$src, (iPTR 0))),
(COPY_TO_REGCLASS VK32:$src, VK1)>;
def : Pat<(i1 (X86Vextract VK64:$src, (iPTR 0))),
(COPY_TO_REGCLASS VK64:$src, VK1)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
// Mask unary operation
// - KNOT
multiclass avx512_mask_unop<bits<8> opc, string OpcodeStr,
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
RegisterClass KRC, SDPatternOperator OpNode,
Predicate prd> {
let Predicates = [prd] in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : I<opc, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set KRC:$dst, (OpNode KRC:$src))]>;
}
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
multiclass avx512_mask_unop_all<bits<8> opc, string OpcodeStr,
SDPatternOperator OpNode> {
defm B : avx512_mask_unop<opc, !strconcat(OpcodeStr, "b"), VK8, OpNode,
HasDQI>, VEX, PD;
defm W : avx512_mask_unop<opc, !strconcat(OpcodeStr, "w"), VK16, OpNode,
HasAVX512>, VEX, PS;
defm D : avx512_mask_unop<opc, !strconcat(OpcodeStr, "d"), VK32, OpNode,
HasBWI>, VEX, PD, VEX_W;
defm Q : avx512_mask_unop<opc, !strconcat(OpcodeStr, "q"), VK64, OpNode,
HasBWI>, VEX, PS, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
defm KNOT : avx512_mask_unop_all<0x44, "knot", not>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
multiclass avx512_mask_unop_int<string IntName, string InstName> {
let Predicates = [HasAVX512] in
def : Pat<(!cast<Intrinsic>("int_x86_avx512_"##IntName##"_w")
(i16 GR16:$src)),
(COPY_TO_REGCLASS (!cast<Instruction>(InstName##"Wrr")
(v16i1 (COPY_TO_REGCLASS GR16:$src, VK16))), GR16)>;
}
defm : avx512_mask_unop_int<"knot", "KNOT">;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasDQI] in
def : Pat<(xor VK8:$src1, (v8i1 immAllOnesV)), (KNOTBrr VK8:$src1)>;
let Predicates = [HasAVX512] in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(xor VK16:$src1, (v16i1 immAllOnesV)), (KNOTWrr VK16:$src1)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
let Predicates = [HasBWI] in
def : Pat<(xor VK32:$src1, (v32i1 immAllOnesV)), (KNOTDrr VK32:$src1)>;
let Predicates = [HasBWI] in
def : Pat<(xor VK64:$src1, (v64i1 immAllOnesV)), (KNOTQrr VK64:$src1)>;
// KNL does not support KMOVB, 8-bit mask is promoted to 16-bit
let Predicates = [HasAVX512] in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(xor VK8:$src1, (v8i1 immAllOnesV)),
(COPY_TO_REGCLASS (KNOTWrr (COPY_TO_REGCLASS VK8:$src1, VK16)), VK8)>;
def : Pat<(not VK8:$src),
(COPY_TO_REGCLASS
(KNOTWrr (COPY_TO_REGCLASS VK8:$src, VK16)), VK8)>;
[SKX] Enabling mask instructions: encoding, lowering KMOVB, KMOVW, KMOVD, KMOVQ, KNOTB, KNOTW, KNOTD, KNOTQ Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 213757
2014-07-23 22:49:42 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask binary operation
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
// - KAND, KANDN, KOR, KXNOR, KXOR
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_mask_binop<bits<8> opc, string OpcodeStr,
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
RegisterClass KRC, SDPatternOperator OpNode,
Predicate prd> {
let Predicates = [prd] in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : I<opc, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src1, KRC:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set KRC:$dst, (OpNode KRC:$src1, KRC:$src2))]>;
}
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
multiclass avx512_mask_binop_all<bits<8> opc, string OpcodeStr,
SDPatternOperator OpNode> {
defm B : avx512_mask_binop<opc, !strconcat(OpcodeStr, "b"), VK8, OpNode,
HasDQI>, VEX_4V, VEX_L, PD;
defm W : avx512_mask_binop<opc, !strconcat(OpcodeStr, "w"), VK16, OpNode,
HasAVX512>, VEX_4V, VEX_L, PS;
defm D : avx512_mask_binop<opc, !strconcat(OpcodeStr, "d"), VK32, OpNode,
HasBWI>, VEX_4V, VEX_L, VEX_W, PD;
defm Q : avx512_mask_binop<opc, !strconcat(OpcodeStr, "q"), VK64, OpNode,
HasBWI>, VEX_4V, VEX_L, VEX_W, PS;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
def andn : PatFrag<(ops node:$i0, node:$i1), (and (not node:$i0), node:$i1)>;
def xnor : PatFrag<(ops node:$i0, node:$i1), (not (xor node:$i0, node:$i1))>;
let isCommutable = 1 in {
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
defm KAND : avx512_mask_binop_all<0x41, "kand", and>;
defm KOR : avx512_mask_binop_all<0x45, "kor", or>;
defm KXNOR : avx512_mask_binop_all<0x46, "kxnor", xnor>;
defm KXOR : avx512_mask_binop_all<0x47, "kxor", xor>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] Enabling mask logic instructions: encoding, lowering Instructions: KAND{BWDQ}, KANDN{BWDQ}, KOR{BWDQ}, KXOR{BWDQ}, KXNOR{BWDQ} Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214081
2014-07-28 21:46:45 +08:00
let isCommutable = 0 in
defm KANDN : avx512_mask_binop_all<0x42, "kandn", andn>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
def : Pat<(xor VK1:$src1, VK1:$src2),
(COPY_TO_REGCLASS (KXORWrr (COPY_TO_REGCLASS VK1:$src1, VK16),
(COPY_TO_REGCLASS VK1:$src2, VK16)), VK1)>;
def : Pat<(or VK1:$src1, VK1:$src2),
(COPY_TO_REGCLASS (KORWrr (COPY_TO_REGCLASS VK1:$src1, VK16),
(COPY_TO_REGCLASS VK1:$src2, VK16)), VK1)>;
AVX-512: Result type of scalar SETCC is MVT::i1 for AVX-512. llvm-svn: 198008
2013-12-25 18:06:40 +08:00
def : Pat<(and VK1:$src1, VK1:$src2),
(COPY_TO_REGCLASS (KANDWrr (COPY_TO_REGCLASS VK1:$src1, VK16),
(COPY_TO_REGCLASS VK1:$src2, VK16)), VK1)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_mask_binop_int<string IntName, string InstName> {
let Predicates = [HasAVX512] in
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
def : Pat<(!cast<Intrinsic>("int_x86_avx512_"##IntName##"_w")
(i16 GR16:$src1), (i16 GR16:$src2)),
(COPY_TO_REGCLASS (!cast<Instruction>(InstName##"Wrr")
(v16i1 (COPY_TO_REGCLASS GR16:$src1, VK16)),
(v16i1 (COPY_TO_REGCLASS GR16:$src2, VK16))), GR16)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm : avx512_mask_binop_int<"kand", "KAND">;
defm : avx512_mask_binop_int<"kandn", "KANDN">;
defm : avx512_mask_binop_int<"kor", "KOR">;
defm : avx512_mask_binop_int<"kxnor", "KXNOR">;
defm : avx512_mask_binop_int<"kxor", "KXOR">;
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// With AVX-512, 8-bit mask is promoted to 16-bit mask.
multiclass avx512_binop_pat<SDPatternOperator OpNode, Instruction Inst> {
let Predicates = [HasAVX512] in
def : Pat<(OpNode VK8:$src1, VK8:$src2),
(COPY_TO_REGCLASS
(Inst (COPY_TO_REGCLASS VK8:$src1, VK16),
(COPY_TO_REGCLASS VK8:$src2, VK16)), VK8)>;
}
defm : avx512_binop_pat<and, KANDWrr>;
defm : avx512_binop_pat<andn, KANDNWrr>;
defm : avx512_binop_pat<or, KORWrr>;
defm : avx512_binop_pat<xnor, KXNORWrr>;
defm : avx512_binop_pat<xor, KXORWrr>;
// Mask unpacking
multiclass avx512_mask_unpck<bits<8> opc, string OpcodeStr,
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
RegisterClass KRC> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
def rr : I<opc, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src1, KRC:$src2),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"), []>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
multiclass avx512_mask_unpck_bw<bits<8> opc, string OpcodeStr> {
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
defm BW : avx512_mask_unpck<opc, !strconcat(OpcodeStr, "bw"), VK16>,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
VEX_4V, VEX_L, PD;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm KUNPCK : avx512_mask_unpck_bw<0x4b, "kunpck">;
AVX-512: Added implementation of CONCAT_VECTORS for v8i1 vectors (by Alexey Bader). Added implementation of "truncate" from integer type (i64/i32/i16/i8) to i1. llvm-svn: 197482
2013-12-17 16:33:15 +08:00
def : Pat<(v16i1 (concat_vectors (v8i1 VK8:$src1), (v8i1 VK8:$src2))),
(KUNPCKBWrr (COPY_TO_REGCLASS VK8:$src2, VK16),
(COPY_TO_REGCLASS VK8:$src1, VK16))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_mask_unpck_int<string IntName, string InstName> {
let Predicates = [HasAVX512] in
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
def : Pat<(!cast<Intrinsic>("int_x86_avx512_"##IntName##"_bw")
(i16 GR16:$src1), (i16 GR16:$src2)),
(COPY_TO_REGCLASS (!cast<Instruction>(InstName##"BWrr")
(v16i1 (COPY_TO_REGCLASS GR16:$src1, VK16)),
(v16i1 (COPY_TO_REGCLASS GR16:$src2, VK16))), GR16)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: changed intrinsics for mask operations llvm-svn: 196918
2013-12-10 21:53:10 +08:00
defm : avx512_mask_unpck_int<"kunpck", "KUNPCK">;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask bit testing
multiclass avx512_mask_testop<bits<8> opc, string OpcodeStr, RegisterClass KRC,
SDNode OpNode> {
let Predicates = [HasAVX512], Defs = [EFLAGS] in
def rr : I<opc, MRMSrcReg, (outs), (ins KRC:$src1, KRC:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1|$src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set EFLAGS, (OpNode KRC:$src1, KRC:$src2))]>;
}
multiclass avx512_mask_testop_w<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm W : avx512_mask_testop<opc, !strconcat(OpcodeStr, "w"), VK16, OpNode>,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
VEX, PS;
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
let Predicates = [HasDQI] in
defm B : avx512_mask_testop<opc, !strconcat(OpcodeStr, "b"), VK8, OpNode>,
VEX, PD;
let Predicates = [HasBWI] in {
defm Q : avx512_mask_testop<opc, !strconcat(OpcodeStr, "q"), VK64, OpNode>,
VEX, PS, VEX_W;
defm D : avx512_mask_testop<opc, !strconcat(OpcodeStr, "d"), VK32, OpNode>,
VEX, PD, VEX_W;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm KORTEST : avx512_mask_testop_w<0x98, "kortest", X86kortest>;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
AVX-512: fixed some patterns for MVT::i1 llvm-svn: 197981
2013-12-24 22:24:07 +08:00
def : Pat<(X86cmp VK1:$src1, (i1 0)),
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
(KORTESTWrr (COPY_TO_REGCLASS VK1:$src1, VK16),
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
(COPY_TO_REGCLASS VK1:$src1, VK16))>, Requires<[HasAVX512, NoDQI]>;
def : Pat<(X86cmp VK1:$src1, (i1 0)),
(KORTESTBrr (COPY_TO_REGCLASS VK1:$src1, VK8),
(COPY_TO_REGCLASS VK1:$src1, VK8))>, Requires<[HasDQI]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask shift
multiclass avx512_mask_shiftop<bits<8> opc, string OpcodeStr, RegisterClass KRC,
SDNode OpNode> {
let Predicates = [HasAVX512] in
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
def ri : Ii8<opc, MRMSrcReg, (outs KRC:$dst), (ins KRC:$src, u8imm:$imm),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$imm, $src, $dst|$dst, $src, $imm}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set KRC:$dst, (OpNode KRC:$src, (i8 imm:$imm)))]>;
}
multiclass avx512_mask_shiftop_w<bits<8> opc1, bits<8> opc2, string OpcodeStr,
SDNode OpNode> {
defm W : avx512_mask_shiftop<opc1, !strconcat(OpcodeStr, "w"), VK16, OpNode>,
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
VEX, TAPD, VEX_W;
let Predicates = [HasDQI] in
defm B : avx512_mask_shiftop<opc1, !strconcat(OpcodeStr, "b"), VK8, OpNode>,
VEX, TAPD;
let Predicates = [HasBWI] in {
defm Q : avx512_mask_shiftop<opc2, !strconcat(OpcodeStr, "q"), VK64, OpNode>,
VEX, TAPD, VEX_W;
let Predicates = [HasDQI] in
defm D : avx512_mask_shiftop<opc2, !strconcat(OpcodeStr, "d"), VK32, OpNode>,
VEX, TAPD;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
defm KSHIFTL : avx512_mask_shiftop_w<0x32, 0x33, "kshiftl", X86vshli>;
defm KSHIFTR : avx512_mask_shiftop_w<0x30, 0x31, "kshiftr", X86vsrli>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Mask setting all 0s or 1s
multiclass avx512_mask_setop<RegisterClass KRC, ValueType VT, PatFrag Val> {
let Predicates = [HasAVX512] in
let isReMaterializable = 1, isAsCheapAsAMove = 1, isPseudo = 1 in
def #NAME# : I<0, Pseudo, (outs KRC:$dst), (ins), "",
[(set KRC:$dst, (VT Val))]>;
}
multiclass avx512_mask_setop_w<PatFrag Val> {
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
defm B : avx512_mask_setop<VK8, v8i1, Val>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm W : avx512_mask_setop<VK16, v16i1, Val>;
}
defm KSET0 : avx512_mask_setop_w<immAllZerosV>;
defm KSET1 : avx512_mask_setop_w<immAllOnesV>;
// With AVX-512 only, 8-bit mask is promoted to 16-bit mask.
let Predicates = [HasAVX512] in {
def : Pat<(v8i1 immAllZerosV), (COPY_TO_REGCLASS (KSET0W), VK8)>;
def : Pat<(v8i1 immAllOnesV), (COPY_TO_REGCLASS (KSET1W), VK8)>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def : Pat<(i1 0), (COPY_TO_REGCLASS (KSET0W), VK1)>;
def : Pat<(i1 1), (COPY_TO_REGCLASS (KSET1W), VK1)>;
def : Pat<(i1 -1), (COPY_TO_REGCLASS (KSET1W), VK1)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
def : Pat<(v8i1 (extract_subvector (v16i1 VK16:$src), (iPTR 0))),
(v8i1 (COPY_TO_REGCLASS VK16:$src, VK8))>;
def : Pat<(v16i1 (insert_subvector undef, (v8i1 VK8:$src), (iPTR 0))),
(v16i1 (COPY_TO_REGCLASS VK8:$src, VK16))>;
def : Pat<(v8i1 (extract_subvector (v16i1 VK16:$src), (iPTR 8))),
(v8i1 (COPY_TO_REGCLASS (KSHIFTRWri VK16:$src, (i8 8)), VK8))>;
[AVX512] Added intrinsics for 128- and 256-bit versions of VCMPEQ{BWDQ} Fixed lowering of this intrinsics in case when mask is v2i1 and v4i1. Now cmp intrinsics lower in the following way: (i8 (int_x86_avx512_mask_pcmpeq_q_128 (v2i64 %a), (v2i64 %b), (i8 %mask))) -> (i8 (bitcast (v8i1 (insert_subvector undef, (v2i1 (and (PCMPEQM %a, %b), (extract_subvector (v8i1 (bitcast %mask)), 0))), 0)))) llvm-svn: 218669
2014-09-30 19:41:54 +08:00
let Predicates = [HasVLX] in {
def : Pat<(v8i1 (insert_subvector undef, (v4i1 VK4:$src), (iPTR 0))),
(v8i1 (COPY_TO_REGCLASS VK4:$src, VK8))>;
def : Pat<(v8i1 (insert_subvector undef, (v2i1 VK2:$src), (iPTR 0))),
(v8i1 (COPY_TO_REGCLASS VK2:$src, VK8))>;
def : Pat<(v4i1 (extract_subvector (v8i1 VK8:$src), (iPTR 0))),
(v4i1 (COPY_TO_REGCLASS VK8:$src, VK4))>;
def : Pat<(v2i1 (extract_subvector (v8i1 VK8:$src), (iPTR 0))),
(v2i1 (COPY_TO_REGCLASS VK8:$src, VK2))>;
}
AVX-512: Fixed extract_vector_elt for v8i1 vector llvm-svn: 202624
2014-03-02 17:19:44 +08:00
def : Pat<(v8i1 (X86vshli VK8:$src, (i8 imm:$imm))),
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
(v8i1 (COPY_TO_REGCLASS
(KSHIFTLWri (COPY_TO_REGCLASS VK8:$src, VK16),
(I8Imm $imm)), VK8))>, Requires<[HasAVX512, NoDQI]>;
AVX-512: Fixed extract_vector_elt for v8i1 vector llvm-svn: 202624
2014-03-02 17:19:44 +08:00
def : Pat<(v8i1 (X86vsrli VK8:$src, (i8 imm:$imm))),
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
(v8i1 (COPY_TO_REGCLASS
(KSHIFTRWri (COPY_TO_REGCLASS VK8:$src, VK16),
(I8Imm $imm)), VK8))>, Requires<[HasAVX512, NoDQI]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Aligned and unaligned load and store
//
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
multiclass avx512_load<bits<8> opc, string OpcodeStr, PatFrag ld_frag,
RegisterClass KRC, RegisterClass RC,
ValueType vt, ValueType zvt, X86MemOperand memop,
Domain d, bit IsReMaterializable = 1> {
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
let hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : AVX512PI<opc, MRMSrcReg, (outs RC:$dst), (ins RC:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), [],
d>, EVEX;
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
def rrkz : AVX512PI<opc, MRMSrcReg, (outs RC:$dst), (ins KRC:$mask, RC:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr, "\t{$src, ${dst} {${mask}} {z}|",
"${dst} {${mask}} {z}, $src}"), [], d>, EVEX, EVEX_KZ;
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
}
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
let canFoldAsLoad = 1, isReMaterializable = IsReMaterializable,
SchedRW = [WriteLoad] in
def rm : AVX512PI<opc, MRMSrcMem, (outs RC:$dst), (ins memop:$src),
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[(set RC:$dst, (vt (bitconvert (ld_frag addr:$src))))],
d>, EVEX;
let AddedComplexity = 20 in {
let Constraints = "$src0 = $dst", hasSideEffects = 0 in {
let hasSideEffects = 0 in
def rrk : AVX512PI<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src0, KRC:$mask, RC:$src1),
!strconcat(OpcodeStr, "\t{$src1, ${dst} {${mask}}|",
"${dst} {${mask}}, $src1}"),
[(set RC:$dst, (vt (vselect KRC:$mask,
(vt RC:$src1),
(vt RC:$src0))))],
d>, EVEX, EVEX_K;
let mayLoad = 1, SchedRW = [WriteLoad] in
def rmk : AVX512PI<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src0, KRC:$mask, memop:$src1),
!strconcat(OpcodeStr, "\t{$src1, ${dst} {${mask}}|",
"${dst} {${mask}}, $src1}"),
[(set RC:$dst, (vt
(vselect KRC:$mask,
(vt (bitconvert (ld_frag addr:$src1))),
(vt RC:$src0))))],
d>, EVEX, EVEX_K;
}
let mayLoad = 1, SchedRW = [WriteLoad] in
def rmkz : AVX512PI<opc, MRMSrcMem, (outs RC:$dst),
(ins KRC:$mask, memop:$src),
!strconcat(OpcodeStr, "\t{$src, ${dst} {${mask}} {z}|",
"${dst} {${mask}} {z}, $src}"),
[(set RC:$dst, (vt
(vselect KRC:$mask,
(vt (bitconvert (ld_frag addr:$src))),
(vt (bitconvert (zvt immAllZerosV))))))],
d>, EVEX, EVEX_KZ;
}
}
multiclass avx512_load_vl<bits<8> opc, string OpcodeStr, string ld_pat,
string elty, string elsz, string vsz512,
string vsz256, string vsz128, Domain d,
Predicate prd, bit IsReMaterializable = 1> {
let Predicates = [prd] in
defm Z : avx512_load<opc, OpcodeStr,
!cast<PatFrag>(ld_pat##"v"##vsz512##elty##elsz),
!cast<RegisterClass>("VK"##vsz512##"WM"), VR512,
!cast<ValueType>("v"##vsz512##elty##elsz), v16i32,
!cast<X86MemOperand>(elty##"512mem"), d,
IsReMaterializable>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_load<opc, OpcodeStr,
!cast<PatFrag>(ld_pat##!if(!eq(elty,"f"),
"v"##vsz256##elty##elsz, "v4i64")),
!cast<RegisterClass>("VK"##vsz256##"WM"), VR256X,
!cast<ValueType>("v"##vsz256##elty##elsz), v8i32,
!cast<X86MemOperand>(elty##"256mem"), d,
IsReMaterializable>, EVEX_V256;
defm Z128 : avx512_load<opc, OpcodeStr,
!cast<PatFrag>(ld_pat##!if(!eq(elty,"f"),
"v"##vsz128##elty##elsz, "v2i64")),
!cast<RegisterClass>("VK"##vsz128##"WM"), VR128X,
!cast<ValueType>("v"##vsz128##elty##elsz), v4i32,
!cast<X86MemOperand>(elty##"128mem"), d,
IsReMaterializable>, EVEX_V128;
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
multiclass avx512_store<bits<8> opc, string OpcodeStr, PatFrag st_frag,
ValueType OpVT, RegisterClass KRC, RegisterClass RC,
X86MemOperand memop, Domain d> {
Hide some redundant AVX512 instructions from the asm parser, but force them to show up in the disassembler. llvm-svn: 226155
2015-01-15 17:37:15 +08:00
let isCodeGenOnly = 1, ForceDisassemble = 1, hasSideEffects = 0 in {
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
def rr_alt : AVX512PI<opc, MRMDestReg, (outs RC:$dst), (ins RC:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"), [], d>,
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
EVEX;
let Constraints = "$src1 = $dst" in
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
def rrk_alt : AVX512PI<opc, MRMDestReg, (outs RC:$dst),
(ins RC:$src1, KRC:$mask, RC:$src2),
!strconcat(OpcodeStr,
"\t{$src2, ${dst} {${mask}}|${dst} {${mask}}, $src2}"), [], d>,
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
EVEX, EVEX_K;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
def rrkz_alt : AVX512PI<opc, MRMDestReg, (outs RC:$dst),
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
(ins KRC:$mask, RC:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
!strconcat(OpcodeStr,
"\t{$src, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src}"),
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
[], d>, EVEX, EVEX_KZ;
}
let mayStore = 1 in {
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
def mr : AVX512PI<opc, MRMDestMem, (outs), (ins memop:$dst, RC:$src),
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[(st_frag (OpVT RC:$src), addr:$dst)], d>, EVEX;
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
def mrk : AVX512PI<opc, MRMDestMem, (outs),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(ins memop:$dst, KRC:$mask, RC:$src),
!strconcat(OpcodeStr,
"\t{$src, ${dst} {${mask}}|${dst} {${mask}}, $src}"),
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
[], d>, EVEX, EVEX_K;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
multiclass avx512_store_vl<bits<8> opc, string OpcodeStr, string st_pat,
string st_suff_512, string st_suff_256,
string st_suff_128, string elty, string elsz,
string vsz512, string vsz256, string vsz128,
Domain d, Predicate prd> {
let Predicates = [prd] in
defm Z : avx512_store<opc, OpcodeStr, !cast<PatFrag>(st_pat##st_suff_512),
!cast<ValueType>("v"##vsz512##elty##elsz),
!cast<RegisterClass>("VK"##vsz512##"WM"), VR512,
!cast<X86MemOperand>(elty##"512mem"), d>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_store<opc, OpcodeStr, !cast<PatFrag>(st_pat##st_suff_256),
!cast<ValueType>("v"##vsz256##elty##elsz),
!cast<RegisterClass>("VK"##vsz256##"WM"), VR256X,
!cast<X86MemOperand>(elty##"256mem"), d>, EVEX_V256;
defm Z128 : avx512_store<opc, OpcodeStr, !cast<PatFrag>(st_pat##st_suff_128),
!cast<ValueType>("v"##vsz128##elty##elsz),
!cast<RegisterClass>("VK"##vsz128##"WM"), VR128X,
!cast<X86MemOperand>(elty##"128mem"), d>, EVEX_V128;
}
}
defm VMOVAPS : avx512_load_vl<0x28, "vmovaps", "alignedload", "f", "32",
"16", "8", "4", SSEPackedSingle, HasAVX512>,
avx512_store_vl<0x29, "vmovaps", "alignedstore",
"512", "256", "", "f", "32", "16", "8", "4",
SSEPackedSingle, HasAVX512>,
PS, EVEX_CD8<32, CD8VF>;
defm VMOVAPD : avx512_load_vl<0x28, "vmovapd", "alignedload", "f", "64",
"8", "4", "2", SSEPackedDouble, HasAVX512>,
avx512_store_vl<0x29, "vmovapd", "alignedstore",
"512", "256", "", "f", "64", "8", "4", "2",
SSEPackedDouble, HasAVX512>,
PD, VEX_W, EVEX_CD8<64, CD8VF>;
defm VMOVUPS : avx512_load_vl<0x10, "vmovups", "load", "f", "32",
"16", "8", "4", SSEPackedSingle, HasAVX512>,
avx512_store_vl<0x11, "vmovups", "store", "", "", "", "f", "32",
"16", "8", "4", SSEPackedSingle, HasAVX512>,
PS, EVEX_CD8<32, CD8VF>;
defm VMOVUPD : avx512_load_vl<0x10, "vmovupd", "load", "f", "64",
"8", "4", "2", SSEPackedDouble, HasAVX512, 0>,
avx512_store_vl<0x11, "vmovupd", "store", "", "", "", "f", "64",
"8", "4", "2", SSEPackedDouble, HasAVX512>,
PD, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
def: Pat<(v8f64 (int_x86_avx512_mask_loadu_pd_512 addr:$ptr,
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(bc_v8f64 (v16i32 immAllZerosV)), GR8:$mask)),
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
(VMOVUPDZrmkz (v8i1 (COPY_TO_REGCLASS GR8:$mask, VK8WM)), addr:$ptr)>;
def: Pat<(v16f32 (int_x86_avx512_mask_loadu_ps_512 addr:$ptr,
(bc_v16f32 (v16i32 immAllZerosV)), GR16:$mask)),
(VMOVUPSZrmkz (v16i1 (COPY_TO_REGCLASS GR16:$mask, VK16WM)), addr:$ptr)>;
[AVX512] Add intrinsics for masked aligned FP loads and stores Similar to the unaligned cases. Test was generated with update_llc_test_checks.py. Part of <rdar://problem/17688758> llvm-svn: 226296
2015-01-17 02:50:09 +08:00
def: Pat<(v8f64 (int_x86_avx512_mask_load_pd_512 addr:$ptr,
(bc_v8f64 (v16i32 immAllZerosV)), GR8:$mask)),
(VMOVAPDZrmkz (v8i1 (COPY_TO_REGCLASS GR8:$mask, VK8WM)), addr:$ptr)>;
def: Pat<(v16f32 (int_x86_avx512_mask_load_ps_512 addr:$ptr,
(bc_v16f32 (v16i32 immAllZerosV)), GR16:$mask)),
(VMOVAPSZrmkz (v16i1 (COPY_TO_REGCLASS GR16:$mask, VK16WM)), addr:$ptr)>;
def: Pat<(v8f64 (int_x86_avx512_mask_load_pd_512 addr:$ptr,
(bc_v8f64 (v16i32 immAllZerosV)), (i8 -1))),
(VMOVAPDZrm addr:$ptr)>;
def: Pat<(v16f32 (int_x86_avx512_mask_load_ps_512 addr:$ptr,
(bc_v16f32 (v16i32 immAllZerosV)), (i16 -1))),
(VMOVAPSZrm addr:$ptr)>;
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
def: Pat<(int_x86_avx512_mask_storeu_ps_512 addr:$ptr, (v16f32 VR512:$src),
GR16:$mask),
(VMOVUPSZmrk addr:$ptr, (v16i1 (COPY_TO_REGCLASS GR16:$mask, VK16WM)),
VR512:$src)>;
def: Pat<(int_x86_avx512_mask_storeu_pd_512 addr:$ptr, (v8f64 VR512:$src),
GR8:$mask),
(VMOVUPDZmrk addr:$ptr, (v8i1 (COPY_TO_REGCLASS GR8:$mask, VK8WM)),
[AVX512] Add intrinsics for masked aligned FP loads and stores Similar to the unaligned cases. Test was generated with update_llc_test_checks.py. Part of <rdar://problem/17688758> llvm-svn: 226296
2015-01-17 02:50:09 +08:00
VR512:$src)>;
def: Pat<(int_x86_avx512_mask_store_ps_512 addr:$ptr, (v16f32 VR512:$src),
GR16:$mask),
(VMOVAPSZmrk addr:$ptr, (v16i1 (COPY_TO_REGCLASS GR16:$mask, VK16WM)),
VR512:$src)>;
def: Pat<(int_x86_avx512_mask_store_pd_512 addr:$ptr, (v8f64 VR512:$src),
GR8:$mask),
(VMOVAPDZmrk addr:$ptr, (v8i1 (COPY_TO_REGCLASS GR8:$mask, VK8WM)),
AVX-512: masked load/store + intrinsics for them. llvm-svn: 203790
2014-03-13 20:05:52 +08:00
VR512:$src)>;
Masked Load / Store Intrinsics - the CodeGen part. I'm recommiting the codegen part of the patch. The vectorizer part will be send to review again. Masked Vector Load and Store Intrinsics. Introduced new target-independent intrinsics in order to support masked vector loads and stores. The loop vectorizer optimizes loops containing conditional memory accesses by generating these intrinsics for existing targets AVX2 and AVX-512. The vectorizer asks the target about availability of masked vector loads and stores. Added SDNodes for masked operations and lowering patterns for X86 code generator. Examples: <16 x i32> @llvm.masked.load.v16i32(i8* %addr, <16 x i32> %passthru, i32 4 /* align */, <16 x i1> %mask) declare void @llvm.masked.store.v8f64(i8* %addr, <8 x double> %value, i32 4, <8 x i1> %mask) Scalarizer for other targets (not AVX2/AVX-512) will be done in a separate patch. http://reviews.llvm.org/D6191 llvm-svn: 223348
2014-12-04 17:40:44 +08:00
def: Pat<(masked_store addr:$ptr, VK8WM:$mask, (v8f32 VR256:$src)),
(VMOVUPSZmrk addr:$ptr,
(v16i1 (COPY_TO_REGCLASS VK8WM:$mask, VK16WM)),
(INSERT_SUBREG (v16f32 (IMPLICIT_DEF)), VR256:$src, sub_ymm))>;
def: Pat<(v8f32 (masked_load addr:$ptr, VK8WM:$mask, undef)),
(v8f32 (EXTRACT_SUBREG (v16f32 (VMOVUPSZrmkz
(v16i1 (COPY_TO_REGCLASS VK8WM:$mask, VK16WM)), addr:$ptr)), sub_ymm))>;
def: Pat<(masked_store addr:$ptr, VK16WM:$mask, (v16f32 VR512:$src)),
(VMOVUPSZmrk addr:$ptr, VK16WM:$mask, VR512:$src)>;
def: Pat<(masked_store addr:$ptr, VK8WM:$mask, (v8f64 VR512:$src)),
(VMOVUPDZmrk addr:$ptr, VK8WM:$mask, VR512:$src)>;
def: Pat<(v16f32 (masked_load addr:$ptr, VK16WM:$mask, undef)),
(VMOVUPSZrmkz VK16WM:$mask, addr:$ptr)>;
def: Pat<(v16f32 (masked_load addr:$ptr, VK16WM:$mask,
(bc_v16f32 (v16i32 immAllZerosV)))),
(VMOVUPSZrmkz VK16WM:$mask, addr:$ptr)>;
def: Pat<(v16f32 (masked_load addr:$ptr, VK16WM:$mask, (v16f32 VR512:$src0))),
(VMOVUPSZrmk VR512:$src0, VK16WM:$mask, addr:$ptr)>;
def: Pat<(v8f64 (masked_load addr:$ptr, VK8WM:$mask, undef)),
(VMOVUPDZrmkz VK8WM:$mask, addr:$ptr)>;
def: Pat<(v8f64 (masked_load addr:$ptr, VK8WM:$mask,
(bc_v8f64 (v16i32 immAllZerosV)))),
(VMOVUPDZrmkz VK8WM:$mask, addr:$ptr)>;
def: Pat<(v8f64 (masked_load addr:$ptr, VK8WM:$mask, (v8f64 VR512:$src0))),
(VMOVUPDZrmk VR512:$src0, VK8WM:$mask, addr:$ptr)>;
Masked load and store codegen - fixed 128-bit vectors The codegen failed on 128-bit types on AVX2. I added patterns and in td files and tests. llvm-svn: 224647
2014-12-20 07:27:57 +08:00
def: Pat<(v8f32 (masked_load addr:$ptr, VK8WM:$mask, (v8f32 VR256:$src0))),
(v8f32 (EXTRACT_SUBREG (v16f32 (VMOVUPSZrmk
(INSERT_SUBREG (v16f32 (IMPLICIT_DEF)), VR256:$src0, sub_ymm),
(v16i1 (COPY_TO_REGCLASS VK8WM:$mask, VK16WM)), addr:$ptr)), sub_ymm))>;
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
defm VMOVDQA32 : avx512_load_vl<0x6F, "vmovdqa32", "alignedload", "i", "32",
"16", "8", "4", SSEPackedInt, HasAVX512>,
avx512_store_vl<0x7F, "vmovdqa32", "alignedstore",
"512", "256", "", "i", "32", "16", "8", "4",
SSEPackedInt, HasAVX512>,
PD, EVEX_CD8<32, CD8VF>;
defm VMOVDQA64 : avx512_load_vl<0x6F, "vmovdqa64", "alignedload", "i", "64",
"8", "4", "2", SSEPackedInt, HasAVX512>,
avx512_store_vl<0x7F, "vmovdqa64", "alignedstore",
"512", "256", "", "i", "64", "8", "4", "2",
SSEPackedInt, HasAVX512>,
PD, VEX_W, EVEX_CD8<64, CD8VF>;
defm VMOVDQU8 : avx512_load_vl<0x6F, "vmovdqu8", "load", "i", "8",
"64", "32", "16", SSEPackedInt, HasBWI>,
avx512_store_vl<0x7F, "vmovdqu8", "store", "", "", "",
"i", "8", "64", "32", "16", SSEPackedInt,
HasBWI>, XD, EVEX_CD8<8, CD8VF>;
defm VMOVDQU16 : avx512_load_vl<0x6F, "vmovdqu16", "load", "i", "16",
"32", "16", "8", SSEPackedInt, HasBWI>,
avx512_store_vl<0x7F, "vmovdqu16", "store", "", "", "",
"i", "16", "32", "16", "8", SSEPackedInt,
HasBWI>, XD, VEX_W, EVEX_CD8<16, CD8VF>;
defm VMOVDQU32 : avx512_load_vl<0x6F, "vmovdqu32", "load", "i", "32",
"16", "8", "4", SSEPackedInt, HasAVX512>,
avx512_store_vl<0x7F, "vmovdqu32", "store", "", "", "",
"i", "32", "16", "8", "4", SSEPackedInt,
HasAVX512>, XS, EVEX_CD8<32, CD8VF>;
defm VMOVDQU64 : avx512_load_vl<0x6F, "vmovdqu64", "load", "i", "64",
"8", "4", "2", SSEPackedInt, HasAVX512>,
avx512_store_vl<0x7F, "vmovdqu64", "store", "", "", "",
"i", "64", "8", "4", "2", SSEPackedInt,
HasAVX512>, XS, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: aligned / unaligned load and store for 512-bit integer vectors. llvm-svn: 193156
2013-10-22 17:19:28 +08:00
AVX-512: insert element to mask vector; store i1 data Implemented INSERT_VECTOR_ELT operation for v16i1 and v8i1 vectors; Implemented "store" for i1 type llvm-svn: 205850
2014-04-09 20:37:50 +08:00
def: Pat<(v16i32 (int_x86_avx512_mask_loadu_d_512 addr:$ptr,
(v16i32 immAllZerosV), GR16:$mask)),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(VMOVDQU32Zrmkz (v16i1 (COPY_TO_REGCLASS GR16:$mask, VK16WM)), addr:$ptr)>;
AVX-512: insert element to mask vector; store i1 data Implemented INSERT_VECTOR_ELT operation for v16i1 and v8i1 vectors; Implemented "store" for i1 type llvm-svn: 205850
2014-04-09 20:37:50 +08:00
def: Pat<(v8i64 (int_x86_avx512_mask_loadu_q_512 addr:$ptr,
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(bc_v8i64 (v16i32 immAllZerosV)), GR8:$mask)),
(VMOVDQU64Zrmkz (v8i1 (COPY_TO_REGCLASS GR8:$mask, VK8WM)), addr:$ptr)>;
AVX-512: insert element to mask vector; store i1 data Implemented INSERT_VECTOR_ELT operation for v16i1 and v8i1 vectors; Implemented "store" for i1 type llvm-svn: 205850
2014-04-09 20:37:50 +08:00
AVX-512: minor change in rndscale intrinsic llvm-svn: 207937
2014-05-04 21:35:37 +08:00
def: Pat<(int_x86_avx512_mask_storeu_d_512 addr:$ptr, (v16i32 VR512:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
GR16:$mask),
(VMOVDQU32Zmrk addr:$ptr, (v16i1 (COPY_TO_REGCLASS GR16:$mask, VK16WM)),
AVX-512: minor change in rndscale intrinsic llvm-svn: 207937
2014-05-04 21:35:37 +08:00
VR512:$src)>;
def: Pat<(int_x86_avx512_mask_storeu_q_512 addr:$ptr, (v8i64 VR512:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
GR8:$mask),
(VMOVDQU64Zmrk addr:$ptr, (v8i1 (COPY_TO_REGCLASS GR8:$mask, VK8WM)),
AVX-512: minor change in rndscale intrinsic llvm-svn: 207937
2014-05-04 21:35:37 +08:00
VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let AddedComplexity = 20 in {
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
def : Pat<(v8i64 (vselect VK8WM:$mask, (v8i64 VR512:$src),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(bc_v8i64 (v16i32 immAllZerosV)))),
(VMOVDQU64Zrrkz VK8WM:$mask, VR512:$src)>;
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
def : Pat<(v8i64 (vselect VK8WM:$mask, (bc_v8i64 (v16i32 immAllZerosV)),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(v8i64 VR512:$src))),
(VMOVDQU64Zrrkz (COPY_TO_REGCLASS (KNOTWrr (COPY_TO_REGCLASS VK8:$mask, VK16)),
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
VK8), VR512:$src)>;
def : Pat<(v16i32 (vselect VK16WM:$mask, (v16i32 VR512:$src),
(v16i32 immAllZerosV))),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(VMOVDQU32Zrrkz VK16WM:$mask, VR512:$src)>;
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
def : Pat<(v16i32 (vselect VK16WM:$mask, (v16i32 immAllZerosV),
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
(v16i32 VR512:$src))),
(VMOVDQU32Zrrkz (KNOTWrr VK16WM:$mask), VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[SKX] Enabling load/store instructions: encoding Instructions: VMOVAPD, VMOVAPS, VMOVDQA8, VMOVDQA16, VMOVDQA32,VMOVDQA64, VMOVDQU8, VMOVDQU16, VMOVDQU32,VMOVDQU64, VMOVUPD, VMOVUPS, Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 214719
2014-08-04 22:35:15 +08:00
Masked Load / Store Intrinsics - the CodeGen part. I'm recommiting the codegen part of the patch. The vectorizer part will be send to review again. Masked Vector Load and Store Intrinsics. Introduced new target-independent intrinsics in order to support masked vector loads and stores. The loop vectorizer optimizes loops containing conditional memory accesses by generating these intrinsics for existing targets AVX2 and AVX-512. The vectorizer asks the target about availability of masked vector loads and stores. Added SDNodes for masked operations and lowering patterns for X86 code generator. Examples: <16 x i32> @llvm.masked.load.v16i32(i8* %addr, <16 x i32> %passthru, i32 4 /* align */, <16 x i1> %mask) declare void @llvm.masked.store.v8f64(i8* %addr, <8 x double> %value, i32 4, <8 x i1> %mask) Scalarizer for other targets (not AVX2/AVX-512) will be done in a separate patch. http://reviews.llvm.org/D6191 llvm-svn: 223348
2014-12-04 17:40:44 +08:00
def: Pat<(v16i32 (masked_load addr:$ptr, VK16WM:$mask, (v16i32 immAllZerosV))),
(VMOVDQU32Zrmkz VK16WM:$mask, addr:$ptr)>;
def: Pat<(v16i32 (masked_load addr:$ptr, VK16WM:$mask, undef)),
(VMOVDQU32Zrmkz VK16WM:$mask, addr:$ptr)>;
def: Pat<(v16i32 (masked_load addr:$ptr, VK16WM:$mask, (v16i32 VR512:$src0))),
(VMOVDQU32Zrmk VR512:$src0, VK16WM:$mask, addr:$ptr)>;
def: Pat<(v8i64 (masked_load addr:$ptr, VK8WM:$mask,
(bc_v8i64 (v16i32 immAllZerosV)))),
(VMOVDQU64Zrmkz VK8WM:$mask, addr:$ptr)>;
def: Pat<(v8i64 (masked_load addr:$ptr, VK8WM:$mask, undef)),
(VMOVDQU64Zrmkz VK8WM:$mask, addr:$ptr)>;
def: Pat<(v8i64 (masked_load addr:$ptr, VK8WM:$mask, (v8i64 VR512:$src0))),
(VMOVDQU64Zrmk VR512:$src0, VK8WM:$mask, addr:$ptr)>;
def: Pat<(masked_store addr:$ptr, VK16WM:$mask, (v16i32 VR512:$src)),
(VMOVDQU32Zmrk addr:$ptr, VK16WM:$mask, VR512:$src)>;
def: Pat<(masked_store addr:$ptr, VK8WM:$mask, (v8i64 VR512:$src)),
(VMOVDQU64Zmrk addr:$ptr, VK8WM:$mask, VR512:$src)>;
// SKX replacement
def: Pat<(masked_store addr:$ptr, VK8WM:$mask, (v8i32 VR256:$src)),
(VMOVDQU32Z256mrk addr:$ptr, VK8WM:$mask, VR256:$src)>;
// KNL replacement
def: Pat<(masked_store addr:$ptr, VK8WM:$mask, (v8i32 VR256:$src)),
(VMOVDQU32Zmrk addr:$ptr,
(v16i1 (COPY_TO_REGCLASS VK8WM:$mask, VK16WM)),
(INSERT_SUBREG (v16i32 (IMPLICIT_DEF)), VR256:$src, sub_ymm))>;
def: Pat<(v8i32 (masked_load addr:$ptr, VK8WM:$mask, undef)),
(v8i32 (EXTRACT_SUBREG (v16i32 (VMOVDQU32Zrmkz
(v16i1 (COPY_TO_REGCLASS VK8WM:$mask, VK16WM)), addr:$ptr)), sub_ymm))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Move Int Doubleword to Packed Double Int
//
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVDI2PDIZrr : AVX512BI<0x6E, MRMSrcReg, (outs VR128X:$dst), (ins GR32:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst,
(v4i32 (scalar_to_vector GR32:$src)))], IIC_SSE_MOVDQ>,
EVEX, VEX_LIG;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVDI2PDIZrm : AVX512BI<0x6E, MRMSrcMem, (outs VR128X:$dst), (ins i32mem:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst,
(v4i32 (scalar_to_vector (loadi32 addr:$src))))],
IIC_SSE_MOVDQ>, EVEX, VEX_LIG, EVEX_CD8<32, CD8VT1>;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOV64toPQIZrr : AVX512BI<0x6E, MRMSrcReg, (outs VR128X:$dst), (ins GR64:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst,
(v2i64 (scalar_to_vector GR64:$src)))],
IIC_SSE_MOVDQ>, EVEX, VEX_W, VEX_LIG;
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
let isCodeGenOnly = 1 in {
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOV64toSDZrr : AVX512BI<0x6E, MRMSrcReg, (outs FR64:$dst), (ins GR64:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set FR64:$dst, (bitconvert GR64:$src))],
IIC_SSE_MOVDQ>, EVEX, VEX_W, Sched<[WriteMove]>;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVSDto64Zrr : AVX512BI<0x7E, MRMDestReg, (outs GR64:$dst), (ins FR64:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set GR64:$dst, (bitconvert FR64:$src))],
IIC_SSE_MOVDQ>, EVEX, VEX_W, Sched<[WriteMove]>;
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
}
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVSDto64Zmr : AVX512BI<0x7E, MRMDestMem, (outs), (ins i64mem:$dst, FR64:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(store (i64 (bitconvert FR64:$src)), addr:$dst)],
IIC_SSE_MOVDQ>, EVEX, VEX_W, Sched<[WriteStore]>,
EVEX_CD8<64, CD8VT1>;
// Move Int Doubleword to Single Scalar
//
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
let isCodeGenOnly = 1 in {
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVDI2SSZrr : AVX512BI<0x6E, MRMSrcReg, (outs FR32X:$dst), (ins GR32:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set FR32X:$dst, (bitconvert GR32:$src))],
IIC_SSE_MOVDQ>, EVEX, VEX_LIG;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVDI2SSZrm : AVX512BI<0x6E, MRMSrcMem, (outs FR32X:$dst), (ins i32mem:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set FR32X:$dst, (bitconvert (loadi32 addr:$src)))],
IIC_SSE_MOVDQ>, EVEX, VEX_LIG, EVEX_CD8<32, CD8VT1>;
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
// Move doubleword from xmm register to r/m32
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVPDI2DIZrr : AVX512BI<0x7E, MRMDestReg, (outs GR32:$dst), (ins VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set GR32:$dst, (vector_extract (v4i32 VR128X:$src),
(iPTR 0)))], IIC_SSE_MOVD_ToGP>,
EVEX, VEX_LIG;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVPDI2DIZmr : AVX512BI<0x7E, MRMDestMem, (outs),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins i32mem:$dst, VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(store (i32 (vector_extract (v4i32 VR128X:$src),
(iPTR 0))), addr:$dst)], IIC_SSE_MOVDQ>,
EVEX, VEX_LIG, EVEX_CD8<32, CD8VT1>;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
// Move quadword from xmm1 register to r/m64
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//
def VMOVPQIto64Zrr : I<0x7E, MRMDestReg, (outs GR64:$dst), (ins VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set GR64:$dst, (extractelt (v2i64 VR128X:$src),
(iPTR 0)))],
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
IIC_SSE_MOVD_ToGP>, PD, EVEX, VEX_LIG, VEX_W,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Requires<[HasAVX512, In64BitMode]>;
AVX-512: Fixed encoding of VMOVQ instruction. llvm-svn: 191889
2013-10-03 20:03:26 +08:00
def VMOVPQIto64Zmr : I<0xD6, MRMDestMem, (outs),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins i64mem:$dst, VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(store (extractelt (v2i64 VR128X:$src), (iPTR 0)),
addr:$dst)], IIC_SSE_MOVDQ>,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
EVEX, PD, VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Sched<[WriteStore]>, Requires<[HasAVX512, In64BitMode]>;
// Move Scalar Single to Double Int
//
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
let isCodeGenOnly = 1 in {
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVSS2DIZrr : AVX512BI<0x7E, MRMDestReg, (outs GR32:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins FR32X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set GR32:$dst, (bitconvert FR32X:$src))],
IIC_SSE_MOVD_ToGP>, EVEX, VEX_LIG;
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVSS2DIZmr : AVX512BI<0x7E, MRMDestMem, (outs),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins i32mem:$dst, FR32X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovd\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(store (i32 (bitconvert FR32X:$src)), addr:$dst)],
IIC_SSE_MOVDQ>, EVEX, VEX_LIG, EVEX_CD8<32, CD8VT1>;
Remove more filters from the disassembler. Mark some AVX512 instructions as CodeGenOnly. llvm-svn: 192525
2013-10-12 13:41:08 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Move Quadword Int to Packed Quadword Int
//
AVX-512: optimized scalar compare patterns removed AVX512SI format, since it is similar to AVX512BI. llvm-svn: 199217
2014-01-14 23:10:08 +08:00
def VMOVQI2PQIZrm : AVX512BI<0x6E, MRMSrcMem, (outs VR128X:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins i64mem:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst,
(v2i64 (scalar_to_vector (loadi64 addr:$src))))]>,
EVEX, VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
//===----------------------------------------------------------------------===//
// AVX-512 MOVSS, MOVSD
//===----------------------------------------------------------------------===//
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
multiclass avx512_move_scalar <string asm, RegisterClass RC,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
SDNode OpNode, ValueType vt,
X86MemOperand x86memop, PatFrag mem_pat> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
def rr : SI<0x10, MRMSrcReg, (outs VR128X:$dst), (ins VR128X:$src1, RC:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst, (vt (OpNode VR128X:$src1,
(scalar_to_vector RC:$src2))))],
IIC_SSE_MOV_S_RR>, EVEX_4V, VEX_LIG;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
let Constraints = "$src1 = $dst" in
def rrk : SI<0x10, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src1, VK1WM:$mask, RC:$src2, RC:$src3),
!strconcat(asm,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst {${mask}}|$dst {${mask}}, $src2, $src3}"),
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
[], IIC_SSE_MOV_S_RR>, EVEX_4V, VEX_LIG, EVEX_K;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rm : SI<0x10, MRMSrcMem, (outs RC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst, (mem_pat addr:$src))], IIC_SSE_MOV_S_RM>,
EVEX, VEX_LIG;
AVX-512: Added intrinsic for VMOVSS store form with mask. llvm-svn: 216530
2014-08-27 15:38:43 +08:00
let mayStore = 1 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def mr: SI<0x11, MRMDestMem, (outs), (ins x86memop:$dst, RC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(store RC:$src, addr:$dst)], IIC_SSE_MOV_S_MR>,
EVEX, VEX_LIG;
AVX-512: Added intrinsic for VMOVSS store form with mask. llvm-svn: 216530
2014-08-27 15:38:43 +08:00
def mrk: SI<0x11, MRMDestMem, (outs), (ins x86memop:$dst, VK1WM:$mask, RC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm, "\t{$src, $dst {${mask}}|$dst {${mask}}, $src}"),
AVX-512: Added intrinsic for VMOVSS store form with mask. llvm-svn: 216530
2014-08-27 15:38:43 +08:00
[], IIC_SSE_MOV_S_MR>,
EVEX, VEX_LIG, EVEX_K;
} // mayStore
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
} //hasSideEffects = 0
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
let ExeDomain = SSEPackedSingle in
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VMOVSSZ : avx512_move_scalar<"movss", FR32X, X86Movss, v4f32, f32mem,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
loadf32>, XS, EVEX_CD8<32, CD8VT1>;
let ExeDomain = SSEPackedDouble in
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VMOVSDZ : avx512_move_scalar<"movsd", FR64X, X86Movsd, v2f64, f64mem,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
loadf64>, XD, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Added legal type MVT::i1 and VK1 register for it. Added scalar compare VCMPSS, VCMPSD. Implemented LowerSELECT for scalar FP operations. I replaced FSETCCss, FSETCCsd with one node type FSETCCs. Node extract_vector_elt(v16i1/v8i1, idx) returns an element of type i1. llvm-svn: 197384
2013-12-16 21:52:35 +08:00
def : Pat<(f32 (X86select VK1WM:$mask, (f32 FR32X:$src1), (f32 FR32X:$src2))),
(COPY_TO_REGCLASS (VMOVSSZrrk (COPY_TO_REGCLASS FR32X:$src2, VR128X),
VK1WM:$mask, (f32 (IMPLICIT_DEF)), FR32X:$src1), FR32X)>;
def : Pat<(f64 (X86select VK1WM:$mask, (f64 FR64X:$src1), (f64 FR64X:$src2))),
(COPY_TO_REGCLASS (VMOVSDZrrk (COPY_TO_REGCLASS FR64X:$src2, VR128X),
VK1WM:$mask, (f64 (IMPLICIT_DEF)), FR64X:$src1), FR64X)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added intrinsic for VMOVSS store form with mask. llvm-svn: 216530
2014-08-27 15:38:43 +08:00
def : Pat<(int_x86_avx512_mask_store_ss addr:$dst, VR128X:$src, GR8:$mask),
(VMOVSSZmrk addr:$dst, (i1 (COPY_TO_REGCLASS GR8:$mask, VK1WM)),
(COPY_TO_REGCLASS VR128X:$src, FR32X))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// For the disassembler
Add a new x86 specific instruction flag to force some isCodeGenOnly instructions to go through to the disassembler tables without resorting to string matches. Apply flag to all _REV instructions. llvm-svn: 198543
2014-01-05 12:17:28 +08:00
let isCodeGenOnly = 1, ForceDisassemble = 1, hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def VMOVSSZrr_REV : SI<0x11, MRMDestReg, (outs VR128X:$dst),
(ins VR128X:$src1, FR32X:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"movss\t{$src2, $src1, $dst|$dst, $src1, $src2}", [],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_MOV_S_RR>,
XS, EVEX_4V, VEX_LIG;
def VMOVSDZrr_REV : SI<0x11, MRMDestReg, (outs VR128X:$dst),
(ins VR128X:$src1, FR64X:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"movsd\t{$src2, $src1, $dst|$dst, $src1, $src2}", [],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_MOV_S_RR>,
XD, EVEX_4V, VEX_LIG, VEX_W;
}
let Predicates = [HasAVX512] in {
let AddedComplexity = 15 in {
// Move scalar to XMM zero-extended, zeroing a VR128X then do a
// MOVS{S,D} to the lower bits.
def : Pat<(v4f32 (X86vzmovl (v4f32 (scalar_to_vector FR32X:$src)))),
(VMOVSSZrr (v4f32 (V_SET0)), FR32X:$src)>;
def : Pat<(v4f32 (X86vzmovl (v4f32 VR128X:$src))),
(VMOVSSZrr (v4f32 (V_SET0)), (COPY_TO_REGCLASS VR128X:$src, FR32X))>;
def : Pat<(v4i32 (X86vzmovl (v4i32 VR128X:$src))),
(VMOVSSZrr (v4i32 (V_SET0)), (COPY_TO_REGCLASS VR128X:$src, FR32X))>;
def : Pat<(v2f64 (X86vzmovl (v2f64 (scalar_to_vector FR64X:$src)))),
(VMOVSDZrr (v2f64 (V_SET0)), FR64X:$src)>;
// Move low f32 and clear high bits.
def : Pat<(v8f32 (X86vzmovl (v8f32 VR256X:$src))),
(SUBREG_TO_REG (i32 0),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(VMOVSSZrr (v4f32 (V_SET0)),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(EXTRACT_SUBREG (v8f32 VR256X:$src), sub_xmm)), sub_xmm)>;
def : Pat<(v8i32 (X86vzmovl (v8i32 VR256X:$src))),
(SUBREG_TO_REG (i32 0),
(VMOVSSZrr (v4i32 (V_SET0)),
(EXTRACT_SUBREG (v8i32 VR256X:$src), sub_xmm)), sub_xmm)>;
}
let AddedComplexity = 20 in {
// MOVSSrm zeros the high parts of the register; represent this
// with SUBREG_TO_REG. The AVX versions also write: DST[255:128] <- 0
def : Pat<(v4f32 (X86vzmovl (v4f32 (scalar_to_vector (loadf32 addr:$src))))),
(COPY_TO_REGCLASS (VMOVSSZrm addr:$src), VR128X)>;
def : Pat<(v4f32 (scalar_to_vector (loadf32 addr:$src))),
(COPY_TO_REGCLASS (VMOVSSZrm addr:$src), VR128X)>;
def : Pat<(v4f32 (X86vzmovl (loadv4f32 addr:$src))),
(COPY_TO_REGCLASS (VMOVSSZrm addr:$src), VR128X)>;
// MOVSDrm zeros the high parts of the register; represent this
// with SUBREG_TO_REG. The AVX versions also write: DST[255:128] <- 0
def : Pat<(v2f64 (X86vzmovl (v2f64 (scalar_to_vector (loadf64 addr:$src))))),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
def : Pat<(v2f64 (scalar_to_vector (loadf64 addr:$src))),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
def : Pat<(v2f64 (X86vzmovl (loadv2f64 addr:$src))),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
def : Pat<(v2f64 (X86vzmovl (bc_v2f64 (loadv4f32 addr:$src)))),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
def : Pat<(v2f64 (X86vzload addr:$src)),
(COPY_TO_REGCLASS (VMOVSDZrm addr:$src), VR128X)>;
// Represent the same patterns above but in the form they appear for
// 256-bit types
def : Pat<(v8i32 (X86vzmovl (insert_subvector undef,
(v4i32 (scalar_to_vector (loadi32 addr:$src))), (iPTR 0)))),
AVX-512: fixed a bug in getLoadStoreRegOpcode() for AVX-512 target llvm-svn: 191818
2013-10-02 20:20:42 +08:00
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrm addr:$src), sub_xmm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f32 (X86vzmovl (insert_subvector undef,
(v4f32 (scalar_to_vector (loadf32 addr:$src))), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVSSZrm addr:$src), sub_xmm)>;
def : Pat<(v4f64 (X86vzmovl (insert_subvector undef,
(v2f64 (scalar_to_vector (loadf64 addr:$src))), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVSDZrm addr:$src), sub_xmm)>;
}
def : Pat<(v8f32 (X86vzmovl (insert_subvector undef,
(v4f32 (scalar_to_vector FR32X:$src)), (iPTR 0)))),
(SUBREG_TO_REG (i32 0), (v4f32 (VMOVSSZrr (v4f32 (V_SET0)),
FR32X:$src)), sub_xmm)>;
def : Pat<(v4f64 (X86vzmovl (insert_subvector undef,
(v2f64 (scalar_to_vector FR64X:$src)), (iPTR 0)))),
(SUBREG_TO_REG (i64 0), (v2f64 (VMOVSDZrr (v2f64 (V_SET0)),
FR64X:$src)), sub_xmm)>;
def : Pat<(v4i64 (X86vzmovl (insert_subvector undef,
(v2i64 (scalar_to_vector (loadi64 addr:$src))), (iPTR 0)))),
AVX-512: fixed a bug in getLoadStoreRegOpcode() for AVX-512 target llvm-svn: 191818
2013-10-02 20:20:42 +08:00
(SUBREG_TO_REG (i64 0), (VMOVQI2PQIZrm addr:$src), sub_xmm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Move low f64 and clear high bits.
def : Pat<(v4f64 (X86vzmovl (v4f64 VR256X:$src))),
(SUBREG_TO_REG (i32 0),
(VMOVSDZrr (v2f64 (V_SET0)),
(EXTRACT_SUBREG (v4f64 VR256X:$src), sub_xmm)), sub_xmm)>;
def : Pat<(v4i64 (X86vzmovl (v4i64 VR256X:$src))),
(SUBREG_TO_REG (i32 0), (VMOVSDZrr (v2i64 (V_SET0)),
(EXTRACT_SUBREG (v4i64 VR256X:$src), sub_xmm)), sub_xmm)>;
// Extract and store.
def : Pat<(store (f32 (vector_extract (v4f32 VR128X:$src), (iPTR 0))),
addr:$dst),
(VMOVSSZmr addr:$dst, (COPY_TO_REGCLASS (v4f32 VR128X:$src), FR32X))>;
def : Pat<(store (f64 (vector_extract (v2f64 VR128X:$src), (iPTR 0))),
addr:$dst),
(VMOVSDZmr addr:$dst, (COPY_TO_REGCLASS (v2f64 VR128X:$src), FR64X))>;
// Shuffle with VMOVSS
def : Pat<(v4i32 (X86Movss VR128X:$src1, VR128X:$src2)),
(VMOVSSZrr (v4i32 VR128X:$src1),
(COPY_TO_REGCLASS (v4i32 VR128X:$src2), FR32X))>;
def : Pat<(v4f32 (X86Movss VR128X:$src1, VR128X:$src2)),
(VMOVSSZrr (v4f32 VR128X:$src1),
(COPY_TO_REGCLASS (v4f32 VR128X:$src2), FR32X))>;
// 256-bit variants
def : Pat<(v8i32 (X86Movss VR256X:$src1, VR256X:$src2)),
(SUBREG_TO_REG (i32 0),
(VMOVSSZrr (EXTRACT_SUBREG (v8i32 VR256X:$src1), sub_xmm),
(EXTRACT_SUBREG (v8i32 VR256X:$src2), sub_xmm)),
sub_xmm)>;
def : Pat<(v8f32 (X86Movss VR256X:$src1, VR256X:$src2)),
(SUBREG_TO_REG (i32 0),
(VMOVSSZrr (EXTRACT_SUBREG (v8f32 VR256X:$src1), sub_xmm),
(EXTRACT_SUBREG (v8f32 VR256X:$src2), sub_xmm)),
sub_xmm)>;
// Shuffle with VMOVSD
def : Pat<(v2i64 (X86Movsd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v2f64 (X86Movsd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v4f32 (X86Movsd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v4i32 (X86Movsd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
// 256-bit variants
def : Pat<(v4i64 (X86Movsd VR256X:$src1, VR256X:$src2)),
(SUBREG_TO_REG (i32 0),
(VMOVSDZrr (EXTRACT_SUBREG (v4i64 VR256X:$src1), sub_xmm),
(EXTRACT_SUBREG (v4i64 VR256X:$src2), sub_xmm)),
sub_xmm)>;
def : Pat<(v4f64 (X86Movsd VR256X:$src1, VR256X:$src2)),
(SUBREG_TO_REG (i32 0),
(VMOVSDZrr (EXTRACT_SUBREG (v4f64 VR256X:$src1), sub_xmm),
(EXTRACT_SUBREG (v4f64 VR256X:$src2), sub_xmm)),
sub_xmm)>;
def : Pat<(v2f64 (X86Movlpd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v2i64 (X86Movlpd VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v4f32 (X86Movlps VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
def : Pat<(v4i32 (X86Movlps VR128X:$src1, VR128X:$src2)),
(VMOVSDZrr VR128X:$src1, (COPY_TO_REGCLASS VR128X:$src2, FR64X))>;
}
let AddedComplexity = 15 in
def VMOVZPQILo2PQIZrr : AVX512XSI<0x7E, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set VR128X:$dst, (v2i64 (X86vzmovl
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v2i64 VR128X:$src))))],
IIC_SSE_MOVQ_RR>, EVEX, VEX_W;
let AddedComplexity = 20 in
def VMOVZPQILo2PQIZrm : AVX512XSI<0x7E, MRMSrcMem, (outs VR128X:$dst),
(ins i128mem:$src),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovq\t{$src, $dst|$dst, $src}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst, (v2i64 (X86vzmovl
(loadv2i64 addr:$src))))],
IIC_SSE_MOVDQ>, EVEX, VEX_W,
EVEX_CD8<8, CD8VT8>;
let Predicates = [HasAVX512] in {
// AVX 128-bit movd/movq instruction write zeros in the high 128-bit part.
let AddedComplexity = 20 in {
def : Pat<(v4i32 (X86vzmovl (v4i32 (scalar_to_vector (loadi32 addr:$src))))),
(VMOVDI2PDIZrm addr:$src)>;
AVX-512: Added X86vzmovl patterns llvm-svn: 191733
2013-10-01 16:38:02 +08:00
def : Pat<(v2i64 (X86vzmovl (v2i64 (scalar_to_vector GR64:$src)))),
(VMOV64toPQIZrr GR64:$src)>;
def : Pat<(v4i32 (X86vzmovl (v4i32 (scalar_to_vector GR32:$src)))),
(VMOVDI2PDIZrr GR32:$src)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v4i32 (X86vzmovl (bc_v4i32 (loadv4f32 addr:$src)))),
(VMOVDI2PDIZrm addr:$src)>;
def : Pat<(v4i32 (X86vzmovl (bc_v4i32 (loadv2i64 addr:$src)))),
(VMOVDI2PDIZrm addr:$src)>;
def : Pat<(v2i64 (X86vzmovl (loadv2i64 addr:$src))),
(VMOVZPQILo2PQIZrm addr:$src)>;
def : Pat<(v2f64 (X86vzmovl (v2f64 VR128X:$src))),
(VMOVZPQILo2PQIZrr VR128X:$src)>;
Add AVX512 patterns for v16i32 broadcast and v2i64 zero extend load. Patch by Aleksey Bader. llvm-svn: 196435
2013-12-05 08:11:25 +08:00
def : Pat<(v2i64 (X86vzload addr:$src)),
(VMOVZPQILo2PQIZrm addr:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added X86vzmovl patterns llvm-svn: 191733
2013-10-01 16:38:02 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Use regular 128-bit instructions to match 256-bit scalar_to_vec+zext.
def : Pat<(v8i32 (X86vzmovl (insert_subvector undef,
(v4i32 (scalar_to_vector GR32:$src)),(iPTR 0)))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src), sub_xmm)>;
def : Pat<(v4i64 (X86vzmovl (insert_subvector undef,
(v2i64 (scalar_to_vector GR64:$src)),(iPTR 0)))),
(SUBREG_TO_REG (i64 0), (VMOV64toPQIZrr GR64:$src), sub_xmm)>;
}
def : Pat<(v16i32 (X86Vinsert (v16i32 immAllZerosV), GR32:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src2), sub_xmm)>;
def : Pat<(v8i64 (X86Vinsert (bc_v8i64 (v16i32 immAllZerosV)), GR64:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOV64toPQIZrr GR64:$src2), sub_xmm)>;
def : Pat<(v16i32 (X86Vinsert undef, GR32:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOVDI2PDIZrr GR32:$src2), sub_xmm)>;
def : Pat<(v8i64 (X86Vinsert undef, GR64:$src2, (iPTR 0))),
(SUBREG_TO_REG (i32 0), (VMOV64toPQIZrr GR64:$src2), sub_xmm)>;
[X86] AVX512: Add vmovntdqa Along with the corresponding intrinsic and tests. llvm-svn: 210543
2014-06-11 00:39:53 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Non-temporals
//===----------------------------------------------------------------------===//
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
let SchedRW = [WriteLoad] in {
def VMOVNTDQAZrm : AVX512PI<0x2A, MRMSrcMem, (outs VR512:$dst),
(ins i512mem:$src), "vmovntdqa\t{$src, $dst|$dst, $src}",
[(set VR512:$dst, (int_x86_avx512_movntdqa addr:$src))],
SSEPackedInt>, EVEX, T8PD, EVEX_V512,
EVEX_CD8<64, CD8VF>;
let Predicates = [HasAVX512, HasVLX] in {
def VMOVNTDQAZ256rm : AVX512PI<0x2A, MRMSrcMem, (outs VR256X:$dst),
(ins i256mem:$src),
"vmovntdqa\t{$src, $dst|$dst, $src}", [],
SSEPackedInt>, EVEX, T8PD, EVEX_V256,
EVEX_CD8<64, CD8VF>;
def VMOVNTDQAZ128rm : AVX512PI<0x2A, MRMSrcMem, (outs VR128X:$dst),
(ins i128mem:$src),
"vmovntdqa\t{$src, $dst|$dst, $src}", [],
SSEPackedInt>, EVEX, T8PD, EVEX_V128,
EVEX_CD8<64, CD8VF>;
}
}
[X86] AVX512: Add vmovntdqa Along with the corresponding intrinsic and tests. llvm-svn: 210543
2014-06-11 00:39:53 +08:00
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
multiclass avx512_movnt<bits<8> opc, string OpcodeStr, PatFrag st_frag,
ValueType OpVT, RegisterClass RC, X86MemOperand memop,
Domain d, InstrItinClass itin = IIC_SSE_MOVNT> {
let SchedRW = [WriteStore], mayStore = 1,
AddedComplexity = 400 in
def mr : AVX512PI<opc, MRMDestMem, (outs), (ins memop:$dst, RC:$src),
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
[(st_frag (OpVT RC:$src), addr:$dst)], d, itin>, EVEX;
[X86] AVX512: Add non-temporal stores Note that I followed the AVX2 convention here and didn't add LLVM intrinsics for stores. These can be generated with the nontemporal hint on LLVM IR stores (see new test). The GCC builtins are lowered directly into nontemporal stores. <rdar://problem/17082571> llvm-svn: 211176
2014-06-19 00:51:10 +08:00
}
[SKX] Extended non-temporal load/store instructions for AVX512VL subsets. Added avx512_movnt_vl multiclass for handling 256/128-bit forms of instruction. Added encoding and lowering tests. Reviewed by Elena Demikhovsky <elena.demikhovsky@intel.com> llvm-svn: 215536
2014-08-13 18:46:00 +08:00
multiclass avx512_movnt_vl<bits<8> opc, string OpcodeStr, PatFrag st_frag,
string elty, string elsz, string vsz512,
string vsz256, string vsz128, Domain d,
Predicate prd, InstrItinClass itin = IIC_SSE_MOVNT> {
let Predicates = [prd] in
defm Z : avx512_movnt<opc, OpcodeStr, st_frag,
!cast<ValueType>("v"##vsz512##elty##elsz), VR512,
!cast<X86MemOperand>(elty##"512mem"), d, itin>,
EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_movnt<opc, OpcodeStr, st_frag,
!cast<ValueType>("v"##vsz256##elty##elsz), VR256X,
!cast<X86MemOperand>(elty##"256mem"), d, itin>,
EVEX_V256;
defm Z128 : avx512_movnt<opc, OpcodeStr, st_frag,
!cast<ValueType>("v"##vsz128##elty##elsz), VR128X,
!cast<X86MemOperand>(elty##"128mem"), d, itin>,
EVEX_V128;
}
}
defm VMOVNTDQ : avx512_movnt_vl<0xE7, "vmovntdq", alignednontemporalstore,
"i", "64", "8", "4", "2", SSEPackedInt,
HasAVX512>, PD, EVEX_CD8<64, CD8VF>;
defm VMOVNTPD : avx512_movnt_vl<0x2B, "vmovntpd", alignednontemporalstore,
"f", "64", "8", "4", "2", SSEPackedDouble,
HasAVX512>, PD, VEX_W, EVEX_CD8<64, CD8VF>;
defm VMOVNTPS : avx512_movnt_vl<0x2B, "vmovntps", alignednontemporalstore,
"f", "32", "16", "8", "4", SSEPackedSingle,
HasAVX512>, PS, EVEX_CD8<32, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Integer arithmetic
//
multiclass avx512_binop_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
X86VectorVTInfo _, OpndItins itins,
bit IsCommutable = 0> {
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
(ins _.RC:$src1, _.RC:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, _.RC:$src2)),
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
"", itins.rr, IsCommutable>,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
AVX512BIBase, EVEX_4V;
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
let mayLoad = 1 in
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1,
(bitconvert (_.LdFrag addr:$src2)))),
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
"", itins.rm>,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
AVX512BIBase, EVEX_4V;
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
}
multiclass avx512_binop_rmb<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _, OpndItins itins,
bit IsCommutable = 0> :
avx512_binop_rm<opc, OpcodeStr, OpNode, _, itins, IsCommutable> {
let mayLoad = 1 in
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rmb : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr##_.Suffix,
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(_.VT (OpNode _.RC:$src1,
(X86VBroadcast
(_.ScalarLdFrag addr:$src2)))),
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
"", itins.rm>,
[AVX512] Refactoring of avx512_binop_rm multiclass through AVX512_masking. Added new argrument for AVX512_masking: InstrItinClass and bit isCommutable. No functional change. llvm-svn: 219310
2014-10-08 22:37:45 +08:00
AVX512BIBase, EVEX_4V, EVEX_B;
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
}
[AVX512] Extended avx512_binop_rm for AVX512VL subsets. Added avx512_binop_rm_vl multiclass for VL subset Added encoding tests llvm-svn: 219390
2014-10-09 16:38:48 +08:00
multiclass avx512_binop_rm_vl<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, OpndItins itins,
Predicate prd, bit IsCommutable = 0> {
let Predicates = [prd] in
defm Z : avx512_binop_rm<opc, OpcodeStr, OpNode, VTInfo.info512, itins,
IsCommutable>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_binop_rm<opc, OpcodeStr, OpNode, VTInfo.info256, itins,
IsCommutable>, EVEX_V256;
defm Z128 : avx512_binop_rm<opc, OpcodeStr, OpNode, VTInfo.info128, itins,
IsCommutable>, EVEX_V128;
}
}
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
multiclass avx512_binop_rmb_vl<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo VTInfo, OpndItins itins,
Predicate prd, bit IsCommutable = 0> {
let Predicates = [prd] in
defm Z : avx512_binop_rmb<opc, OpcodeStr, OpNode, VTInfo.info512, itins,
IsCommutable>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : avx512_binop_rmb<opc, OpcodeStr, OpNode, VTInfo.info256, itins,
IsCommutable>, EVEX_V256;
defm Z128 : avx512_binop_rmb<opc, OpcodeStr, OpNode, VTInfo.info128, itins,
IsCommutable>, EVEX_V128;
}
}
multiclass avx512_binop_rm_vl_q<bits<8> opc, string OpcodeStr, SDNode OpNode,
OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm NAME : avx512_binop_rmb_vl<opc, OpcodeStr, OpNode, avx512vl_i64_info,
itins, prd, IsCommutable>,
VEX_W, EVEX_CD8<64, CD8VF>;
}
multiclass avx512_binop_rm_vl_d<bits<8> opc, string OpcodeStr, SDNode OpNode,
OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm NAME : avx512_binop_rmb_vl<opc, OpcodeStr, OpNode, avx512vl_i32_info,
itins, prd, IsCommutable>, EVEX_CD8<32, CD8VF>;
}
multiclass avx512_binop_rm_vl_w<bits<8> opc, string OpcodeStr, SDNode OpNode,
OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm NAME : avx512_binop_rm_vl<opc, OpcodeStr, OpNode, avx512vl_i16_info,
itins, prd, IsCommutable>, EVEX_CD8<16, CD8VF>;
}
multiclass avx512_binop_rm_vl_b<bits<8> opc, string OpcodeStr, SDNode OpNode,
OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm NAME : avx512_binop_rm_vl<opc, OpcodeStr, OpNode, avx512vl_i8_info,
itins, prd, IsCommutable>, EVEX_CD8<8, CD8VF>;
}
multiclass avx512_binop_rm_vl_dq<bits<8> opc_d, bits<8> opc_q, string OpcodeStr,
SDNode OpNode, OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm Q : avx512_binop_rm_vl_q<opc_q, OpcodeStr, OpNode, itins, prd,
IsCommutable>;
defm D : avx512_binop_rm_vl_d<opc_d, OpcodeStr, OpNode, itins, prd,
IsCommutable>;
}
multiclass avx512_binop_rm_vl_bw<bits<8> opc_b, bits<8> opc_w, string OpcodeStr,
SDNode OpNode, OpndItins itins, Predicate prd,
bit IsCommutable = 0> {
defm W : avx512_binop_rm_vl_w<opc_w, OpcodeStr, OpNode, itins, prd,
IsCommutable>;
defm B : avx512_binop_rm_vl_b<opc_b, OpcodeStr, OpNode, itins, prd,
IsCommutable>;
}
multiclass avx512_binop_rm_vl_all<bits<8> opc_b, bits<8> opc_w,
bits<8> opc_d, bits<8> opc_q,
string OpcodeStr, SDNode OpNode,
OpndItins itins, bit IsCommutable = 0> {
defm NAME : avx512_binop_rm_vl_dq<opc_d, opc_q, OpcodeStr, OpNode,
itins, HasAVX512, IsCommutable>,
avx512_binop_rm_vl_bw<opc_b, opc_w, OpcodeStr, OpNode,
itins, HasBWI, IsCommutable>;
}
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
multiclass avx512_binop_rm2<bits<8> opc, string OpcodeStr, ValueType DstVT,
ValueType SrcVT, RegisterClass KRC, RegisterClass RC,
PatFrag memop_frag, X86MemOperand x86memop,
PatFrag scalar_mfrag, X86MemOperand x86scalar_mop,
string BrdcstStr, OpndItins itins, bit IsCommutable = 0> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let isCommutable = IsCommutable in
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
{
def rr : AVX512BI<opc, MRMSrcReg, (outs RC:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins RC:$src1, RC:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[]>, EVEX_4V;
def rrk : AVX512BI<opc, MRMSrcReg, (outs RC:$dst),
(ins KRC:$mask, RC:$src1, RC:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst {${mask}}|$dst {${mask}}, $src1, $src2}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[], itins.rr>, EVEX_4V, EVEX_K;
def rrkz : AVX512BI<opc, MRMSrcReg, (outs RC:$dst),
(ins KRC:$mask, RC:$src1, RC:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst {${mask}} {z}" ,
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
"|$dst {${mask}} {z}, $src1, $src2}"),
[], itins.rr>, EVEX_4V, EVEX_KZ;
}
let mayLoad = 1 in {
def rm : AVX512BI<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, x86memop:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[]>, EVEX_4V;
def rmk : AVX512BI<opc, MRMSrcMem, (outs RC:$dst),
(ins KRC:$mask, RC:$src1, x86memop:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst {${mask}}|$dst {${mask}}, $src1, $src2}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[], itins.rm>, EVEX_4V, EVEX_K;
def rmkz : AVX512BI<opc, MRMSrcMem, (outs RC:$dst),
(ins KRC:$mask, RC:$src1, x86memop:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst {${mask}} {z}|$dst {${mask}} {z}, $src1, $src2}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[], itins.rm>, EVEX_4V, EVEX_KZ;
def rmb : AVX512BI<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, x86scalar_mop:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src2}", BrdcstStr,
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
", $src1, $dst|$dst, $src1, ${src2}", BrdcstStr, "}"),
[], itins.rm>, EVEX_4V, EVEX_B;
def rmbk : AVX512BI<opc, MRMSrcMem, (outs RC:$dst),
(ins KRC:$mask, RC:$src1, x86scalar_mop:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src2}", BrdcstStr,
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
", $src1, $dst {${mask}}|$dst {${mask}}, $src1, ${src2}",
BrdcstStr, "}"),
[], itins.rm>, EVEX_4V, EVEX_B, EVEX_K;
def rmbkz : AVX512BI<opc, MRMSrcMem, (outs RC:$dst),
(ins KRC:$mask, RC:$src1, x86scalar_mop:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src2}", BrdcstStr,
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
", $src1, $dst {${mask}} {z}|$dst {${mask}} {z}, $src1, ${src2}",
BrdcstStr, "}"),
[], itins.rm>, EVEX_4V, EVEX_B, EVEX_KZ;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
defm VPADD : avx512_binop_rm_vl_all<0xFC, 0xFD, 0xFE, 0xD4, "vpadd", add,
SSE_INTALU_ITINS_P, 1>;
defm VPSUB : avx512_binop_rm_vl_all<0xF8, 0xF9, 0xFA, 0xFB, "vpsub", sub,
SSE_INTALU_ITINS_P, 0>;
defm VPMULLD : avx512_binop_rm_vl_d<0x40, "vpmull", mul,
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
defm VPMULLW : avx512_binop_rm_vl_w<0xD5, "vpmull", mul,
SSE_INTALU_ITINS_P, HasBWI, 1>;
[AVX512] Extended avx512_binop_rm to DQ/VL subsets. Added encoding tests. llvm-svn: 219686
2014-10-14 23:13:56 +08:00
defm VPMULLQ : avx512_binop_rm_vl_q<0x40, "vpmull", mul,
SSE_INTALU_ITINS_P, HasDQI, 1>, T8PD;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
defm VPMULDQZ : avx512_binop_rm2<0x28, "vpmuldq", v8i64, v16i32, VK8WM, VR512,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8i64, i512mem, loadi64, i64mem, "{1to8}",
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
SSE_INTALU_ITINS_P, 1>, T8PD, EVEX_V512,
EVEX_CD8<64, CD8VF>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
defm VPMULUDQZ : avx512_binop_rm2<0xF4, "vpmuludq", v8i64, v16i32, VK8WM, VR512,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8i64, i512mem, loadi64, i64mem, "{1to8}",
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
SSE_INTMUL_ITINS_P, 1>, EVEX_V512, EVEX_CD8<64, CD8VF>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i64 (X86pmuludq (v16i32 VR512:$src1), (v16i32 VR512:$src2))),
(VPMULUDQZrr VR512:$src1, VR512:$src2)>;
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
def : Pat<(v8i64 (int_x86_avx512_mask_pmulu_dq_512 (v16i32 VR512:$src1),
(v16i32 VR512:$src2), (bc_v8i64 (v16i32 immAllZerosV)), (i8 -1))),
(VPMULUDQZrr VR512:$src1, VR512:$src2)>;
def : Pat<(v8i64 (int_x86_avx512_mask_pmul_dq_512 (v16i32 VR512:$src1),
(v16i32 VR512:$src2), (bc_v8i64 (v16i32 immAllZerosV)), (i8 -1))),
(VPMULDQZrr VR512:$src1, VR512:$src2)>;
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
defm VPMAXSB : avx512_binop_rm_vl_b<0x3C, "vpmaxs", X86smax,
SSE_INTALU_ITINS_P, HasBWI, 1>, T8PD;
defm VPMAXSW : avx512_binop_rm_vl_w<0xEE, "vpmaxs", X86smax,
SSE_INTALU_ITINS_P, HasBWI, 1>;
defm VPMAXS : avx512_binop_rm_vl_dq<0x3D, 0x3D, "vpmaxs", X86smax,
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
defm VPMAXUB : avx512_binop_rm_vl_b<0xDE, "vpmaxu", X86umax,
SSE_INTALU_ITINS_P, HasBWI, 1>;
defm VPMAXUW : avx512_binop_rm_vl_w<0x3E, "vpmaxu", X86umax,
SSE_INTALU_ITINS_P, HasBWI, 1>, T8PD;
defm VPMAXU : avx512_binop_rm_vl_dq<0x3F, 0x3F, "vpmaxu", X86umax,
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
defm VPMINSB : avx512_binop_rm_vl_b<0x38, "vpmins", X86smin,
SSE_INTALU_ITINS_P, HasBWI, 1>, T8PD;
defm VPMINSW : avx512_binop_rm_vl_w<0xEA, "vpmins", X86smin,
SSE_INTALU_ITINS_P, HasBWI, 1>;
defm VPMINS : avx512_binop_rm_vl_dq<0x39, 0x39, "vpmins", X86smin,
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
defm VPMINUB : avx512_binop_rm_vl_b<0xDA, "vpminu", X86umin,
SSE_INTALU_ITINS_P, HasBWI, 1>;
defm VPMINUW : avx512_binop_rm_vl_w<0x3A, "vpminu", X86umin,
SSE_INTALU_ITINS_P, HasBWI, 1>, T8PD;
defm VPMINU : avx512_binop_rm_vl_dq<0x3B, 0x3B, "vpminu", X86umin,
SSE_INTALU_ITINS_P, HasAVX512, 1>, T8PD;
AVX-512: PMIN/PMAX intrinsics and patterns Patch by Cameron McInally <cameron.mcinally@nyu.edu> llvm-svn: 193497
2013-10-27 16:18:37 +08:00
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
def : Pat <(v16i32 (int_x86_avx512_mask_pmaxs_d_512 (v16i32 VR512:$src1),
(v16i32 VR512:$src2), (v16i32 immAllZerosV), (i16 -1))),
(VPMAXSDZrr VR512:$src1, VR512:$src2)>;
def : Pat <(v16i32 (int_x86_avx512_mask_pmaxu_d_512 (v16i32 VR512:$src1),
(v16i32 VR512:$src2), (v16i32 immAllZerosV), (i16 -1))),
(VPMAXUDZrr VR512:$src1, VR512:$src2)>;
def : Pat <(v8i64 (int_x86_avx512_mask_pmaxs_q_512 (v8i64 VR512:$src1),
(v8i64 VR512:$src2), (bc_v8i64 (v16i32 immAllZerosV)), (i8 -1))),
(VPMAXSQZrr VR512:$src1, VR512:$src2)>;
def : Pat <(v8i64 (int_x86_avx512_mask_pmaxu_q_512 (v8i64 VR512:$src1),
(v8i64 VR512:$src2), (bc_v8i64 (v16i32 immAllZerosV)), (i8 -1))),
(VPMAXUQZrr VR512:$src1, VR512:$src2)>;
def : Pat <(v16i32 (int_x86_avx512_mask_pmins_d_512 (v16i32 VR512:$src1),
(v16i32 VR512:$src2), (v16i32 immAllZerosV), (i16 -1))),
(VPMINSDZrr VR512:$src1, VR512:$src2)>;
def : Pat <(v16i32 (int_x86_avx512_mask_pminu_d_512 (v16i32 VR512:$src1),
(v16i32 VR512:$src2), (v16i32 immAllZerosV), (i16 -1))),
(VPMINUDZrr VR512:$src1, VR512:$src2)>;
def : Pat <(v8i64 (int_x86_avx512_mask_pmins_q_512 (v8i64 VR512:$src1),
(v8i64 VR512:$src2), (bc_v8i64 (v16i32 immAllZerosV)), (i8 -1))),
(VPMINSQZrr VR512:$src1, VR512:$src2)>;
def : Pat <(v8i64 (int_x86_avx512_mask_pminu_q_512 (v8i64 VR512:$src1),
(v8i64 VR512:$src2), (bc_v8i64 (v16i32 immAllZerosV)), (i8 -1))),
(VPMINUQZrr VR512:$src1, VR512:$src2)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - Unpack Instructions
//===----------------------------------------------------------------------===//
multiclass avx512_unpack_fp<bits<8> opc, SDNode OpNode, ValueType vt,
PatFrag mem_frag, RegisterClass RC,
X86MemOperand x86memop, string asm,
Domain d> {
def rr : AVX512PI<opc, MRMSrcReg,
(outs RC:$dst), (ins RC:$src1, RC:$src2),
asm, [(set RC:$dst,
(vt (OpNode RC:$src1, RC:$src2)))],
AVX-512: Added TB prefix to all instructions without prefixes, otherwise encoding fails after the last change in X86MCCodeEmitter.cpp. llvm-svn: 191812
2013-10-02 14:39:07 +08:00
d>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rm : AVX512PI<opc, MRMSrcMem,
(outs RC:$dst), (ins RC:$src1, x86memop:$src2),
asm, [(set RC:$dst,
(vt (OpNode RC:$src1,
(bitconvert (mem_frag addr:$src2)))))],
AVX-512: Added TB prefix to all instructions without prefixes, otherwise encoding fails after the last change in X86MCCodeEmitter.cpp. llvm-svn: 191812
2013-10-02 14:39:07 +08:00
d>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VUNPCKHPSZ: avx512_unpack_fp<0x15, X86Unpckh, v16f32, loadv8f64,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VR512, f512mem, "vunpckhps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
SSEPackedSingle>, PS, EVEX_V512, EVEX_CD8<32, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VUNPCKHPDZ: avx512_unpack_fp<0x15, X86Unpckh, v8f64, loadv8f64,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VR512, f512mem, "vunpckhpd\t{$src2, $src1, $dst|$dst, $src1, $src2}",
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
SSEPackedDouble>, PD, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VUNPCKLPSZ: avx512_unpack_fp<0x14, X86Unpckl, v16f32, loadv8f64,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VR512, f512mem, "vunpcklps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
SSEPackedSingle>, PS, EVEX_V512, EVEX_CD8<32, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VUNPCKLPDZ: avx512_unpack_fp<0x14, X86Unpckl, v8f64, loadv8f64,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VR512, f512mem, "vunpcklpd\t{$src2, $src1, $dst|$dst, $src1, $src2}",
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
SSEPackedDouble>, PD, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
multiclass avx512_unpack_int<bits<8> opc, string OpcodeStr, SDNode OpNode,
ValueType OpVT, RegisterClass RC, PatFrag memop_frag,
X86MemOperand x86memop> {
def rr : AVX512BI<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src1, RC:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set RC:$dst, (OpVT (OpNode (OpVT RC:$src1), (OpVT RC:$src2))))],
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
IIC_SSE_UNPCK>, EVEX_4V;
def rm : AVX512BI<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, x86memop:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst, (OpVT (OpNode (OpVT RC:$src1),
(bitconvert (memop_frag addr:$src2)))))],
IIC_SSE_UNPCK>, EVEX_4V;
}
defm VPUNPCKLDQZ : avx512_unpack_int<0x62, "vpunpckldq", X86Unpckl, v16i32,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
VR512, loadv16i32, i512mem>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VF>;
defm VPUNPCKLQDQZ : avx512_unpack_int<0x6C, "vpunpcklqdq", X86Unpckl, v8i64,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
VR512, loadv8i64, i512mem>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VEX_W, EVEX_CD8<64, CD8VF>;
defm VPUNPCKHDQZ : avx512_unpack_int<0x6A, "vpunpckhdq", X86Unpckh, v16i32,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
VR512, loadv16i32, i512mem>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VF>;
defm VPUNPCKHQDQZ : avx512_unpack_int<0x6D, "vpunpckhqdq", X86Unpckh, v8i64,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
VR512, loadv8i64, i512mem>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VEX_W, EVEX_CD8<64, CD8VF>;
//===----------------------------------------------------------------------===//
// AVX-512 - PSHUFD
//
multiclass avx512_pshuf_imm<bits<8> opc, string OpcodeStr, RegisterClass RC,
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
SDNode OpNode, PatFrag mem_frag,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
X86MemOperand x86memop, ValueType OpVT> {
def ri : AVX512Ii8<opc, MRMSrcReg, (outs RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins RC:$src1, u8imm:$src2),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
(OpVT (OpNode RC:$src1, (i8 imm:$src2))))]>,
EVEX;
def mi : AVX512Ii8<opc, MRMSrcMem, (outs RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins x86memop:$src1, u8imm:$src2),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
(OpVT (OpNode (mem_frag addr:$src1),
(i8 imm:$src2))))]>, EVEX;
}
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VPSHUFDZ : avx512_pshuf_imm<0x70, "vpshufd", VR512, X86PShufd, loadv16i32,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
i512mem, v16i32>, PD, EVEX_V512, EVEX_CD8<32, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Logical Instructions
//===----------------------------------------------------------------------===//
[AVX512] Extended avx512_binop_rm to BW/VL subsets. Added encoding tests. llvm-svn: 219685
2014-10-14 22:36:19 +08:00
defm VPAND : avx512_binop_rm_vl_dq<0xDB, 0xDB, "vpand", and,
SSE_INTALU_ITINS_P, HasAVX512, 1>;
defm VPOR : avx512_binop_rm_vl_dq<0xEB, 0xEB, "vpor", or,
SSE_INTALU_ITINS_P, HasAVX512, 1>;
defm VPXOR : avx512_binop_rm_vl_dq<0xEF, 0xEF, "vpxor", xor,
SSE_INTALU_ITINS_P, HasAVX512, 1>;
defm VPANDN : avx512_binop_rm_vl_dq<0xDF, 0xDF, "vpandn", X86andnp,
SSE_INTALU_ITINS_P, HasAVX512, 1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 FP arithmetic
//===----------------------------------------------------------------------===//
multiclass avx512_binop_s<bits<8> opc, string OpcodeStr, SDNode OpNode,
SizeItins itins> {
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm SSZ : sse12_fp_scalar<opc, !strconcat(OpcodeStr, "ss"), OpNode, FR32X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f32mem, itins.s, 0>, XS, EVEX_4V, VEX_LIG,
EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm SDZ : sse12_fp_scalar<opc, !strconcat(OpcodeStr, "sd"), OpNode, FR64X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f64mem, itins.d, 0>, XD, VEX_W, EVEX_4V, VEX_LIG,
EVEX_CD8<64, CD8VT1>;
}
let isCommutable = 1 in {
defm VADD : avx512_binop_s<0x58, "add", fadd, SSE_ALU_ITINS_S>;
defm VMUL : avx512_binop_s<0x59, "mul", fmul, SSE_ALU_ITINS_S>;
defm VMIN : avx512_binop_s<0x5D, "min", X86fmin, SSE_ALU_ITINS_S>;
defm VMAX : avx512_binop_s<0x5F, "max", X86fmax, SSE_ALU_ITINS_S>;
}
let isCommutable = 0 in {
defm VSUB : avx512_binop_s<0x5C, "sub", fsub, SSE_ALU_ITINS_S>;
defm VDIV : avx512_binop_s<0x5E, "div", fdiv, SSE_ALU_ITINS_S>;
}
multiclass avx512_fp_packed<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Implemented AVX512VL FP bnary packed instructions (VADDP*, VSUBP*, VMULP*, VDIVP*, VMAXP*, VMINP*) Refactored through AVX512_maskable Added encoding tests for them. llvm-svn: 220858
2014-10-29 23:43:02 +08:00
X86VectorVTInfo _, bit IsCommutable> {
defm rr: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, _.RC:$src2))>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in {
[AVX512] Implemented AVX512VL FP bnary packed instructions (VADDP*, VSUBP*, VMULP*, VDIVP*, VMAXP*, VMINP*) Refactored through AVX512_maskable Added encoding tests for them. llvm-svn: 220858
2014-10-29 23:43:02 +08:00
defm rm: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr##_.Suffix,
"$src2, $src1", "$src1, $src2",
(OpNode _.RC:$src1, (_.LdFrag addr:$src2))>, EVEX_4V;
defm rmb: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.ScalarMemOp:$src2), OpcodeStr##_.Suffix,
"${src2}"##_.BroadcastStr##", $src1",
"$src1, ${src2}"##_.BroadcastStr,
(OpNode _.RC:$src1, (_.VT (X86VBroadcast
(_.ScalarLdFrag addr:$src2))))>,
EVEX_4V, EVEX_B;
}//let mayLoad = 1
}
multiclass avx512_fp_binop_p<bits<8> opc, string OpcodeStr, SDNode OpNode,
bit IsCommutable = 0> {
defm PSZ : avx512_fp_packed<opc, OpcodeStr, OpNode, v16f32_info,
IsCommutable>, EVEX_V512, PS,
EVEX_CD8<32, CD8VF>;
defm PDZ : avx512_fp_packed<opc, OpcodeStr, OpNode, v8f64_info,
IsCommutable>, EVEX_V512, PD, VEX_W,
EVEX_CD8<64, CD8VF>;
// Define only if AVX512VL feature is present.
let Predicates = [HasVLX] in {
defm PSZ128 : avx512_fp_packed<opc, OpcodeStr, OpNode, v4f32x_info,
IsCommutable>, EVEX_V128, PS,
EVEX_CD8<32, CD8VF>;
defm PSZ256 : avx512_fp_packed<opc, OpcodeStr, OpNode, v8f32x_info,
IsCommutable>, EVEX_V256, PS,
EVEX_CD8<32, CD8VF>;
defm PDZ128 : avx512_fp_packed<opc, OpcodeStr, OpNode, v2f64x_info,
IsCommutable>, EVEX_V128, PD, VEX_W,
EVEX_CD8<64, CD8VF>;
defm PDZ256 : avx512_fp_packed<opc, OpcodeStr, OpNode, v4f64x_info,
IsCommutable>, EVEX_V256, PD, VEX_W,
EVEX_CD8<64, CD8VF>;
AVX-512: Added rrk, rrkz, rmk, rmkz, rmbk, rmbkz versions of AVX512 FP packed instructions, added encoding tests for them. By Robert Khazanov. llvm-svn: 203098
2014-03-06 16:45:30 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Implemented AVX512VL FP bnary packed instructions (VADDP*, VSUBP*, VMULP*, VDIVP*, VMAXP*, VMINP*) Refactored through AVX512_maskable Added encoding tests for them. llvm-svn: 220858
2014-10-29 23:43:02 +08:00
defm VADD : avx512_fp_binop_p<0x58, "vadd", fadd, 1>;
defm VMUL : avx512_fp_binop_p<0x59, "vmul", fmul, 1>;
defm VMIN : avx512_fp_binop_p<0x5D, "vmin", X86fmin, 1>;
defm VMAX : avx512_fp_binop_p<0x5F, "vmax", X86fmax, 1>;
defm VSUB : avx512_fp_binop_p<0x5C, "vsub", fsub>;
defm VDIV : avx512_fp_binop_p<0x5E, "vdiv", fdiv>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
def : Pat<(v16f32 (int_x86_avx512_mask_max_ps_512 (v16f32 VR512:$src1),
(v16f32 VR512:$src2), (bc_v16f32 (v16i32 immAllZerosV)),
(i16 -1), FROUND_CURRENT)),
(VMAXPSZrr VR512:$src1, VR512:$src2)>;
def : Pat<(v8f64 (int_x86_avx512_mask_max_pd_512 (v8f64 VR512:$src1),
(v8f64 VR512:$src2), (bc_v8f64 (v16i32 immAllZerosV)),
(i8 -1), FROUND_CURRENT)),
(VMAXPDZrr VR512:$src1, VR512:$src2)>;
def : Pat<(v16f32 (int_x86_avx512_mask_min_ps_512 (v16f32 VR512:$src1),
(v16f32 VR512:$src2), (bc_v16f32 (v16i32 immAllZerosV)),
(i16 -1), FROUND_CURRENT)),
(VMINPSZrr VR512:$src1, VR512:$src2)>;
def : Pat<(v8f64 (int_x86_avx512_mask_min_pd_512 (v8f64 VR512:$src1),
(v8f64 VR512:$src2), (bc_v8f64 (v16i32 immAllZerosV)),
(i8 -1), FROUND_CURRENT)),
(VMINPDZrr VR512:$src1, VR512:$src2)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 VPTESTM instructions
//===----------------------------------------------------------------------===//
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
multiclass avx512_vptest<bits<8> opc, string OpcodeStr, RegisterClass KRC,
RegisterClass RC, X86MemOperand x86memop, PatFrag memop_frag,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
SDNode OpNode, ValueType vt> {
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
def rr : AVX512PI<opc, MRMSrcReg,
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(outs KRC:$dst), (ins RC:$src1, RC:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
[(set KRC:$dst, (OpNode (vt RC:$src1), (vt RC:$src2)))],
SSEPackedInt>, EVEX_4V;
def rm : AVX512PI<opc, MRMSrcMem,
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(outs KRC:$dst), (ins RC:$src1, x86memop:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set KRC:$dst, (OpNode (vt RC:$src1),
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
(bitconvert (memop_frag addr:$src2))))], SSEPackedInt>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm VPTESTMDZ : avx512_vptest<0x27, "vptestmd", VK16, VR512, f512mem,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv16i32, X86testm, v16i32>, T8PD, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VF>;
defm VPTESTMQZ : avx512_vptest<0x27, "vptestmq", VK8, VR512, f512mem,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8i64, X86testm, v8i64>, T8PD, EVEX_V512, VEX_W,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>;
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
let Predicates = [HasCDI] in {
defm VPTESTNMDZ : avx512_vptest<0x27, "vptestnmd", VK16, VR512, f512mem,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv16i32, X86testnm, v16i32>, T8XS, EVEX_V512,
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
EVEX_CD8<32, CD8VF>;
defm VPTESTNMQZ : avx512_vptest<0x27, "vptestnmq", VK8, VR512, f512mem,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8i64, X86testnm, v8i64>, T8XS, EVEX_V512, VEX_W,
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
EVEX_CD8<64, CD8VF>;
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def : Pat <(i16 (int_x86_avx512_mask_ptestm_d_512 (v16i32 VR512:$src1),
(v16i32 VR512:$src2), (i16 -1))),
(COPY_TO_REGCLASS (VPTESTMDZrr VR512:$src1, VR512:$src2), GR16)>;
def : Pat <(i8 (int_x86_avx512_mask_ptestm_q_512 (v8i64 VR512:$src1),
(v8i64 VR512:$src2), (i8 -1))),
AVX-512: Fixed encoding of VPTESTMQ llvm-svn: 201980
2014-02-23 22:28:35 +08:00
(COPY_TO_REGCLASS (VPTESTMQZrr VR512:$src1, VR512:$src2), GR8)>;
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Shift instructions
//===----------------------------------------------------------------------===//
multiclass avx512_shift_rmi<bits<8> opc, Format ImmFormR, Format ImmFormM,
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
string OpcodeStr, SDNode OpNode, X86VectorVTInfo _> {
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
defm ri : AVX512_maskable<opc, ImmFormR, _, (outs _.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins _.RC:$src1, u8imm:$src2), OpcodeStr,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, (i8 imm:$src2))),
" ", SSE_INTSHIFT_ITINS_P.rr>, AVX512BIi8Base, EVEX_4V;
defm mi : AVX512_maskable<opc, ImmFormM, _, (outs _.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins _.MemOp:$src1, u8imm:$src2), OpcodeStr,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
"$src2, $src1", "$src1, $src2",
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(_.VT (OpNode (_.LdFrag addr:$src1), (i8 imm:$src2))),
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
" ", SSE_INTSHIFT_ITINS_P.rm>, AVX512BIi8Base, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
multiclass avx512_shift_rrm<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
ValueType SrcVT, PatFrag bc_frag, X86VectorVTInfo _> {
// src2 is always 128-bit
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, VR128X:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, (SrcVT VR128X:$src2))),
" ", SSE_INTSHIFT_ITINS_P.rr>, AVX512BIBase, EVEX_4V;
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, i128mem:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(_.VT (OpNode _.RC:$src1, (bc_frag (loadv2i64 addr:$src2)))),
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
" ", SSE_INTSHIFT_ITINS_P.rm>, AVX512BIBase, EVEX_4V;
}
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
multiclass avx512_shift_sizes<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
ValueType SrcVT, PatFrag bc_frag, X86VectorVTInfo _> {
defm Z : avx512_shift_rrm<opc, OpcodeStr, OpNode, SrcVT, bc_frag, _>, EVEX_V512;
}
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
multiclass avx512_shift_types<bits<8> opcd, bits<8> opcq, string OpcodeStr,
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
SDNode OpNode> {
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
defm D : avx512_shift_sizes<opcd, OpcodeStr#"d", OpNode, v4i32, bc_v4i32,
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
v16i32_info>, EVEX_CD8<32, CD8VQ>;
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
defm Q : avx512_shift_sizes<opcq, OpcodeStr#"q", OpNode, v2i64, bc_v2i64,
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
v8i64_info>, EVEX_CD8<64, CD8VQ>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
defm VPSRLDZ : avx512_shift_rmi<0x72, MRM2r, MRM2m, "vpsrld", X86vsrli,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
v16i32_info>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_V512, EVEX_CD8<32, CD8VF>;
defm VPSRLQZ : avx512_shift_rmi<0x73, MRM2r, MRM2m, "vpsrlq", X86vsrli,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
v8i64_info>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>, VEX_W;
defm VPSLLDZ : avx512_shift_rmi<0x72, MRM6r, MRM6m, "vpslld", X86vshli,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
v16i32_info>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VF>;
defm VPSLLQZ : avx512_shift_rmi<0x73, MRM6r, MRM6m, "vpsllq", X86vshli,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
v8i64_info>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>, VEX_W;
defm VPSRADZ : avx512_shift_rmi<0x72, MRM4r, MRM4m, "vpsrad", X86vsrai,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
v16i32_info>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_V512, EVEX_CD8<32, CD8VF>;
defm VPSRAQZ : avx512_shift_rmi<0x72, MRM4r, MRM4m, "vpsraq", X86vsrai,
[AVX512] Add 512b masked integer shift by immediate patterns. llvm-svn: 222002
2014-11-14 23:43:00 +08:00
v8i64_info>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>, VEX_W;
[AVX512] Add 512b integer shift by variable intrinsics and patterns. llvm-svn: 222786
2014-11-26 04:41:51 +08:00
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
defm VPSLL : avx512_shift_types<0xF2, 0xF3, "vpsll", X86vshl>;
defm VPSRA : avx512_shift_types<0xE2, 0xE2, "vpsra", X86vsra>;
defm VPSRL : avx512_shift_types<0xD2, 0xD3, "vpsrl", X86vsrl>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===-------------------------------------------------------------------===//
// Variable Bit Shifts
//===-------------------------------------------------------------------===//
multiclass avx512_var_shift<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
X86VectorVTInfo _> {
defm rr : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(_.VT (OpNode _.RC:$src1, (_.VT _.RC:$src2))),
" ", SSE_INTSHIFT_ITINS_P.rr>, AVX5128IBase, EVEX_4V;
defm rm : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(_.VT (OpNode _.RC:$src1, (_.LdFrag addr:$src2))),
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
" ", SSE_INTSHIFT_ITINS_P.rm>, AVX5128IBase, EVEX_4V;
}
multiclass avx512_var_shift_sizes<bits<8> opc, string OpcodeStr, SDNode OpNode,
AVX512VLVectorVTInfo _> {
defm Z : avx512_var_shift<opc, OpcodeStr, OpNode, _.info512>, EVEX_V512;
}
multiclass avx512_var_shift_types<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
defm D : avx512_var_shift_sizes<opc, OpcodeStr#"d", OpNode,
avx512vl_i32_info>, EVEX_CD8<32, CD8VQ>;
defm Q : avx512_var_shift_sizes<opc, OpcodeStr#"q", OpNode,
avx512vl_i64_info>, EVEX_CD8<64, CD8VQ>, VEX_W;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Add support for 512b variable bit shift intrinsics. llvm-svn: 224028
2014-12-12 01:13:05 +08:00
defm VPSLLV : avx512_var_shift_types<0x47, "vpsllv", shl>;
defm VPSRAV : avx512_var_shift_types<0x46, "vpsrav", sra>;
defm VPSRLV : avx512_var_shift_types<0x45, "vpsrlv", srl>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - MOVDDUP
//===----------------------------------------------------------------------===//
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
multiclass avx512_movddup<string OpcodeStr, RegisterClass RC, ValueType VT,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
X86MemOperand x86memop, PatFrag memop_frag> {
def rr : AVX512PDI<0x12, MRMSrcReg, (outs RC:$dst), (ins RC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst, (VT (X86Movddup RC:$src)))]>, EVEX;
def rm : AVX512PDI<0x12, MRMSrcMem, (outs RC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
(VT (X86Movddup (memop_frag addr:$src))))]>, EVEX;
}
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VMOVDDUPZ : avx512_movddup<"vmovddup", VR512, v8f64, f512mem, loadv8f64>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VEX_W, EVEX_V512, EVEX_CD8<64, CD8VF>;
def : Pat<(X86Movddup (v8f64 (scalar_to_vector (loadf64 addr:$src)))),
(VMOVDDUPZrm addr:$src)>;
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
//===---------------------------------------------------------------------===//
// Replicate Single FP - MOVSHDUP and MOVSLDUP
//===---------------------------------------------------------------------===//
multiclass avx512_replicate_sfp<bits<8> op, SDNode OpNode, string OpcodeStr,
ValueType vt, RegisterClass RC, PatFrag mem_frag,
X86MemOperand x86memop> {
def rr : AVX512XSI<op, MRMSrcReg, (outs RC:$dst), (ins RC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
[(set RC:$dst, (vt (OpNode RC:$src)))]>, EVEX;
let mayLoad = 1 in
def rm : AVX512XSI<op, MRMSrcMem, (outs RC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
[(set RC:$dst, (OpNode (mem_frag addr:$src)))]>, EVEX;
}
defm VMOVSHDUPZ : avx512_replicate_sfp<0x16, X86Movshdup, "vmovshdup",
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
v16f32, VR512, loadv16f32, f512mem>, EVEX_V512,
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
EVEX_CD8<32, CD8VF>;
defm VMOVSLDUPZ : avx512_replicate_sfp<0x12, X86Movsldup, "vmovsldup",
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
v16f32, VR512, loadv16f32, f512mem>, EVEX_V512,
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
EVEX_CD8<32, CD8VF>;
def : Pat<(v16i32 (X86Movshdup VR512:$src)), (VMOVSHDUPZrr VR512:$src)>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
def : Pat<(v16i32 (X86Movshdup (loadv16i32 addr:$src))),
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
(VMOVSHDUPZrm addr:$src)>;
def : Pat<(v16i32 (X86Movsldup VR512:$src)), (VMOVSLDUPZrr VR512:$src)>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
def : Pat<(v16i32 (X86Movsldup (loadv16i32 addr:$src))),
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
(VMOVSLDUPZrm addr:$src)>;
//===----------------------------------------------------------------------===//
// Move Low to High and High to Low packed FP Instructions
//===----------------------------------------------------------------------===//
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def VMOVLHPSZrr : AVX512PSI<0x16, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src1, VR128X:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovlhps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst, (v4f32 (X86Movlhps VR128X:$src1, VR128X:$src2)))],
IIC_SSE_MOV_LH>, EVEX_4V;
def VMOVHLPSZrr : AVX512PSI<0x12, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src1, VR128X:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vmovhlps\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set VR128X:$dst, (v4f32 (X86Movhlps VR128X:$src1, VR128X:$src2)))],
IIC_SSE_MOV_LH>, EVEX_4V;
Put HasAVX512 predicate on some patterns to properly disable them when AVX512 isn't enabled. Currently it works simply because the SSE and AVX version of the same patterns are checked first in the DAG isel table. llvm-svn: 191490
2013-09-27 15:20:47 +08:00
let Predicates = [HasAVX512] in {
// MOVLHPS patterns
def : Pat<(v4i32 (X86Movlhps VR128X:$src1, VR128X:$src2)),
(VMOVLHPSZrr VR128X:$src1, VR128X:$src2)>;
def : Pat<(v2i64 (X86Movlhps VR128X:$src1, VR128X:$src2)),
(VMOVLHPSZrr (v2i64 VR128X:$src1), VR128X:$src2)>;
// MOVHLPS patterns
def : Pat<(v4i32 (X86Movhlps VR128X:$src1, VR128X:$src2)),
(VMOVHLPSZrr VR128X:$src1, VR128X:$src2)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// FMA - Fused Multiply Operations
//
[AVX512] Introduce fma3p_forms from AVX This multiclass generates the different forms: 213, 231, 132 in AVX. 132 in AVX512 is a separate class but I am planning to use this same multiclass to generate 231 relying on the nice the null_frag trick from AVX to disable codegen pattern for 231. No functionality change, no change in X86.td.expanded except for the different instruction definition names. llvm-svn: 220539
2014-10-24 08:02:55 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Constraints = "$src1 = $dst" in {
[AVX512] Introduce fma3p_forms from AVX This multiclass generates the different forms: 213, 231, 132 in AVX. 132 in AVX512 is a separate class but I am planning to use this same multiclass to generate 231 relying on the nice the null_frag trick from AVX to disable codegen pattern for 231. No functionality change, no change in X86.td.expanded except for the different instruction definition names. llvm-svn: 220539
2014-10-24 08:02:55 +08:00
// Omitting the parameter OpNode (= null_frag) disables ISel pattern matching.
multiclass avx512_fma3p_rm<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
SDPatternOperator OpNode = null_frag> {
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm r: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
(ins _.RC:$src2, _.RC:$src3),
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
OpcodeStr, "$src3, $src2", "$src2, $src3",
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
(_.VT (OpNode _.RC:$src1, _.RC:$src2, _.RC:$src3))>,
[AVX512] Add masking variant for the FMA instructions This change further evolves the base class AVX512_masking in order to make it suitable for the masking variants of the FMA instructions. Besides AVX512_masking there is now a new base class that instructions including FMAs can use: AVX512_masking_3src. With three-source (destructive) instructions one of the sources is already tied to the destination. This difference from AVX512_masking is captured by this new class. The common bits between _masking and _masking_3src are broken out into a new super class called AVX512_masking_common. As with valign, there is some corresponding restructuring of the underlying format classes. The idea is the same we want to derive from two classes essentially: one providing the format bits and another format-independent multiclass supplying the various masking and non-masking instruction variants. Existing fma tests in avx512-fma*.ll provide coverage here for the non-masking variants. For masking, the next patches in the series will add intrinsics and intrinsic tests. For AVX512_masking_3src to work, the (ins ...) dag has to be passed *without* the leading source operand that is tied to dst ($src1). This is necessary to properly construct the (ins ...) for the different variants. For the record, I did check that if $src is mistakenly included, you do get a fairly intuitive error message from the tablegen backend. Part of <rdar://problem/17688758> llvm-svn: 215660
2014-08-15 01:13:19 +08:00
AVX512FMA3Base;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm m: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.MemOp:$src3),
OpcodeStr, "$src3, $src2", "$src2, $src3",
(_.VT (OpNode _.RC:$src1, _.RC:$src2, (_.LdFrag addr:$src3)))>,
AVX512FMA3Base;
defm mb: AVX512_maskable_3src<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.ScalarMemOp:$src3),
OpcodeStr, !strconcat("${src3}", _.BroadcastStr,", $src2"), !strconcat("$src2, ${src3}", _.BroadcastStr ),
(OpNode _.RC:$src1, _.RC:$src2,(_.VT (X86VBroadcast (_.ScalarLdFrag addr:$src3))))>,
AVX512FMA3Base, EVEX_B;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
} // Constraints = "$src1 = $dst"
AVX-512: Added FMA intrinsics with rounding mode By Asaf Badouh and Elena Demikhovsky Added special nodes for rounding: FMADD_RND, FMSUB_RND.. It will prevent merge between nodes with rounding and other standard nodes. llvm-svn: 227303
2015-01-28 18:21:27 +08:00
let Constraints = "$src1 = $dst" in {
// Omitting the parameter OpNode (= null_frag) disables ISel pattern matching.
multiclass avx512_fma3_round_rrb<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
SDPatternOperator OpNode> {
defm rb: AVX512_maskable_3src<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src2, _.RC:$src3, AVX512RC:$rc),
OpcodeStr, "$rc, $src3, $src2", "$src2, $src3, $rc",
(_.VT ( OpNode _.RC:$src1, _.RC:$src2, _.RC:$src3, (i32 imm:$rc)))>,
AVX512FMA3Base, EVEX_B, EVEX_RC;
}
} // Constraints = "$src1 = $dst"
multiclass avx512_fma3_round_forms<bits<8> opc213, string OpcodeStr,
X86VectorVTInfo VTI, SDPatternOperator OpNode> {
defm v213r : avx512_fma3_round_rrb<opc213, !strconcat(OpcodeStr, "213", VTI.Suffix),
VTI, OpNode>, EVEX_CD8<VTI.EltSize, CD8VF>;
}
[AVX512] FMA support for the 231 variants This is asm/diasm-only support, similar to AVX. For ISeling the register variant, they are no different from 213 other than whether the multiplication or the addition operand is destructed. For ISeling the memory variant, i.e. to fold a load, they are no different than the 132 variant. The addition operand (op3) in both cases can come from memory. Again the ony difference is which operand is destructed. There could be a post-RA pass that would convert a 213 or 132 into a 231. Part of <rdar://problem/17082571> llvm-svn: 220540
2014-10-24 08:03:00 +08:00
multiclass avx512_fma3p_forms<bits<8> opc213, bits<8> opc231,
[AVX512] Introduce fma3p_forms from AVX This multiclass generates the different forms: 213, 231, 132 in AVX. 132 in AVX512 is a separate class but I am planning to use this same multiclass to generate 231 relying on the nice the null_frag trick from AVX to disable codegen pattern for 231. No functionality change, no change in X86.td.expanded except for the different instruction definition names. llvm-svn: 220539
2014-10-24 08:02:55 +08:00
string OpcodeStr, X86VectorVTInfo VTI,
SDPatternOperator OpNode> {
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm v213r : avx512_fma3p_rm<opc213, !strconcat(OpcodeStr, "213", VTI.Suffix),
VTI, OpNode>, EVEX_CD8<VTI.EltSize, CD8VF>;
[AVX512] FMA support for the 231 variants This is asm/diasm-only support, similar to AVX. For ISeling the register variant, they are no different from 213 other than whether the multiplication or the addition operand is destructed. For ISeling the memory variant, i.e. to fold a load, they are no different than the 132 variant. The addition operand (op3) in both cases can come from memory. Again the ony difference is which operand is destructed. There could be a post-RA pass that would convert a 213 or 132 into a 231. Part of <rdar://problem/17082571> llvm-svn: 220540
2014-10-24 08:03:00 +08:00
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm v231r : avx512_fma3p_rm<opc231, !strconcat(OpcodeStr, "231", VTI.Suffix),
VTI>, EVEX_CD8<VTI.EltSize, CD8VF>;
[AVX512] Introduce fma3p_forms from AVX This multiclass generates the different forms: 213, 231, 132 in AVX. 132 in AVX512 is a separate class but I am planning to use this same multiclass to generate 231 relying on the nice the null_frag trick from AVX to disable codegen pattern for 231. No functionality change, no change in X86.td.expanded except for the different instruction definition names. llvm-svn: 220539
2014-10-24 08:02:55 +08:00
}
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
multiclass avx512_fma3p<bits<8> opc213, bits<8> opc231,
string OpcodeStr,
AVX-512: Added FMA intrinsics with rounding mode By Asaf Badouh and Elena Demikhovsky Added special nodes for rounding: FMADD_RND, FMSUB_RND.. It will prevent merge between nodes with rounding and other standard nodes. llvm-svn: 227303
2015-01-28 18:21:27 +08:00
SDPatternOperator OpNode,
SDPatternOperator OpNodeRnd> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let ExeDomain = SSEPackedSingle in {
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm NAME##PSZ : avx512_fma3p_forms<opc213, opc231, OpcodeStr,
AVX-512: Added FMA intrinsics with rounding mode By Asaf Badouh and Elena Demikhovsky Added special nodes for rounding: FMADD_RND, FMSUB_RND.. It will prevent merge between nodes with rounding and other standard nodes. llvm-svn: 227303
2015-01-28 18:21:27 +08:00
v16f32_info, OpNode>,
avx512_fma3_round_forms<opc213, OpcodeStr,
v16f32_info, OpNodeRnd>, EVEX_V512;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm NAME##PSZ256 : avx512_fma3p_forms<opc213, opc231, OpcodeStr,
v8f32x_info, OpNode>, EVEX_V256;
defm NAME##PSZ128 : avx512_fma3p_forms<opc213, opc231, OpcodeStr,
v4f32x_info, OpNode>, EVEX_V128;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let ExeDomain = SSEPackedDouble in {
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm NAME##PDZ : avx512_fma3p_forms<opc213, opc231, OpcodeStr,
AVX-512: Added FMA intrinsics with rounding mode By Asaf Badouh and Elena Demikhovsky Added special nodes for rounding: FMADD_RND, FMSUB_RND.. It will prevent merge between nodes with rounding and other standard nodes. llvm-svn: 227303
2015-01-28 18:21:27 +08:00
v8f64_info, OpNode>,
avx512_fma3_round_forms<opc213, OpcodeStr,
v8f64_info, OpNodeRnd>, EVEX_V512, VEX_W;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm NAME##PDZ256 : avx512_fma3p_forms<opc213, opc231, OpcodeStr,
v4f64x_info, OpNode>, EVEX_V256, VEX_W;
defm NAME##PDZ128 : avx512_fma3p_forms<opc213, opc231, OpcodeStr,
v2f64x_info, OpNode>, EVEX_V128, VEX_W;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added FMA intrinsics with rounding mode By Asaf Badouh and Elena Demikhovsky Added special nodes for rounding: FMADD_RND, FMSUB_RND.. It will prevent merge between nodes with rounding and other standard nodes. llvm-svn: 227303
2015-01-28 18:21:27 +08:00
defm VFMADD : avx512_fma3p<0xA8, 0xB8, "vfmadd", X86Fmadd, X86FmaddRnd>;
defm VFMSUB : avx512_fma3p<0xAA, 0xBA, "vfmsub", X86Fmsub, X86FmsubRnd>;
defm VFMADDSUB : avx512_fma3p<0xA6, 0xB6, "vfmaddsub", X86Fmaddsub, X86FmaddsubRnd>;
defm VFMSUBADD : avx512_fma3p<0xA7, 0xB7, "vfmsubadd", X86Fmsubadd, X86FmsubaddRnd>;
defm VFNMADD : avx512_fma3p<0xAC, 0xBC, "vfnmadd", X86Fnmadd, X86FnmaddRnd>;
defm VFNMSUB : avx512_fma3p<0xAE, 0xBE, "vfnmsub", X86Fnmsub, X86FnmsubRnd>;
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Constraints = "$src1 = $dst" in {
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
multiclass avx512_fma3p_m132<bits<8> opc, string OpcodeStr, SDNode OpNode,
X86VectorVTInfo _> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
def m: AVX512FMA3<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src3, _.MemOp:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src2, $src3, $dst|$dst, $src3, $src2}"),
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
[(set _.RC:$dst, (_.VT (OpNode _.RC:$src1, (_.LdFrag addr:$src2),
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
_.RC:$src3)))]>;
def mb: AVX512FMA3<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src3, _.ScalarMemOp:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src2}", _.BroadcastStr,
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
", $src3, $dst|$dst, $src3, ${src2}", _.BroadcastStr, "}"),
[(set _.RC:$dst,
(OpNode _.RC:$src1, (_.VT (X86VBroadcast
(_.ScalarLdFrag addr:$src2))),
_.RC:$src3))]>, EVEX_B;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
} // Constraints = "$src1 = $dst"
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
multiclass avx512_fma3p_m132_f<bits<8> opc,
string OpcodeStr,
SDNode OpNode> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let ExeDomain = SSEPackedSingle in {
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm NAME##PSZ : avx512_fma3p_m132<opc, OpcodeStr##ps,
OpNode,v16f32_info>, EVEX_V512, EVEX_CD8<32, CD8VF>;
defm NAME##PSZ256 : avx512_fma3p_m132<opc, OpcodeStr##ps,
OpNode, v8f32x_info>, EVEX_V256, EVEX_CD8<32, CD8VF>;
defm NAME##PSZ128 : avx512_fma3p_m132<opc, OpcodeStr##ps,
OpNode, v4f32x_info>, EVEX_V128, EVEX_CD8<32, CD8VF>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let ExeDomain = SSEPackedDouble in {
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm NAME##PDZ : avx512_fma3p_m132<opc, OpcodeStr##pd,
OpNode, v8f64_info>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VF>;
defm NAME##PDZ256 : avx512_fma3p_m132<opc, OpcodeStr##pd,
OpNode, v4f64x_info>, EVEX_V256, VEX_W, EVEX_CD8<32, CD8VF>;
defm NAME##PDZ128 : avx512_fma3p_m132<opc, OpcodeStr##pd,
OpNode, v2f64x_info>, EVEX_V128, VEX_W, EVEX_CD8<32, CD8VF>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added FMA instructions, intrinsics an tests for KNL and SKX targets by Asaf Badouh http://reviews.llvm.org/D6456 llvm-svn: 224764
2014-12-23 18:30:39 +08:00
defm VFMADD132 : avx512_fma3p_m132_f<0x98, "vfmadd132", X86Fmadd>;
defm VFMSUB132 : avx512_fma3p_m132_f<0x9A, "vfmsub132", X86Fmsub>;
defm VFMADDSUB132 : avx512_fma3p_m132_f<0x96, "vfmaddsub132", X86Fmaddsub>;
defm VFMSUBADD132 : avx512_fma3p_m132_f<0x97, "vfmsubadd132", X86Fmsubadd>;
defm VFNMADD132 : avx512_fma3p_m132_f<0x9C, "vfnmadd132", X86Fnmadd>;
defm VFNMSUB132 : avx512_fma3p_m132_f<0x9E, "vfnmsub132", X86Fnmsub>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
// Scalar FMA
let Constraints = "$src1 = $dst" in {
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
multiclass avx512_fma3s_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
RegisterClass RC, ValueType OpVT,
X86MemOperand x86memop, Operand memop,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
PatFrag mem_frag> {
let isCommutable = 1 in
def r : AVX512FMA3<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src1, RC:$src2, RC:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
(OpVT (OpNode RC:$src2, RC:$src1, RC:$src3)))]>;
let mayLoad = 1 in
def m : AVX512FMA3<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, RC:$src2, f128mem:$src3),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst,
(OpVT (OpNode RC:$src2, RC:$src1,
(mem_frag addr:$src3))))]>;
}
} // Constraints = "$src1 = $dst"
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VFMADDSSZ : avx512_fma3s_rm<0xA9, "vfmadd213ss", X86Fmadd, FR32X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f32, f32mem, ssmem, loadf32>, EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VFMADDSDZ : avx512_fma3s_rm<0xA9, "vfmadd213sd", X86Fmadd, FR64X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f64, f64mem, sdmem, loadf64>, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VFMSUBSSZ : avx512_fma3s_rm<0xAB, "vfmsub213ss", X86Fmsub, FR32X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f32, f32mem, ssmem, loadf32>, EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VFMSUBSDZ : avx512_fma3s_rm<0xAB, "vfmsub213sd", X86Fmsub, FR64X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f64, f64mem, sdmem, loadf64>, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VFNMADDSSZ : avx512_fma3s_rm<0xAD, "vfnmadd213ss", X86Fnmadd, FR32X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f32, f32mem, ssmem, loadf32>, EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VFNMADDSDZ : avx512_fma3s_rm<0xAD, "vfnmadd213sd", X86Fnmadd, FR64X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f64, f64mem, sdmem, loadf64>, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VFNMSUBSSZ : avx512_fma3s_rm<0xAF, "vfnmsub213ss", X86Fnmsub, FR32X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f32, f32mem, ssmem, loadf32>, EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VFNMSUBSDZ : avx512_fma3s_rm<0xAF, "vfnmsub213sd", X86Fnmsub, FR64X,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
f64, f64mem, sdmem, loadf64>, VEX_W, EVEX_CD8<64, CD8VT1>;
//===----------------------------------------------------------------------===//
// AVX-512 Scalar convert from sign integer to float/double
//===----------------------------------------------------------------------===//
multiclass avx512_vcvtsi<bits<8> opc, RegisterClass SrcRC, RegisterClass DstRC,
X86MemOperand x86memop, string asm> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : SI<opc, MRMSrcReg, (outs DstRC:$dst), (ins DstRC:$src1, SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $src1, $dst|$dst, $src1, $src}"), []>,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in
def rm : SI<opc, MRMSrcMem, (outs DstRC:$dst),
(ins DstRC:$src1, x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $src1, $dst|$dst, $src1, $src}"), []>,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_4V;
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
} // hasSideEffects = 0
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
Add missing HasAVX512 predicate. This was only working because AVX had cheaper rules in all cases. I'm sure there are other places in this file where predicates are missing. llvm-svn: 192276
2013-10-09 13:11:10 +08:00
let Predicates = [HasAVX512] in {
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VCVTSI2SSZ : avx512_vcvtsi<0x2A, GR32, FR32X, i32mem, "cvtsi2ss{l}">,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
XS, VEX_LIG, EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VCVTSI642SSZ : avx512_vcvtsi<0x2A, GR64, FR32X, i64mem, "cvtsi2ss{q}">,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
XS, VEX_W, VEX_LIG, EVEX_CD8<64, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VCVTSI2SDZ : avx512_vcvtsi<0x2A, GR32, FR64X, i32mem, "cvtsi2sd{l}">,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
XD, VEX_LIG, EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VCVTSI642SDZ : avx512_vcvtsi<0x2A, GR64, FR64X, i64mem, "cvtsi2sd{q}">,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
XD, VEX_W, VEX_LIG, EVEX_CD8<64, CD8VT1>;
def : Pat<(f32 (sint_to_fp (loadi32 addr:$src))),
(VCVTSI2SSZrm (f32 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f32 (sint_to_fp (loadi64 addr:$src))),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
(VCVTSI642SSZrm (f32 (IMPLICIT_DEF)), addr:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(f64 (sint_to_fp (loadi32 addr:$src))),
(VCVTSI2SDZrm (f64 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f64 (sint_to_fp (loadi64 addr:$src))),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
(VCVTSI642SDZrm (f64 (IMPLICIT_DEF)), addr:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(f32 (sint_to_fp GR32:$src)),
(VCVTSI2SSZrr (f32 (IMPLICIT_DEF)), GR32:$src)>;
def : Pat<(f32 (sint_to_fp GR64:$src)),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
(VCVTSI642SSZrr (f32 (IMPLICIT_DEF)), GR64:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(f64 (sint_to_fp GR32:$src)),
(VCVTSI2SDZrr (f64 (IMPLICIT_DEF)), GR32:$src)>;
def : Pat<(f64 (sint_to_fp GR64:$src)),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
(VCVTSI642SDZrr (f64 (IMPLICIT_DEF)), GR64:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VCVTUSI2SSZ : avx512_vcvtsi<0x7B, GR32, FR32X, i32mem, "cvtusi2ss{l}">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XS, VEX_LIG, EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VCVTUSI642SSZ : avx512_vcvtsi<0x7B, GR64, FR32X, i64mem, "cvtusi2ss{q}">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XS, VEX_W, VEX_LIG, EVEX_CD8<64, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VCVTUSI2SDZ : avx512_vcvtsi<0x7B, GR32, FR64X, i32mem, "cvtusi2sd{l}">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XD, VEX_LIG, EVEX_CD8<32, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
defm VCVTUSI642SDZ : avx512_vcvtsi<0x7B, GR64, FR64X, i64mem, "cvtusi2sd{q}">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XD, VEX_W, VEX_LIG, EVEX_CD8<64, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
def : Pat<(f32 (uint_to_fp (loadi32 addr:$src))),
(VCVTUSI2SSZrm (f32 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f32 (uint_to_fp (loadi64 addr:$src))),
(VCVTUSI642SSZrm (f32 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f64 (uint_to_fp (loadi32 addr:$src))),
(VCVTUSI2SDZrm (f64 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f64 (uint_to_fp (loadi64 addr:$src))),
(VCVTUSI642SDZrm (f64 (IMPLICIT_DEF)), addr:$src)>;
def : Pat<(f32 (uint_to_fp GR32:$src)),
(VCVTUSI2SSZrr (f32 (IMPLICIT_DEF)), GR32:$src)>;
def : Pat<(f32 (uint_to_fp GR64:$src)),
(VCVTUSI642SSZrr (f32 (IMPLICIT_DEF)), GR64:$src)>;
def : Pat<(f64 (uint_to_fp GR32:$src)),
(VCVTUSI2SDZrr (f64 (IMPLICIT_DEF)), GR32:$src)>;
def : Pat<(f64 (uint_to_fp GR64:$src)),
(VCVTUSI642SDZrr (f64 (IMPLICIT_DEF)), GR64:$src)>;
Add missing HasAVX512 predicate. This was only working because AVX had cheaper rules in all cases. I'm sure there are other places in this file where predicates are missing. llvm-svn: 192276
2013-10-09 13:11:10 +08:00
}
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Scalar convert from float/double to integer
//===----------------------------------------------------------------------===//
multiclass avx512_cvt_s_int<bits<8> opc, RegisterClass SrcRC, RegisterClass DstRC,
Intrinsic Int, Operand memop, ComplexPattern mem_cpat,
string asm> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
def rr : SI<opc, MRMSrcReg, (outs DstRC:$dst), (ins SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
[(set DstRC:$dst, (Int SrcRC:$src))]>, EVEX, VEX_LIG,
Requires<[HasAVX512]>;
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
let mayLoad = 1 in
def rm : SI<opc, MRMSrcMem, (outs DstRC:$dst), (ins memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"), []>, EVEX, VEX_LIG,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
Requires<[HasAVX512]>;
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
} // hasSideEffects = 0
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
}
let Predicates = [HasAVX512] in {
// Convert float/double to signed/unsigned int 32/64
defm VCVTSS2SIZ: avx512_cvt_s_int<0x2D, VR128X, GR32, int_x86_sse_cvtss2si,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
ssmem, sse_load_f32, "cvtss2si">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XS, EVEX_CD8<32, CD8VT1>;
defm VCVTSS2SI64Z: avx512_cvt_s_int<0x2D, VR128X, GR64, int_x86_sse_cvtss2si64,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
ssmem, sse_load_f32, "cvtss2si">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XS, VEX_W, EVEX_CD8<32, CD8VT1>;
defm VCVTSS2USIZ: avx512_cvt_s_int<0x79, VR128X, GR32, int_x86_avx512_cvtss2usi,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
ssmem, sse_load_f32, "cvtss2usi">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XS, EVEX_CD8<32, CD8VT1>;
defm VCVTSS2USI64Z: avx512_cvt_s_int<0x79, VR128X, GR64,
int_x86_avx512_cvtss2usi64, ssmem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
sse_load_f32, "cvtss2usi">, XS, VEX_W,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<32, CD8VT1>;
defm VCVTSD2SIZ: avx512_cvt_s_int<0x2D, VR128X, GR32, int_x86_sse2_cvtsd2si,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
sdmem, sse_load_f64, "cvtsd2si">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XD, EVEX_CD8<64, CD8VT1>;
defm VCVTSD2SI64Z: avx512_cvt_s_int<0x2D, VR128X, GR64, int_x86_sse2_cvtsd2si64,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
sdmem, sse_load_f64, "cvtsd2si">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XD, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VCVTSD2USIZ: avx512_cvt_s_int<0x79, VR128X, GR32, int_x86_avx512_cvtsd2usi,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
sdmem, sse_load_f64, "cvtsd2usi">,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
XD, EVEX_CD8<64, CD8VT1>;
defm VCVTSD2USI64Z: avx512_cvt_s_int<0x79, VR128X, GR64,
int_x86_avx512_cvtsd2usi64, sdmem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
sse_load_f64, "cvtsd2usi">, XD, VEX_W,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<64, CD8VT1>;
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
let isCodeGenOnly = 1 in {
defm Int_VCVTSI2SSZ : sse12_cvt_sint_3addr<0x2A, GR32, VR128X,
int_x86_sse_cvtsi2ss, i32mem, loadi32, "cvtsi2ss{l}",
SSE_CVT_Scalar, 0>, XS, EVEX_4V;
defm Int_VCVTSI2SS64Z : sse12_cvt_sint_3addr<0x2A, GR64, VR128X,
int_x86_sse_cvtsi642ss, i64mem, loadi64, "cvtsi2ss{q}",
SSE_CVT_Scalar, 0>, XS, EVEX_4V, VEX_W;
defm Int_VCVTSI2SDZ : sse12_cvt_sint_3addr<0x2A, GR32, VR128X,
int_x86_sse2_cvtsi2sd, i32mem, loadi32, "cvtsi2sd{l}",
SSE_CVT_Scalar, 0>, XD, EVEX_4V;
defm Int_VCVTSI2SD64Z : sse12_cvt_sint_3addr<0x2A, GR64, VR128X,
int_x86_sse2_cvtsi642sd, i64mem, loadi64, "cvtsi2sd{q}",
SSE_CVT_Scalar, 0>, XD, EVEX_4V, VEX_W;
defm Int_VCVTUSI2SSZ : sse12_cvt_sint_3addr<0x2A, GR32, VR128X,
int_x86_avx512_cvtusi2ss, i32mem, loadi32, "cvtusi2ss{l}",
SSE_CVT_Scalar, 0>, XS, EVEX_4V;
defm Int_VCVTUSI2SS64Z : sse12_cvt_sint_3addr<0x2A, GR64, VR128X,
int_x86_avx512_cvtusi642ss, i64mem, loadi64, "cvtusi2ss{q}",
SSE_CVT_Scalar, 0>, XS, EVEX_4V, VEX_W;
defm Int_VCVTUSI2SDZ : sse12_cvt_sint_3addr<0x2A, GR32, VR128X,
int_x86_avx512_cvtusi2sd, i32mem, loadi32, "cvtusi2sd{l}",
SSE_CVT_Scalar, 0>, XD, EVEX_4V;
defm Int_VCVTUSI2SD64Z : sse12_cvt_sint_3addr<0x2A, GR64, VR128X,
int_x86_avx512_cvtusi642sd, i64mem, loadi64, "cvtusi2sd{q}",
SSE_CVT_Scalar, 0>, XD, EVEX_4V, VEX_W;
} // isCodeGenOnly = 1
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
// Convert float/double to signed/unsigned int 32/64 with truncation
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
let isCodeGenOnly = 1 in {
defm Int_VCVTTSS2SIZ : avx512_cvt_s_int<0x2C, VR128X, GR32, int_x86_sse_cvttss2si,
ssmem, sse_load_f32, "cvttss2si">,
XS, EVEX_CD8<32, CD8VT1>;
defm Int_VCVTTSS2SI64Z : avx512_cvt_s_int<0x2C, VR128X, GR64,
int_x86_sse_cvttss2si64, ssmem, sse_load_f32,
"cvttss2si">, XS, VEX_W,
EVEX_CD8<32, CD8VT1>;
defm Int_VCVTTSD2SIZ : avx512_cvt_s_int<0x2C, VR128X, GR32, int_x86_sse2_cvttsd2si,
sdmem, sse_load_f64, "cvttsd2si">, XD,
EVEX_CD8<64, CD8VT1>;
defm Int_VCVTTSD2SI64Z : avx512_cvt_s_int<0x2C, VR128X, GR64,
int_x86_sse2_cvttsd2si64, sdmem, sse_load_f64,
"cvttsd2si">, XD, VEX_W,
EVEX_CD8<64, CD8VT1>;
defm Int_VCVTTSS2USIZ : avx512_cvt_s_int<0x78, VR128X, GR32,
int_x86_avx512_cvttss2usi, ssmem, sse_load_f32,
"cvttss2usi">, XS, EVEX_CD8<32, CD8VT1>;
defm Int_VCVTTSS2USI64Z : avx512_cvt_s_int<0x78, VR128X, GR64,
int_x86_avx512_cvttss2usi64, ssmem,
sse_load_f32, "cvttss2usi">, XS, VEX_W,
EVEX_CD8<32, CD8VT1>;
defm Int_VCVTTSD2USIZ : avx512_cvt_s_int<0x78, VR128X, GR32,
int_x86_avx512_cvttsd2usi,
sdmem, sse_load_f64, "cvttsd2usi">, XD,
EVEX_CD8<64, CD8VT1>;
defm Int_VCVTTSD2USI64Z : avx512_cvt_s_int<0x78, VR128X, GR64,
int_x86_avx512_cvttsd2usi64, sdmem,
sse_load_f64, "cvttsd2usi">, XD, VEX_W,
EVEX_CD8<64, CD8VT1>;
} // isCodeGenOnly = 1
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
multiclass avx512_cvt_s<bits<8> opc, RegisterClass SrcRC, RegisterClass DstRC,
SDNode OpNode, X86MemOperand x86memop, PatFrag ld_frag,
string asm> {
def rr : SI<opc, MRMSrcReg, (outs DstRC:$dst), (ins SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
[(set DstRC:$dst, (OpNode SrcRC:$src))]>, EVEX;
def rm : SI<opc, MRMSrcMem, (outs DstRC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
[(set DstRC:$dst, (OpNode (ld_frag addr:$src)))]>, EVEX;
}
defm VCVTTSS2SIZ : avx512_cvt_s<0x2C, FR32X, GR32, fp_to_sint, f32mem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
loadf32, "cvttss2si">, XS,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<32, CD8VT1>;
defm VCVTTSS2USIZ : avx512_cvt_s<0x78, FR32X, GR32, fp_to_uint, f32mem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
loadf32, "cvttss2usi">, XS,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<32, CD8VT1>;
defm VCVTTSS2SI64Z : avx512_cvt_s<0x2C, FR32X, GR64, fp_to_sint, f32mem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
loadf32, "cvttss2si">, XS, VEX_W,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<32, CD8VT1>;
defm VCVTTSS2USI64Z : avx512_cvt_s<0x78, FR32X, GR64, fp_to_uint, f32mem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
loadf32, "cvttss2usi">, XS, VEX_W,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<32, CD8VT1>;
defm VCVTTSD2SIZ : avx512_cvt_s<0x2C, FR64X, GR32, fp_to_sint, f64mem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
loadf64, "cvttsd2si">, XD,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<64, CD8VT1>;
defm VCVTTSD2USIZ : avx512_cvt_s<0x78, FR64X, GR32, fp_to_uint, f64mem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
loadf64, "cvttsd2usi">, XD,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<64, CD8VT1>;
defm VCVTTSD2SI64Z : avx512_cvt_s<0x2C, FR64X, GR64, fp_to_sint, f64mem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
loadf64, "cvttsd2si">, XD, VEX_W,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<64, CD8VT1>;
defm VCVTTSD2USI64Z : avx512_cvt_s<0x78, FR64X, GR64, fp_to_uint, f64mem,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
loadf64, "cvttsd2usi">, XD, VEX_W,
AVX-512: added scalar convert instructions and intrinsics. Fixed load folding in VPERM2I instruction. llvm-svn: 192063
2013-10-06 21:11:09 +08:00
EVEX_CD8<64, CD8VT1>;
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
} // HasAVX512
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Convert form float to double and back
//===----------------------------------------------------------------------===//
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def VCVTSS2SDZrr : AVX512XSI<0x5A, MRMSrcReg, (outs FR64X:$dst),
(ins FR32X:$src1, FR32X:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vcvtss2sd\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX_4V, VEX_LIG, Sched<[WriteCvtF2F]>;
let mayLoad = 1 in
def VCVTSS2SDZrm : AVX512XSI<0x5A, MRMSrcMem, (outs FR64X:$dst),
(ins FR32X:$src1, f32mem:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vcvtss2sd\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX_4V, VEX_LIG, Sched<[WriteCvtF2FLd, ReadAfterLd]>,
EVEX_CD8<32, CD8VT1>;
// Convert scalar double to scalar single
def VCVTSD2SSZrr : AVX512XDI<0x5A, MRMSrcReg, (outs FR32X:$dst),
(ins FR64X:$src1, FR64X:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vcvtsd2ss\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX_4V, VEX_LIG, VEX_W, Sched<[WriteCvtF2F]>;
let mayLoad = 1 in
def VCVTSD2SSZrm : AVX512XDI<0x5A, MRMSrcMem, (outs FR32X:$dst),
(ins FR64X:$src1, f64mem:$src2),
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"vcvtsd2ss\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX_4V, VEX_LIG, VEX_W,
Sched<[WriteCvtF2FLd, ReadAfterLd]>, EVEX_CD8<64, CD8VT1>;
}
def : Pat<(f64 (fextend FR32X:$src)), (VCVTSS2SDZrr FR32X:$src, FR32X:$src)>,
Requires<[HasAVX512]>;
def : Pat<(fextend (loadf32 addr:$src)),
(VCVTSS2SDZrm (f32 (IMPLICIT_DEF)), addr:$src)>, Requires<[HasAVX512]>;
def : Pat<(extloadf32 addr:$src),
(VCVTSS2SDZrm (f32 (IMPLICIT_DEF)), addr:$src)>,
Requires<[HasAVX512, OptForSize]>;
def : Pat<(extloadf32 addr:$src),
(VCVTSS2SDZrr (f32 (IMPLICIT_DEF)), (VMOVSSZrm addr:$src))>,
Requires<[HasAVX512, OptForSpeed]>;
def : Pat<(f32 (fround FR64X:$src)), (VCVTSD2SSZrr FR64X:$src, FR64X:$src)>,
Requires<[HasAVX512]>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
multiclass avx512_vcvt_fp_with_rc<bits<8> opc, string asm, RegisterClass SrcRC,
RegisterClass DstRC, SDNode OpNode, PatFrag mem_frag,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
X86MemOperand x86memop, ValueType OpVT, ValueType InVT,
Domain d> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : AVX512PI<opc, MRMSrcReg, (outs DstRC:$dst), (ins SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set DstRC:$dst,
(OpVT (OpNode (InVT SrcRC:$src))))], d>, EVEX;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def rrb : AVX512PI<opc, MRMSrcReg, (outs DstRC:$dst), (ins SrcRC:$src, AVX512RC:$rc),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$rc, $src, $dst|$dst, $src, $rc}"),
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
[], d>, EVEX, EVEX_B, EVEX_RC;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in
def rm : AVX512PI<opc, MRMSrcMem, (outs DstRC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set DstRC:$dst,
(OpVT (OpNode (InVT (bitconvert (mem_frag addr:$src))))))], d>, EVEX;
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
} // hasSideEffects = 0
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
multiclass avx512_vcvt_fp<bits<8> opc, string asm, RegisterClass SrcRC,
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
RegisterClass DstRC, SDNode OpNode, PatFrag mem_frag,
X86MemOperand x86memop, ValueType OpVT, ValueType InVT,
Domain d> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def rr : AVX512PI<opc, MRMSrcReg, (outs DstRC:$dst), (ins SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[(set DstRC:$dst,
(OpVT (OpNode (InVT SrcRC:$src))))], d>, EVEX;
let mayLoad = 1 in
def rm : AVX512PI<opc, MRMSrcMem, (outs DstRC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[(set DstRC:$dst,
(OpVT (OpNode (InVT (bitconvert (mem_frag addr:$src))))))], d>, EVEX;
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
} // hasSideEffects = 0
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
}
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
defm VCVTPD2PSZ : avx512_vcvt_fp_with_rc<0x5A, "vcvtpd2ps", VR512, VR256X, fround,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8f64, f512mem, v8f32, v8f64,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
SSEPackedSingle>, EVEX_V512, VEX_W, PD,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>;
defm VCVTPS2PDZ : avx512_vcvt_fp<0x5A, "vcvtps2pd", VR256X, VR512, fextend,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv4f64, f256mem, v8f64, v8f32,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
SSEPackedDouble>, EVEX_V512, PS,
Simplify some x86 format classes and remove some ambiguities in their application. llvm-svn: 200608
2014-02-01 16:17:56 +08:00
EVEX_CD8<32, CD8VH>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (extloadv8f32 addr:$src)),
(VCVTPS2PDZrm addr:$src)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: added intrinsic vcvtpd2ps (with rounding mode and without) llvm-svn: 198593
2014-01-06 16:45:54 +08:00
def : Pat<(v8f32 (int_x86_avx512_mask_cvtpd2ps_512 (v8f64 VR512:$src),
(bc_v8f32(v8i32 immAllZerosV)), (i8 -1), (i32 FROUND_CURRENT))),
(VCVTPD2PSZrr VR512:$src)>;
def : Pat<(v8f32 (int_x86_avx512_mask_cvtpd2ps_512 (v8f64 VR512:$src),
(bc_v8f32(v8i32 immAllZerosV)), (i8 -1), imm:$rc)),
(VCVTPD2PSZrrb VR512:$src, imm:$rc)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 Vector convert from sign integer to float/double
//===----------------------------------------------------------------------===//
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
defm VCVTDQ2PSZ : avx512_vcvt_fp_with_rc<0x5B, "vcvtdq2ps", VR512, VR512, sint_to_fp,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8i64, i512mem, v16f32, v16i32,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
SSEPackedSingle>, EVEX_V512, PS,
Simplify some x86 format classes and remove some ambiguities in their application. llvm-svn: 200608
2014-02-01 16:17:56 +08:00
EVEX_CD8<32, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm VCVTDQ2PDZ : avx512_vcvt_fp<0xE6, "vcvtdq2pd", VR256X, VR512, sint_to_fp,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv4i64, i256mem, v8f64, v8i32,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
SSEPackedDouble>, EVEX_V512, XS,
EVEX_CD8<32, CD8VH>;
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
defm VCVTTPS2DQZ : avx512_vcvt_fp<0x5B, "vcvttps2dq", VR512, VR512, fp_to_sint,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv16f32, f512mem, v16i32, v16f32,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
SSEPackedSingle>, EVEX_V512, XS,
EVEX_CD8<32, CD8VF>;
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
defm VCVTTPD2DQZ : avx512_vcvt_fp<0xE6, "vcvttpd2dq", VR512, VR256X, fp_to_sint,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8f64, f512mem, v8i32, v8f64,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
SSEPackedDouble>, EVEX_V512, PD, VEX_W,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>;
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
defm VCVTTPS2UDQZ : avx512_vcvt_fp<0x78, "vcvttps2udq", VR512, VR512, fp_to_uint,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv16f32, f512mem, v16i32, v16f32,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
SSEPackedSingle>, EVEX_V512, PS,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VF>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
// cvttps2udq (src, 0, mask-all-ones, sae-current)
def : Pat<(v16i32 (int_x86_avx512_mask_cvttps2udq_512 (v16f32 VR512:$src),
(v16i32 immAllZerosV), (i16 -1), FROUND_CURRENT)),
(VCVTTPS2UDQZrr VR512:$src)>;
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
defm VCVTTPD2UDQZ : avx512_vcvt_fp<0x78, "vcvttpd2udq", VR512, VR256X, fp_to_uint,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8f64, f512mem, v8i32, v8f64,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
SSEPackedDouble>, EVEX_V512, PS, VEX_W,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<64, CD8VF>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
// cvttpd2udq (src, 0, mask-all-ones, sae-current)
def : Pat<(v8i32 (int_x86_avx512_mask_cvttpd2udq_512 (v8f64 VR512:$src),
(v8i32 immAllZerosV), (i8 -1), FROUND_CURRENT)),
(VCVTTPD2UDQZrr VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm VCVTUDQ2PDZ : avx512_vcvt_fp<0x7A, "vcvtudq2pd", VR256X, VR512, uint_to_fp,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv4i64, f256mem, v8f64, v8i32,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
SSEPackedDouble>, EVEX_V512, XS,
EVEX_CD8<32, CD8VH>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
defm VCVTUDQ2PSZ : avx512_vcvt_fp_with_rc<0x7A, "vcvtudq2ps", VR512, VR512, uint_to_fp,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv16i32, f512mem, v16f32, v16i32,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
SSEPackedSingle>, EVEX_V512, XD,
EVEX_CD8<32, CD8VF>;
def : Pat<(v8i32 (fp_to_uint (v8f32 VR256X:$src1))),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
(EXTRACT_SUBREG (v16i32 (VCVTTPS2UDQZrr
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(v16f32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_ymm)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added fp_to_uint and uint_to_fp patterns. llvm-svn: 205754
2014-04-08 15:24:02 +08:00
def : Pat<(v4i32 (fp_to_uint (v4f32 VR128X:$src1))),
(EXTRACT_SUBREG (v16i32 (VCVTTPS2UDQZrr
(v16f32 (SUBREG_TO_REG (i32 0), VR128X:$src1, sub_xmm)))), sub_xmm)>;
def : Pat<(v8f32 (uint_to_fp (v8i32 VR256X:$src1))),
(EXTRACT_SUBREG (v16f32 (VCVTUDQ2PSZrr
(v16i32 (SUBREG_TO_REG (i32 0), VR256X:$src1, sub_ymm)))), sub_ymm)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added fp_to_uint and uint_to_fp patterns. llvm-svn: 205754
2014-04-08 15:24:02 +08:00
def : Pat<(v4f32 (uint_to_fp (v4i32 VR128X:$src1))),
(EXTRACT_SUBREG (v16f32 (VCVTUDQ2PSZrr
(v16i32 (SUBREG_TO_REG (i32 0), VR128X:$src1, sub_xmm)))), sub_xmm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
Add pattern for unsigned v4i32->v4f64 convert on AVX512. llvm-svn: 211164
2014-06-18 22:04:37 +08:00
def : Pat<(v4f64 (uint_to_fp (v4i32 VR128X:$src1))),
(EXTRACT_SUBREG (v8f64 (VCVTUDQ2PDZrr
(v8i32 (SUBREG_TO_REG (i32 0), VR128X:$src1, sub_xmm)))), sub_ymm)>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def : Pat<(v16f32 (int_x86_avx512_mask_cvtdq2ps_512 (v16i32 VR512:$src),
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
(bc_v16f32 (v16i32 immAllZerosV)), (i16 -1), imm:$rc)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VCVTDQ2PSZrrb VR512:$src, imm:$rc)>;
AVX-512: Added more intrinsics for convert and min/max. Removed vzeroupper from AVX-512 mode - our optimization gude does not recommend to insert vzeroupper at all. llvm-svn: 198557
2014-01-05 18:46:09 +08:00
def : Pat<(v8f64 (int_x86_avx512_mask_cvtdq2pd_512 (v8i32 VR256X:$src),
(bc_v8f64 (v16i32 immAllZerosV)), (i8 -1))),
(VCVTDQ2PDZrr VR256X:$src)>;
def : Pat<(v16f32 (int_x86_avx512_mask_cvtudq2ps_512 (v16i32 VR512:$src),
(bc_v16f32 (v16i32 immAllZerosV)), (i16 -1), imm:$rc)),
(VCVTUDQ2PSZrrb VR512:$src, imm:$rc)>;
def : Pat<(v8f64 (int_x86_avx512_mask_cvtudq2pd_512 (v8i32 VR256X:$src),
(bc_v8f64 (v16i32 immAllZerosV)), (i8 -1))),
(VCVTUDQ2PDZrr VR256X:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
multiclass avx512_vcvt_fp2int<bits<8> opc, string asm, RegisterClass SrcRC,
RegisterClass DstRC, PatFrag mem_frag,
X86MemOperand x86memop, Domain d> {
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0 in {
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def rr : AVX512PI<opc, MRMSrcReg, (outs DstRC:$dst), (ins SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[], d>, EVEX;
def rrb : AVX512PI<opc, MRMSrcReg, (outs DstRC:$dst), (ins SrcRC:$src, AVX512RC:$rc),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$rc, $src, $dst|$dst, $src, $rc}"),
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
[], d>, EVEX, EVEX_B, EVEX_RC;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
let mayLoad = 1 in
def rm : AVX512PI<opc, MRMSrcMem, (outs DstRC:$dst), (ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(asm,"\t{$src, $dst|$dst, $src}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[], d>, EVEX;
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
} // hasSideEffects = 0
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
}
defm VCVTPS2DQZ : avx512_vcvt_fp2int<0x5B, "vcvtps2dq", VR512, VR512,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv16f32, f512mem, SSEPackedSingle>, PD,
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
EVEX_V512, EVEX_CD8<32, CD8VF>;
defm VCVTPD2DQZ : avx512_vcvt_fp2int<0xE6, "vcvtpd2dq", VR512, VR256X,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8f64, f512mem, SSEPackedDouble>, XD, VEX_W,
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
EVEX_V512, EVEX_CD8<64, CD8VF>;
def : Pat <(v16i32 (int_x86_avx512_mask_cvtps2dq_512 (v16f32 VR512:$src),
(v16i32 immAllZerosV), (i16 -1), imm:$rc)),
(VCVTPS2DQZrrb VR512:$src, imm:$rc)>;
def : Pat <(v8i32 (int_x86_avx512_mask_cvtpd2dq_512 (v8f64 VR512:$src),
(v8i32 immAllZerosV), (i8 -1), imm:$rc)),
(VCVTPD2DQZrrb VR512:$src, imm:$rc)>;
defm VCVTPS2UDQZ : avx512_vcvt_fp2int<0x79, "vcvtps2udq", VR512, VR512,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv16f32, f512mem, SSEPackedSingle>,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
PS, EVEX_V512, EVEX_CD8<32, CD8VF>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
defm VCVTPD2UDQZ : avx512_vcvt_fp2int<0x79, "vcvtpd2udq", VR512, VR256X,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8f64, f512mem, SSEPackedDouble>, VEX_W,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
PS, EVEX_V512, EVEX_CD8<64, CD8VF>;
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
def : Pat <(v16i32 (int_x86_avx512_mask_cvtps2udq_512 (v16f32 VR512:$src),
(v16i32 immAllZerosV), (i16 -1), imm:$rc)),
(VCVTPS2UDQZrrb VR512:$src, imm:$rc)>;
def : Pat <(v8i32 (int_x86_avx512_mask_cvtpd2udq_512 (v8f64 VR512:$src),
(v8i32 immAllZerosV), (i8 -1), imm:$rc)),
(VCVTPD2UDQZrrb VR512:$src, imm:$rc)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(v8f32 (fround (loadv8f64 addr:$src))),
(VCVTPD2PSZrm addr:$src)>;
def : Pat<(v8f64 (extloadv8f32 addr:$src)),
(VCVTPS2PDZrm addr:$src)>;
}
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
//===----------------------------------------------------------------------===//
// Half precision conversion instructions
//===----------------------------------------------------------------------===//
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
multiclass avx512_cvtph2ps<RegisterClass destRC, RegisterClass srcRC,
X86MemOperand x86memop> {
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
def rr : AVX5128I<0x13, MRMSrcReg, (outs destRC:$dst), (ins srcRC:$src),
"vcvtph2ps\t{$src, $dst|$dst, $src}",
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
[]>, EVEX;
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0, mayLoad = 1 in
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
def rm : AVX5128I<0x13, MRMSrcMem, (outs destRC:$dst), (ins x86memop:$src),
"vcvtph2ps\t{$src, $dst|$dst, $src}", []>, EVEX;
}
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
multiclass avx512_cvtps2ph<RegisterClass destRC, RegisterClass srcRC,
X86MemOperand x86memop> {
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
def rr : AVX512AIi8<0x1D, MRMDestReg, (outs destRC:$dst),
[X86] Replace i32i8imm on SSE/AVX instructions with i32u8imm which will make the assembler bounds check them. It will also make them print as unsigned. llvm-svn: 227032
2015-01-25 10:21:16 +08:00
(ins srcRC:$src1, i32u8imm:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"vcvtps2ph\t{$src2, $src1, $dst|$dst, $src1, $src2}",
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
[]>, EVEX;
AVX-512: changed property name from "neverHasSideEffects=1" to "hasSideEffects=0", added this property to VMOVSS/VMOVSD; Optimized a truncate pattern. llvm-svn: 198562
2014-01-05 22:21:07 +08:00
let hasSideEffects = 0, mayStore = 1 in
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
def mr : AVX512AIi8<0x1D, MRMDestMem, (outs),
[X86] Replace i32i8imm on SSE/AVX instructions with i32u8imm which will make the assembler bounds check them. It will also make them print as unsigned. llvm-svn: 227032
2015-01-25 10:21:16 +08:00
(ins x86memop:$dst, srcRC:$src1, i32u8imm:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"vcvtps2ph\t{$src2, $src1, $dst|$dst, $src1, $src2}", []>, EVEX;
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
}
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
defm VCVTPH2PSZ : avx512_cvtph2ps<VR512, VR256X, f256mem>, EVEX_V512,
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
EVEX_CD8<32, CD8VH>;
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
defm VCVTPS2PHZ : avx512_cvtps2ph<VR256X, VR512, f256mem>, EVEX_V512,
AVX-512: added VCVTPH2PS, VCVTPS2PH with intrinsics llvm-svn: 193312
2013-10-24 15:16:35 +08:00
EVEX_CD8<32, CD8VH>;
AVX-512: Added intrinsic for cvtph2ps. Added VPTESTNM instruction. Added a pattern to vselect (lit tests will follow). llvm-svn: 200823
2014-02-05 15:05:03 +08:00
def : Pat<(v16i16 (int_x86_avx512_mask_vcvtps2ph_512 (v16f32 VR512:$src),
imm:$rc, (bc_v16i16(v8i32 immAllZerosV)), (i16 -1))),
(VCVTPS2PHZrr VR512:$src, imm:$rc)>;
def : Pat<(v16f32 (int_x86_avx512_mask_vcvtph2ps_512 (v16i16 VR256X:$src),
(bc_v16f32(v16i32 immAllZerosV)), (i16 -1), (i32 FROUND_CURRENT))),
(VCVTPH2PSZrr VR256X:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let Defs = [EFLAGS], Predicates = [HasAVX512] in {
defm VUCOMISSZ : sse12_ord_cmp<0x2E, FR32X, X86cmp, f32, f32mem, loadf32,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
"ucomiss">, PS, EVEX, VEX_LIG,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VT1>;
defm VUCOMISDZ : sse12_ord_cmp<0x2E, FR64X, X86cmp, f64, f64mem, loadf64,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
"ucomisd">, PD, EVEX,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
let Pattern = []<dag> in {
defm VCOMISSZ : sse12_ord_cmp<0x2F, VR128X, undef, v4f32, f128mem, load,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
"comiss">, PS, EVEX, VEX_LIG,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VT1>;
defm VCOMISDZ : sse12_ord_cmp<0x2F, VR128X, undef, v2f64, f128mem, load,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
"comisd">, PD, EVEX,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
}
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
let isCodeGenOnly = 1 in {
defm Int_VUCOMISSZ : sse12_ord_cmp<0x2E, VR128X, X86ucomi, v4f32, f128mem,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
load, "ucomiss">, PS, EVEX, VEX_LIG,
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
EVEX_CD8<32, CD8VT1>;
defm Int_VUCOMISDZ : sse12_ord_cmp<0x2E, VR128X, X86ucomi, v2f64, f128mem,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
load, "ucomisd">, PD, EVEX,
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
defm Int_VCOMISSZ : sse12_ord_cmp<0x2F, VR128X, X86comi, v4f32, f128mem,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
load, "comiss">, PS, EVEX, VEX_LIG,
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
EVEX_CD8<32, CD8VT1>;
defm Int_VCOMISDZ : sse12_ord_cmp<0x2F, VR128X, X86comi, v2f64, f128mem,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
load, "comisd">, PD, EVEX,
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
VEX_LIG, VEX_W, EVEX_CD8<64, CD8VT1>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
/// avx512_fp14_s rcp14ss, rcp14sd, rsqrt14ss, rsqrt14sd
multiclass avx512_fp14_s<bits<8> opc, string OpcodeStr, RegisterClass RC,
X86MemOperand x86memop> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let hasSideEffects = 0 in {
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def rr : AVX5128I<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src1, RC:$src2),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"), []>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
let mayLoad = 1 in {
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def rm : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, x86memop:$src2),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"), []>, EVEX_4V;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
}
}
defm VRCP14SS : avx512_fp14_s<0x4D, "vrcp14ss", FR32X, f32mem>,
EVEX_CD8<32, CD8VT1>;
defm VRCP14SD : avx512_fp14_s<0x4D, "vrcp14sd", FR64X, f64mem>,
VEX_W, EVEX_CD8<64, CD8VT1>;
defm VRSQRT14SS : avx512_fp14_s<0x4F, "vrsqrt14ss", FR32X, f32mem>,
EVEX_CD8<32, CD8VT1>;
defm VRSQRT14SD : avx512_fp14_s<0x4F, "vrsqrt14sd", FR64X, f64mem>,
VEX_W, EVEX_CD8<64, CD8VT1>;
def : Pat <(v4f32 (int_x86_avx512_rcp14_ss (v4f32 VR128X:$src1),
(v4f32 VR128X:$src2), (bc_v4f32 (v4i32 immAllZerosV)), (i8 -1))),
(COPY_TO_REGCLASS (VRCP14SSrr (COPY_TO_REGCLASS VR128X:$src1, FR32X),
(COPY_TO_REGCLASS VR128X:$src2, FR32X)), VR128X)>;
def : Pat <(v2f64 (int_x86_avx512_rcp14_sd (v2f64 VR128X:$src1),
(v2f64 VR128X:$src2), (bc_v2f64 (v4i32 immAllZerosV)), (i8 -1))),
(COPY_TO_REGCLASS (VRCP14SDrr (COPY_TO_REGCLASS VR128X:$src1, FR64X),
(COPY_TO_REGCLASS VR128X:$src2, FR64X)), VR128X)>;
def : Pat <(v4f32 (int_x86_avx512_rsqrt14_ss (v4f32 VR128X:$src1),
(v4f32 VR128X:$src2), (bc_v4f32 (v4i32 immAllZerosV)), (i8 -1))),
(COPY_TO_REGCLASS (VRSQRT14SSrr (COPY_TO_REGCLASS VR128X:$src1, FR32X),
(COPY_TO_REGCLASS VR128X:$src2, FR32X)), VR128X)>;
def : Pat <(v2f64 (int_x86_avx512_rsqrt14_sd (v2f64 VR128X:$src1),
(v2f64 VR128X:$src2), (bc_v2f64 (v4i32 immAllZerosV)), (i8 -1))),
(COPY_TO_REGCLASS (VRSQRT14SDrr (COPY_TO_REGCLASS VR128X:$src1, FR64X),
(COPY_TO_REGCLASS VR128X:$src2, FR64X)), VR128X)>;
/// avx512_fp14_p rcp14ps, rcp14pd, rsqrt14ps, rsqrt14pd
multiclass avx512_fp14_p<bits<8> opc, string OpcodeStr, SDNode OpNode,
[AVX-512] Expanded rsqrt/rcp instructions to VL subset. Refactored multiclass through AVX512_maskable llvm-svn: 220783
2014-10-29 00:37:13 +08:00
X86VectorVTInfo _> {
defm r: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src), OpcodeStr, "$src", "$src",
(_.FloatVT (OpNode _.RC:$src))>, EVEX, T8PD;
let mayLoad = 1 in {
defm m: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.MemOp:$src), OpcodeStr, "$src", "$src",
(OpNode (_.FloatVT
(bitconvert (_.LdFrag addr:$src))))>, EVEX, T8PD;
defm mb: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.ScalarMemOp:$src), OpcodeStr,
"${src}"##_.BroadcastStr, "${src}"##_.BroadcastStr,
(OpNode (_.FloatVT
(X86VBroadcast (_.ScalarLdFrag addr:$src))))>,
EVEX, T8PD, EVEX_B;
}
}
multiclass avx512_fp14_p_vl_all<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm PSZ : avx512_fp14_p<opc, !strconcat(OpcodeStr, "ps"), OpNode, v16f32_info>,
EVEX_V512, EVEX_CD8<32, CD8VF>;
defm PDZ : avx512_fp14_p<opc, !strconcat(OpcodeStr, "pd"), OpNode, v8f64_info>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
// Define only if AVX512VL feature is present.
let Predicates = [HasVLX] in {
defm PSZ128 : avx512_fp14_p<opc, !strconcat(OpcodeStr, "ps"),
OpNode, v4f32x_info>,
EVEX_V128, EVEX_CD8<32, CD8VF>;
defm PSZ256 : avx512_fp14_p<opc, !strconcat(OpcodeStr, "ps"),
OpNode, v8f32x_info>,
EVEX_V256, EVEX_CD8<32, CD8VF>;
defm PDZ128 : avx512_fp14_p<opc, !strconcat(OpcodeStr, "pd"),
OpNode, v2f64x_info>,
EVEX_V128, VEX_W, EVEX_CD8<64, CD8VF>;
defm PDZ256 : avx512_fp14_p<opc, !strconcat(OpcodeStr, "pd"),
OpNode, v4f64x_info>,
EVEX_V256, VEX_W, EVEX_CD8<64, CD8VF>;
}
}
defm VRSQRT14 : avx512_fp14_p_vl_all<0x4E, "vrsqrt14", X86frsqrt>;
defm VRCP14 : avx512_fp14_p_vl_all<0x4C, "vrcp14", X86frcp>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
def : Pat <(v16f32 (int_x86_avx512_rsqrt14_ps_512 (v16f32 VR512:$src),
(bc_v16f32 (v16i32 immAllZerosV)), (i16 -1))),
(VRSQRT14PSZr VR512:$src)>;
def : Pat <(v8f64 (int_x86_avx512_rsqrt14_pd_512 (v8f64 VR512:$src),
(bc_v8f64 (v16i32 immAllZerosV)), (i8 -1))),
(VRSQRT14PDZr VR512:$src)>;
def : Pat <(v16f32 (int_x86_avx512_rcp14_ps_512 (v16f32 VR512:$src),
(bc_v16f32 (v16i32 immAllZerosV)), (i16 -1))),
(VRCP14PSZr VR512:$src)>;
def : Pat <(v8f64 (int_x86_avx512_rcp14_pd_512 (v8f64 VR512:$src),
(bc_v8f64 (v16i32 immAllZerosV)), (i8 -1))),
(VRCP14PDZr VR512:$src)>;
/// avx512_fp28_s rcp28ss, rcp28sd, rsqrt28ss, rsqrt28sd
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
multiclass avx512_fp28_s<bits<8> opc, string OpcodeStr,X86VectorVTInfo _,
SDNode OpNode> {
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
defm r : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
(i32 FROUND_CURRENT))>;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
defm rb : AVX512_maskable_scalar<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.RC:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1), (_.VT _.RC:$src2),
(i32 FROUND_NO_EXC)), "{sae}">, EVEX_B;
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
defm m : AVX512_maskable_scalar<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.RC:$src1, _.MemOp:$src2), OpcodeStr,
"$src2, $src1", "$src1, $src2",
(OpNode (_.VT _.RC:$src1),
(_.VT (scalar_to_vector (_.ScalarLdFrag addr:$src2))),
(i32 FROUND_CURRENT))>;
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
multiclass avx512_eri_s<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm SS : avx512_fp28_s<opc, OpcodeStr#"ss", f32x_info, OpNode>,
EVEX_CD8<32, CD8VT1>;
defm SD : avx512_fp28_s<opc, OpcodeStr#"sd", f64x_info, OpNode>,
EVEX_CD8<64, CD8VT1>, VEX_W;
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
let hasSideEffects = 0, Predicates = [HasERI] in {
defm VRCP28 : avx512_eri_s<0xCB, "vrcp28", X86rcp28s>, T8PD, EVEX_4V;
defm VRSQRT28 : avx512_eri_s<0xCD, "vrsqrt28", X86rsqrt28s>, T8PD, EVEX_4V;
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
/// avx512_fp28_p rcp28ps, rcp28pd, rsqrt28ps, rsqrt28pd
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
multiclass avx512_fp28_p<bits<8> opc, string OpcodeStr, X86VectorVTInfo _,
SDNode OpNode> {
defm r : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src), OpcodeStr, "$src", "$src",
(OpNode (_.VT _.RC:$src), (i32 FROUND_CURRENT))>;
defm rb : AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
(ins _.RC:$src), OpcodeStr,
"$src", "$src",
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
(OpNode (_.VT _.RC:$src), (i32 FROUND_NO_EXC)),
"{sae}">, EVEX_B;
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
defm m : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.MemOp:$src), OpcodeStr, "$src", "$src",
(OpNode (_.FloatVT
AVX-512: Scalar ERI intrinsics including SAE mode and memory operand. Added AVX512_maskable_scalar template, that should cover all scalar instructions in the future. The main difference between AVX512_maskable_scalar<> and AVX512_maskable<> is using X86select instead of vselect. I need it, because I can't create vselect node for MVT::i1 mask for scalar instruction. http://reviews.llvm.org/D6378 llvm-svn: 222820
2014-11-26 18:46:49 +08:00
(bitconvert (_.LdFrag addr:$src))),
(i32 FROUND_CURRENT))>;
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
defm mb : AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
(ins _.MemOp:$src), OpcodeStr, "$src", "$src",
(OpNode (_.FloatVT
(X86VBroadcast (_.ScalarLdFrag addr:$src))),
(i32 FROUND_CURRENT))>, EVEX_B;
}
multiclass avx512_eri<bits<8> opc, string OpcodeStr, SDNode OpNode> {
defm PS : avx512_fp28_p<opc, OpcodeStr#"ps", v16f32_info, OpNode>,
EVEX_CD8<32, CD8VF>;
defm PD : avx512_fp28_p<opc, OpcodeStr#"pd", v8f64_info, OpNode>,
VEX_W, EVEX_CD8<32, CD8VF>;
}
let Predicates = [HasERI], hasSideEffects = 0 in {
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Intrinsics for ERI 3 instructions: vrcp28, vrsqrt28, vexp2, only vector forms. Intrinsics include SAE (Suppres All Exceptions) parameter. http://reviews.llvm.org/D6214 llvm-svn: 221774
2014-11-12 15:31:03 +08:00
defm VRSQRT28 : avx512_eri<0xCC, "vrsqrt28", X86rsqrt28>, EVEX, EVEX_V512, T8PD;
defm VRCP28 : avx512_eri<0xCA, "vrcp28", X86rcp28>, EVEX, EVEX_V512, T8PD;
defm VEXP2 : avx512_eri<0xC8, "vexp2", X86exp2>, EVEX, EVEX_V512, T8PD;
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
}
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
multiclass avx512_sqrt_packed<bits<8> opc, string OpcodeStr,
SDNode OpNode, X86VectorVTInfo _>{
[AVX512] Fix VSQRT packed instructions internal names. No functional change llvm-svn: 220808
2014-10-29 02:22:41 +08:00
defm r: AVX512_maskable<opc, MRMSrcReg, _, (outs _.RC:$dst),
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
(ins _.RC:$src), OpcodeStr, "$src", "$src",
(_.FloatVT (OpNode _.RC:$src))>, EVEX;
let mayLoad = 1 in {
[AVX512] Fix VSQRT packed instructions internal names. No functional change llvm-svn: 220808
2014-10-29 02:22:41 +08:00
defm m: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
(ins _.MemOp:$src), OpcodeStr, "$src", "$src",
(OpNode (_.FloatVT
(bitconvert (_.LdFrag addr:$src))))>, EVEX;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Fix VSQRT packed instructions internal names. No functional change llvm-svn: 220808
2014-10-29 02:22:41 +08:00
defm mb: AVX512_maskable<opc, MRMSrcMem, _, (outs _.RC:$dst),
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
(ins _.ScalarMemOp:$src), OpcodeStr,
"${src}"##_.BroadcastStr, "${src}"##_.BroadcastStr,
(OpNode (_.FloatVT
(X86VBroadcast (_.ScalarLdFrag addr:$src))))>,
EVEX, EVEX_B;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
multiclass avx512_sqrt_scalar<bits<8> opc, string OpcodeStr,
Intrinsic F32Int, Intrinsic F64Int,
OpndItins itins_s, OpndItins itins_d> {
def SSZr : SI<opc, MRMSrcReg, (outs FR32X:$dst),
(ins FR32X:$src1, FR32X:$src2),
!strconcat(OpcodeStr,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"ss\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[], itins_s.rr>, XS, EVEX_4V;
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
let isCodeGenOnly = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def SSZr_Int : SIi8<opc, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src1, VR128X:$src2),
!strconcat(OpcodeStr,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"ss\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set VR128X:$dst,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(F32Int VR128X:$src1, VR128X:$src2))],
itins_s.rr>, XS, EVEX_4V;
let mayLoad = 1 in {
def SSZm : SI<opc, MRMSrcMem, (outs FR32X:$dst),
(ins FR32X:$src1, f32mem:$src2),
!strconcat(OpcodeStr,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"ss\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[], itins_s.rm>, XS, EVEX_4V, EVEX_CD8<32, CD8VT1>;
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
let isCodeGenOnly = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def SSZm_Int : SIi8<opc, MRMSrcMem, (outs VR128X:$dst),
(ins VR128X:$src1, ssmem:$src2),
!strconcat(OpcodeStr,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"ss\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set VR128X:$dst,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(F32Int VR128X:$src1, sse_load_f32:$src2))],
itins_s.rm>, XS, EVEX_4V, EVEX_CD8<32, CD8VT1>;
}
def SDZr : SI<opc, MRMSrcReg, (outs FR64X:$dst),
(ins FR64X:$src1, FR64X:$src2),
!strconcat(OpcodeStr,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"sd\t{$src2, $src1, $dst|$dst, $src1, $src2}"), []>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
XD, EVEX_4V, VEX_W;
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
let isCodeGenOnly = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def SDZr_Int : SIi8<opc, MRMSrcReg, (outs VR128X:$dst),
(ins VR128X:$src1, VR128X:$src2),
!strconcat(OpcodeStr,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"sd\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set VR128X:$dst,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(F64Int VR128X:$src1, VR128X:$src2))],
itins_s.rr>, XD, EVEX_4V, VEX_W;
let mayLoad = 1 in {
def SDZm : SI<opc, MRMSrcMem, (outs FR64X:$dst),
(ins FR64X:$src1, f64mem:$src2),
!strconcat(OpcodeStr,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"sd\t{$src2, $src1, $dst|$dst, $src1, $src2}"), []>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
XD, EVEX_4V, VEX_W, EVEX_CD8<64, CD8VT1>;
Mark all x86 Int_ and _Int patterns as isCodeGenOnly so the disassembler table builder doesn't need to string match them to exclude them. llvm-svn: 198323
2014-01-03 01:28:14 +08:00
let isCodeGenOnly = 1 in
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def SDZm_Int : SIi8<opc, MRMSrcMem, (outs VR128X:$dst),
(ins VR128X:$src1, sdmem:$src2),
!strconcat(OpcodeStr,
AVX-512: Removed "z" suffix from AVX-512 instructions, since it is incompatible with GCC. I moved a test from avx512-vbroadcast-crash.ll to avx512-vbroadcast.ll I defined HasAVX512 predicate as AssemblerPredicate. It means that you should invoke llvm-mc with "-mcpu=knl" to get encoding for AVX-512 instructions. I need this to let AsmMatcher to set different encoding for AVX and AVX-512 instructions that have the same mnemonic and operands (all scalar instructions). llvm-svn: 197041
2013-12-11 22:31:04 +08:00
"sd\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
[(set VR128X:$dst,
(F64Int VR128X:$src1, sse_load_f64:$src2))]>,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
XD, EVEX_4V, VEX_W, EVEX_CD8<64, CD8VT1>;
}
}
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
multiclass avx512_sqrt_packed_all<bits<8> opc, string OpcodeStr,
SDNode OpNode> {
defm PSZ : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "ps"), OpNode,
v16f32_info>,
EVEX_V512, PS, EVEX_CD8<32, CD8VF>;
defm PDZ : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "pd"), OpNode,
v8f64_info>,
EVEX_V512, VEX_W, PD, EVEX_CD8<64, CD8VF>;
// Define only if AVX512VL feature is present.
let Predicates = [HasVLX] in {
defm PSZ128 : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "ps"),
OpNode, v4f32x_info>,
EVEX_V128, PS, EVEX_CD8<32, CD8VF>;
defm PSZ256 : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "ps"),
OpNode, v8f32x_info>,
EVEX_V256, PS, EVEX_CD8<32, CD8VF>;
defm PDZ128 : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "pd"),
OpNode, v2f64x_info>,
EVEX_V128, VEX_W, PD, EVEX_CD8<64, CD8VF>;
defm PDZ256 : avx512_sqrt_packed<opc, !strconcat(OpcodeStr, "pd"),
OpNode, v4f64x_info>,
EVEX_V256, VEX_W, PD, EVEX_CD8<64, CD8VF>;
}
}
defm VSQRT : avx512_sqrt_packed_all<0x51, "vsqrt", fsqrt>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
defm VSQRT : avx512_sqrt_scalar<0x51, "sqrt",
int_x86_avx512_sqrt_ss, int_x86_avx512_sqrt_sd,
[AVX512] Extended avx512_sqrt_packed (sqrt instructions) to VL subset. Refactored through AVX512_maskable llvm-svn: 220806
2014-10-29 02:15:20 +08:00
SSE_SQRTSS, SSE_SQRTSD>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
let Predicates = [HasAVX512] in {
AVX-512: Fixed intrinsic of VSQRTPS/PD instructions. I set number and types of parameters according to GCC intrinsics. llvm-svn: 213640
2014-07-22 19:07:31 +08:00
def : Pat<(v16f32 (int_x86_avx512_sqrt_ps_512 (v16f32 VR512:$src1),
(bc_v16f32 (v16i32 immAllZerosV)), (i16 -1), FROUND_CURRENT)),
[AVX512] Fix VSQRT packed instructions internal names. No functional change llvm-svn: 220808
2014-10-29 02:22:41 +08:00
(VSQRTPSZr VR512:$src1)>;
AVX-512: Fixed intrinsic of VSQRTPS/PD instructions. I set number and types of parameters according to GCC intrinsics. llvm-svn: 213640
2014-07-22 19:07:31 +08:00
def : Pat<(v8f64 (int_x86_avx512_sqrt_pd_512 (v8f64 VR512:$src1),
(bc_v8f64 (v16i32 immAllZerosV)), (i8 -1), FROUND_CURRENT)),
[AVX512] Fix VSQRT packed instructions internal names. No functional change llvm-svn: 220808
2014-10-29 02:22:41 +08:00
(VSQRTPDZr VR512:$src1)>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
def : Pat<(f32 (fsqrt FR32X:$src)),
(VSQRTSSZr (f32 (IMPLICIT_DEF)), FR32X:$src)>;
def : Pat<(f32 (fsqrt (load addr:$src))),
(VSQRTSSZm (f32 (IMPLICIT_DEF)), addr:$src)>,
Requires<[OptForSize]>;
def : Pat<(f64 (fsqrt FR64X:$src)),
(VSQRTSDZr (f64 (IMPLICIT_DEF)), FR64X:$src)>;
def : Pat<(f64 (fsqrt (load addr:$src))),
(VSQRTSDZm (f64 (IMPLICIT_DEF)), addr:$src)>,
Requires<[OptForSize]>;
def : Pat<(f32 (X86frsqrt FR32X:$src)),
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
(VRSQRT14SSrr (f32 (IMPLICIT_DEF)), FR32X:$src)>;
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
def : Pat<(f32 (X86frsqrt (load addr:$src))),
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
(VRSQRT14SSrm (f32 (IMPLICIT_DEF)), addr:$src)>,
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
Requires<[OptForSize]>;
def : Pat<(f32 (X86frcp FR32X:$src)),
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
(VRCP14SSrr (f32 (IMPLICIT_DEF)), FR32X:$src)>;
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
def : Pat<(f32 (X86frcp (load addr:$src))),
AVX-512: Embedded Rounding Control - encoding and printing Changed intrinsics for vrcp14/vrcp28 vrsqrt14/vrsqrt28 - aligned with GCC. llvm-svn: 199102
2014-01-13 20:55:03 +08:00
(VRCP14SSrm (f32 (IMPLICIT_DEF)), addr:$src)>,
AVX-512: Added VRCP28 and VRSQRT28 instructions and intrinsics. llvm-svn: 192283
2013-10-09 16:16:14 +08:00
Requires<[OptForSize]>;
def : Pat<(int_x86_sse_sqrt_ss VR128X:$src),
(COPY_TO_REGCLASS (VSQRTSSZr (f32 (IMPLICIT_DEF)),
(COPY_TO_REGCLASS VR128X:$src, FR32)),
VR128X)>;
def : Pat<(int_x86_sse_sqrt_ss sse_load_f32:$src),
(VSQRTSSZm_Int (v4f32 (IMPLICIT_DEF)), sse_load_f32:$src)>;
def : Pat<(int_x86_sse2_sqrt_sd VR128X:$src),
(COPY_TO_REGCLASS (VSQRTSDZr (f64 (IMPLICIT_DEF)),
(COPY_TO_REGCLASS VR128X:$src, FR64)),
VR128X)>;
def : Pat<(int_x86_sse2_sqrt_sd sse_load_f64:$src),
(VSQRTSDZm_Int (v2f64 (IMPLICIT_DEF)), sse_load_f64:$src)>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
multiclass avx512_rndscale<bits<8> opc, string OpcodeStr,
X86MemOperand x86memop, RegisterClass RC,
PatFrag mem_frag, Domain d> {
let ExeDomain = d in {
// Intrinsic operation, reg.
// Vector intrinsic operation, reg
def r : AVX512AIi8<opc, MRMSrcReg,
[X86] Replace i32i8imm on SSE/AVX instructions with i32u8imm which will make the assembler bounds check them. It will also make them print as unsigned. llvm-svn: 227032
2015-01-25 10:21:16 +08:00
(outs RC:$dst), (ins RC:$src1, i32u8imm:$src2),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[]>, EVEX;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
// Vector intrinsic operation, mem
def m : AVX512AIi8<opc, MRMSrcMem,
[X86] Replace i32i8imm on SSE/AVX instructions with i32u8imm which will make the assembler bounds check them. It will also make them print as unsigned. llvm-svn: 227032
2015-01-25 10:21:16 +08:00
(outs RC:$dst), (ins x86memop:$src1, i32u8imm:$src2),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[]>, EVEX;
} // ExeDomain
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
defm VRNDSCALEPSZ : avx512_rndscale<0x08, "vrndscaleps", f512mem, VR512,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv16f32, SSEPackedSingle>, EVEX_V512,
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
EVEX_CD8<32, CD8VF>;
def : Pat<(v16f32 (int_x86_avx512_mask_rndscale_ps_512 (v16f32 VR512:$src1),
AVX-512: minor change in rndscale intrinsic llvm-svn: 207937
2014-05-04 21:35:37 +08:00
imm:$src2, (v16f32 VR512:$src1), (i16 -1),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
FROUND_CURRENT)),
(VRNDSCALEPSZr VR512:$src1, imm:$src2)>;
defm VRNDSCALEPDZ : avx512_rndscale<0x09, "vrndscalepd", f512mem, VR512,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv8f64, SSEPackedDouble>, EVEX_V512,
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
VEX_W, EVEX_CD8<64, CD8VF>;
def : Pat<(v8f64 (int_x86_avx512_mask_rndscale_pd_512 (v8f64 VR512:$src1),
AVX-512: minor change in rndscale intrinsic llvm-svn: 207937
2014-05-04 21:35:37 +08:00
imm:$src2, (v8f64 VR512:$src1), (i8 -1),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
FROUND_CURRENT)),
(VRNDSCALEPDZr VR512:$src1, imm:$src2)>;
multiclass avx512_rndscale_scalar<bits<8> opc, string OpcodeStr,
Operand x86memop, RegisterClass RC, Domain d> {
let ExeDomain = d in {
def r : AVX512AIi8<opc, MRMSrcReg,
[X86] Replace i32i8imm on SSE/AVX instructions with i32u8imm which will make the assembler bounds check them. It will also make them print as unsigned. llvm-svn: 227032
2015-01-25 10:21:16 +08:00
(outs RC:$dst), (ins RC:$src1, RC:$src2, i32u8imm:$src3),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[]>, EVEX_4V;
def m : AVX512AIi8<opc, MRMSrcMem,
[X86] Replace i32i8imm on SSE/AVX instructions with i32u8imm which will make the assembler bounds check them. It will also make them print as unsigned. llvm-svn: 227032
2015-01-25 10:21:16 +08:00
(outs RC:$dst), (ins RC:$src1, x86memop:$src2, i32u8imm:$src3),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, $src1, $dst|$dst, $src1, $src2}"),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
[]>, EVEX_4V;
} // ExeDomain
}
defm VRNDSCALESS : avx512_rndscale_scalar<0x0A, "vrndscaless", ssmem, FR32X,
SSEPackedSingle>, EVEX_CD8<32, CD8VT1>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
defm VRNDSCALESD : avx512_rndscale_scalar<0x0B, "vrndscalesd", sdmem, FR64X,
SSEPackedDouble>, EVEX_CD8<64, CD8VT1>;
[X86] Give scalar VRNDSCALE instructions priority in AVX512 mode. llvm-svn: 227039
2015-01-25 16:49:22 +08:00
let Predicates = [HasAVX512] in {
def : Pat<(ffloor FR32X:$src),
(VRNDSCALESSr (f32 (IMPLICIT_DEF)), FR32X:$src, (i32 0x1))>;
def : Pat<(f64 (ffloor FR64X:$src)),
(VRNDSCALESDr (f64 (IMPLICIT_DEF)), FR64X:$src, (i32 0x1))>;
def : Pat<(f32 (fnearbyint FR32X:$src)),
(VRNDSCALESSr (f32 (IMPLICIT_DEF)), FR32X:$src, (i32 0xC))>;
def : Pat<(f64 (fnearbyint FR64X:$src)),
(VRNDSCALESDr (f64 (IMPLICIT_DEF)), FR64X:$src, (i32 0xC))>;
def : Pat<(f32 (fceil FR32X:$src)),
(VRNDSCALESSr (f32 (IMPLICIT_DEF)), FR32X:$src, (i32 0x2))>;
def : Pat<(f64 (fceil FR64X:$src)),
(VRNDSCALESDr (f64 (IMPLICIT_DEF)), FR64X:$src, (i32 0x2))>;
def : Pat<(f32 (frint FR32X:$src)),
(VRNDSCALESSr (f32 (IMPLICIT_DEF)), FR32X:$src, (i32 0x4))>;
def : Pat<(f64 (frint FR64X:$src)),
(VRNDSCALESDr (f64 (IMPLICIT_DEF)), FR64X:$src, (i32 0x4))>;
def : Pat<(f32 (ftrunc FR32X:$src)),
(VRNDSCALESSr (f32 (IMPLICIT_DEF)), FR32X:$src, (i32 0x3))>;
def : Pat<(f64 (ftrunc FR64X:$src)),
(VRNDSCALESDr (f64 (IMPLICIT_DEF)), FR64X:$src, (i32 0x3))>;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (ffloor VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPSZr VR512:$src, (i32 0x1))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (fnearbyint VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPSZr VR512:$src, (i32 0xC))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (fceil VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPSZr VR512:$src, (i32 0x2))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (frint VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPSZr VR512:$src, (i32 0x4))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16f32 (ftrunc VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPSZr VR512:$src, (i32 0x3))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (ffloor VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPDZr VR512:$src, (i32 0x1))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (fnearbyint VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPDZr VR512:$src, (i32 0xC))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (fceil VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPDZr VR512:$src, (i32 0x2))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (frint VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPDZr VR512:$src, (i32 0x4))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8f64 (ftrunc VR512:$src)),
AVX-512: Added intrinsics for vcvt, vcvtt, vrndscale, vcmp Printing rounding control. Enncoding for EVEX_RC (rounding control). llvm-svn: 198277
2014-01-01 23:12:34 +08:00
(VRNDSCALEPDZr VR512:$src, (i32 0x3))>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//-------------------------------------------------
// Integer truncate and extend operations
//-------------------------------------------------
multiclass avx512_trunc_sat<bits<8> opc, string OpcodeStr,
RegisterClass dstRC, RegisterClass srcRC,
RegisterClass KRC, X86MemOperand x86memop> {
def rr : AVX512XS8I<opc, MRMDestReg, (outs dstRC:$dst),
(ins srcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr,"\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
def rrk : AVX512XS8I<opc, MRMDestReg, (outs dstRC:$dst),
(ins KRC:$mask, srcRC:$src),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, ${dst} {${mask}}|${dst} {${mask}}, $src}"),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
[]>, EVEX, EVEX_K;
def rrkz : AVX512XS8I<opc, MRMDestReg, (outs dstRC:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins KRC:$mask, srcRC:$src),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX, EVEX_KZ;
def mr : AVX512XS8I<opc, MRMDestMem, (outs), (ins x86memop:$dst, srcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
def mrk : AVX512XS8I<opc, MRMDestMem, (outs),
(ins x86memop:$dst, KRC:$mask, srcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst {${mask}}|${dst} {${mask}}, $src}"),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
[]>, EVEX, EVEX_K;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
defm VPMOVQB : avx512_trunc_sat<0x32, "vpmovqb", VR128X, VR512, VK8WM,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
i128mem>, EVEX_V512, EVEX_CD8<8, CD8VO>;
defm VPMOVSQB : avx512_trunc_sat<0x22, "vpmovsqb", VR128X, VR512, VK8WM,
i128mem>, EVEX_V512, EVEX_CD8<8, CD8VO>;
defm VPMOVUSQB : avx512_trunc_sat<0x12, "vpmovusqb", VR128X, VR512, VK8WM,
i128mem>, EVEX_V512, EVEX_CD8<8, CD8VO>;
defm VPMOVQW : avx512_trunc_sat<0x34, "vpmovqw", VR128X, VR512, VK8WM,
i128mem>, EVEX_V512, EVEX_CD8<16, CD8VQ>;
defm VPMOVSQW : avx512_trunc_sat<0x24, "vpmovsqw", VR128X, VR512, VK8WM,
i128mem>, EVEX_V512, EVEX_CD8<16, CD8VQ>;
defm VPMOVUSQW : avx512_trunc_sat<0x14, "vpmovusqw", VR128X, VR512, VK8WM,
i128mem>, EVEX_V512, EVEX_CD8<16, CD8VQ>;
defm VPMOVQD : avx512_trunc_sat<0x35, "vpmovqd", VR256X, VR512, VK8WM,
i256mem>, EVEX_V512, EVEX_CD8<32, CD8VH>;
defm VPMOVSQD : avx512_trunc_sat<0x25, "vpmovsqd", VR256X, VR512, VK8WM,
i256mem>, EVEX_V512, EVEX_CD8<32, CD8VH>;
defm VPMOVUSQD : avx512_trunc_sat<0x15, "vpmovusqd", VR256X, VR512, VK8WM,
i256mem>, EVEX_V512, EVEX_CD8<32, CD8VH>;
defm VPMOVDW : avx512_trunc_sat<0x33, "vpmovdw", VR256X, VR512, VK16WM,
i256mem>, EVEX_V512, EVEX_CD8<16, CD8VH>;
defm VPMOVSDW : avx512_trunc_sat<0x23, "vpmovsdw", VR256X, VR512, VK16WM,
i256mem>, EVEX_V512, EVEX_CD8<16, CD8VH>;
defm VPMOVUSDW : avx512_trunc_sat<0x13, "vpmovusdw", VR256X, VR512, VK16WM,
i256mem>, EVEX_V512, EVEX_CD8<16, CD8VH>;
defm VPMOVDB : avx512_trunc_sat<0x31, "vpmovdb", VR128X, VR512, VK16WM,
i128mem>, EVEX_V512, EVEX_CD8<8, CD8VQ>;
defm VPMOVSDB : avx512_trunc_sat<0x21, "vpmovsdb", VR128X, VR512, VK16WM,
i128mem>, EVEX_V512, EVEX_CD8<8, CD8VQ>;
defm VPMOVUSDB : avx512_trunc_sat<0x11, "vpmovusdb", VR128X, VR512, VK16WM,
i128mem>, EVEX_V512, EVEX_CD8<8, CD8VQ>;
def : Pat<(v16i8 (X86vtrunc (v8i64 VR512:$src))), (VPMOVQBrr VR512:$src)>;
def : Pat<(v8i16 (X86vtrunc (v8i64 VR512:$src))), (VPMOVQWrr VR512:$src)>;
def : Pat<(v16i16 (X86vtrunc (v16i32 VR512:$src))), (VPMOVDWrr VR512:$src)>;
def : Pat<(v16i8 (X86vtrunc (v16i32 VR512:$src))), (VPMOVDBrr VR512:$src)>;
def : Pat<(v8i32 (X86vtrunc (v8i64 VR512:$src))), (VPMOVQDrr VR512:$src)>;
def : Pat<(v16i8 (X86vtruncm VK16WM:$mask, (v16i32 VR512:$src))),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
(VPMOVDBrrkz VK16WM:$mask, VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v16i16 (X86vtruncm VK16WM:$mask, (v16i32 VR512:$src))),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
(VPMOVDWrrkz VK16WM:$mask, VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i16 (X86vtruncm VK8WM:$mask, (v8i64 VR512:$src))),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
(VPMOVQWrrkz VK8WM:$mask, VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def : Pat<(v8i32 (X86vtruncm VK8WM:$mask, (v8i64 VR512:$src))),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
(VPMOVQDrrkz VK8WM:$mask, VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
multiclass avx512_extend<bits<8> opc, string OpcodeStr, RegisterClass KRC,
RegisterClass DstRC, RegisterClass SrcRC, SDNode OpNode,
PatFrag mem_frag, X86MemOperand x86memop,
ValueType OpVT, ValueType InVT> {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rr : AVX5128I<opc, MRMSrcReg, (outs DstRC:$dst),
(ins SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set DstRC:$dst, (OpVT (OpNode (InVT SrcRC:$src))))]>, EVEX;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
def rrk : AVX5128I<opc, MRMSrcReg, (outs DstRC:$dst),
(ins KRC:$mask, SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst {${mask}} |$dst {${mask}}, $src}"),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
[]>, EVEX, EVEX_K;
def rrkz : AVX5128I<opc, MRMSrcReg, (outs DstRC:$dst),
(ins KRC:$mask, SrcRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst {${mask}} {z}|$dst {${mask}} {z}, $src}"),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
[]>, EVEX, EVEX_KZ;
let mayLoad = 1 in {
def rm : AVX5128I<opc, MRMSrcMem, (outs DstRC:$dst),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
(ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr,"\t{$src, $dst|$dst, $src}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set DstRC:$dst,
(OpVT (OpNode (InVT (bitconvert (mem_frag addr:$src))))))]>,
EVEX;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
def rmk : AVX5128I<opc, MRMSrcMem, (outs DstRC:$dst),
(ins KRC:$mask, x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr,"\t{$src, $dst {${mask}} |$dst {${mask}}, $src}"),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
[]>,
EVEX, EVEX_K;
def rmkz : AVX5128I<opc, MRMSrcMem, (outs DstRC:$dst),
(ins KRC:$mask, x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr,"\t{$src, $dst {${mask}} {z}|$dst {${mask}} {z}, $src}"),
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
[]>,
EVEX, EVEX_KZ;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVZXBDZ: avx512_extend<0x31, "vpmovzxbd", VK16WM, VR512, VR128X, X86vzext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv2i64, i128mem, v16i32, v16i8>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<8, CD8VQ>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVZXBQZ: avx512_extend<0x32, "vpmovzxbq", VK8WM, VR512, VR128X, X86vzext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv2i64, i128mem, v8i64, v16i8>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<8, CD8VO>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVZXWDZ: avx512_extend<0x33, "vpmovzxwd", VK16WM, VR512, VR256X, X86vzext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv4i64, i256mem, v16i32, v16i16>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<16, CD8VH>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVZXWQZ: avx512_extend<0x34, "vpmovzxwq", VK8WM, VR512, VR128X, X86vzext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv2i64, i128mem, v8i64, v8i16>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<16, CD8VQ>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVZXDQZ: avx512_extend<0x35, "vpmovzxdq", VK8WM, VR512, VR256X, X86vzext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv4i64, i256mem, v8i64, v8i32>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VH>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVSXBDZ: avx512_extend<0x21, "vpmovsxbd", VK16WM, VR512, VR128X, X86vsext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv2i64, i128mem, v16i32, v16i8>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<8, CD8VQ>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVSXBQZ: avx512_extend<0x22, "vpmovsxbq", VK8WM, VR512, VR128X, X86vsext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv2i64, i128mem, v8i64, v16i8>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<8, CD8VO>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVSXWDZ: avx512_extend<0x23, "vpmovsxwd", VK16WM, VR512, VR256X, X86vsext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv4i64, i256mem, v16i32, v16i16>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<16, CD8VH>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVSXWQZ: avx512_extend<0x24, "vpmovsxwq", VK8WM, VR512, VR128X, X86vsext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv2i64, i128mem, v8i64, v8i16>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<16, CD8VQ>;
[AVX512] Implemented integer conversions up/down with masking. Added encoding tests. llvm-svn: 206884
2014-04-22 19:36:19 +08:00
defm VPMOVSXDQZ: avx512_extend<0x25, "vpmovsxdq", VK8WM, VR512, VR256X, X86vsext,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
loadv4i64, i256mem, v8i64, v8i32>, EVEX_V512,
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
EVEX_CD8<32, CD8VH>;
//===----------------------------------------------------------------------===//
// GATHER - SCATTER Operations
multiclass avx512_gather<bits<8> opc, string OpcodeStr, RegisterClass KRC,
RegisterClass RC, X86MemOperand memop> {
let mayLoad = 1,
Constraints = "@earlyclobber $dst, $src1 = $dst, $mask = $mask_wb" in
def rm : AVX5128I<opc, MRMSrcMem, (outs RC:$dst, KRC:$mask_wb),
(ins RC:$src1, KRC:$mask, memop:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, ${dst} {${mask}}|${dst} {${mask}}, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX, EVEX_K;
}
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
let ExeDomain = SSEPackedDouble in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm VGATHERDPDZ : avx512_gather<0x92, "vgatherdpd", VK8WM, VR512, vy64xmem>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VGATHERQPDZ : avx512_gather<0x93, "vgatherqpd", VK8WM, VR512, vz64mem>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
}
let ExeDomain = SSEPackedSingle in {
defm VGATHERDPSZ : avx512_gather<0x92, "vgatherdps", VK16WM, VR512, vz32mem>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm VGATHERQPSZ : avx512_gather<0x93, "vgatherqps", VK8WM, VR256X, vz64mem>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm VPGATHERDQZ : avx512_gather<0x90, "vpgatherdq", VK8WM, VR512, vy64xmem>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VPGATHERDDZ : avx512_gather<0x90, "vpgatherdd", VK16WM, VR512, vz32mem>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
defm VPGATHERQQZ : avx512_gather<0x91, "vpgatherqq", VK8WM, VR512, vz64mem>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VPGATHERQDZ : avx512_gather<0x91, "vpgatherqd", VK8WM, VR256X, vz64mem>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
multiclass avx512_scatter<bits<8> opc, string OpcodeStr, RegisterClass KRC,
RegisterClass RC, X86MemOperand memop> {
let mayStore = 1, Constraints = "$mask = $mask_wb" in
def mr : AVX5128I<opc, MRMDestMem, (outs KRC:$mask_wb),
(ins memop:$dst, KRC:$mask, RC:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, ${dst} {${mask}}|${dst} {${mask}}, $src2}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX, EVEX_K;
}
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
let ExeDomain = SSEPackedDouble in {
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm VSCATTERDPDZ : avx512_scatter<0xA2, "vscatterdpd", VK8WM, VR512, vy64xmem>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VSCATTERQPDZ : avx512_scatter<0xA3, "vscatterqpd", VK8WM, VR512, vz64mem>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
}
let ExeDomain = SSEPackedSingle in {
defm VSCATTERDPSZ : avx512_scatter<0xA2, "vscatterdps", VK16WM, VR512, vz32mem>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm VSCATTERQPSZ : avx512_scatter<0xA3, "vscatterqps", VK8WM, VR256X, vz64mem>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
Fix AVX512 Gather and Scatter execution domains. llvm-svn: 204804
2014-03-26 21:50:50 +08:00
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
defm VPSCATTERDQZ : avx512_scatter<0xA0, "vpscatterdq", VK8WM, VR512, vy64xmem>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VPSCATTERDDZ : avx512_scatter<0xA0, "vpscatterdd", VK16WM, VR512, vz32mem>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
defm VPSCATTERQQZ : avx512_scatter<0xA1, "vpscatterqq", VK8WM, VR512, vz64mem>,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VPSCATTERQDZ : avx512_scatter<0xA1, "vpscatterqd", VK8WM, VR256X, vz64mem>,
EVEX_V512, EVEX_CD8<32, CD8VT1>;
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
// prefetch
multiclass avx512_gather_scatter_prefetch<bits<8> opc, Format F, string OpcodeStr,
RegisterClass KRC, X86MemOperand memop> {
let Predicates = [HasPFI], hasSideEffects = 1 in
def m : AVX5128I<opc, F, (outs), (ins KRC:$mask, memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src {${mask}}|{${mask}}, $src}"),
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
[]>, EVEX, EVEX_K;
}
defm VGATHERPF0DPS: avx512_gather_scatter_prefetch<0xC6, MRM1m, "vgatherpf0dps",
VK16WM, vz32mem>, EVEX_V512, EVEX_CD8<32, CD8VT1>;
defm VGATHERPF0QPS: avx512_gather_scatter_prefetch<0xC7, MRM1m, "vgatherpf0qps",
VK8WM, vz64mem>, EVEX_V512, EVEX_CD8<64, CD8VT1>;
defm VGATHERPF0DPD: avx512_gather_scatter_prefetch<0xC6, MRM1m, "vgatherpf0dpd",
VK8WM, vy32mem>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VT1>;
defm VGATHERPF0QPD: avx512_gather_scatter_prefetch<0xC7, MRM1m, "vgatherpf0qpd",
VK8WM, vz64mem>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: changes in intrinsics 1) Changed gather and scatter intrinsics. Now they are aligned with GCC built-ins. There is no more non-masked form. Masked intrinsic receives -1 if all lanes are executed. 2) I changed the function that works with intrinsics inside X86ISelLowering.cpp. I put all intrinsics in one table. I did it for INTRINSICS_W_CHAIN and plan to put all intrinsics from WO_CHAIN set to the same table in order to avoid the long-long "switch". (I wanted to use static map initialization that allowed by C++11 but I wasn't able to compile it on VS2012). 3) I added gather/scatter prefetch intrinsics. 4) I fixed MRMm encoding for masked instructions. llvm-svn: 208522
2014-05-12 15:18:51 +08:00
defm VGATHERPF1DPS: avx512_gather_scatter_prefetch<0xC6, MRM2m, "vgatherpf1dps",
VK16WM, vz32mem>, EVEX_V512, EVEX_CD8<32, CD8VT1>;
defm VGATHERPF1QPS: avx512_gather_scatter_prefetch<0xC7, MRM2m, "vgatherpf1qps",
VK8WM, vz64mem>, EVEX_V512, EVEX_CD8<64, CD8VT1>;
defm VGATHERPF1DPD: avx512_gather_scatter_prefetch<0xC6, MRM2m, "vgatherpf1dpd",
VK8WM, vy32mem>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VT1>;
defm VGATHERPF1QPD: avx512_gather_scatter_prefetch<0xC7, MRM2m, "vgatherpf1qpd",
VK8WM, vz64mem>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VSCATTERPF0DPS: avx512_gather_scatter_prefetch<0xC6, MRM5m, "vscatterpf0dps",
VK16WM, vz32mem>, EVEX_V512, EVEX_CD8<32, CD8VT1>;
defm VSCATTERPF0QPS: avx512_gather_scatter_prefetch<0xC7, MRM5m, "vscatterpf0qps",
VK8WM, vz64mem>, EVEX_V512, EVEX_CD8<64, CD8VT1>;
defm VSCATTERPF0DPD: avx512_gather_scatter_prefetch<0xC6, MRM5m, "vscatterpf0dpd",
VK8WM, vy32mem>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VT1>;
defm VSCATTERPF0QPD: avx512_gather_scatter_prefetch<0xC7, MRM5m, "vscatterpf0qpd",
VK8WM, vz64mem>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
defm VSCATTERPF1DPS: avx512_gather_scatter_prefetch<0xC6, MRM6m, "vscatterpf1dps",
VK16WM, vz32mem>, EVEX_V512, EVEX_CD8<32, CD8VT1>;
defm VSCATTERPF1QPS: avx512_gather_scatter_prefetch<0xC7, MRM6m, "vscatterpf1qps",
VK8WM, vz64mem>, EVEX_V512, EVEX_CD8<64, CD8VT1>;
defm VSCATTERPF1DPD: avx512_gather_scatter_prefetch<0xC6, MRM6m, "vscatterpf1dpd",
VK8WM, vy32mem>, EVEX_V512, VEX_W, EVEX_CD8<32, CD8VT1>;
defm VSCATTERPF1QPD: avx512_gather_scatter_prefetch<0xC7, MRM6m, "vscatterpf1qpd",
VK8WM, vz64mem>, EVEX_V512, VEX_W, EVEX_CD8<64, CD8VT1>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
//===----------------------------------------------------------------------===//
// VSHUFPS - VSHUFPD Operations
multiclass avx512_shufp<RegisterClass RC, X86MemOperand x86memop,
ValueType vt, string OpcodeStr, PatFrag mem_frag,
Domain d> {
def rmi : AVX512PIi8<0xC6, MRMSrcMem, (outs RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins RC:$src1, x86memop:$src2, u8imm:$src3),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst, (vt (X86Shufp RC:$src1, (mem_frag addr:$src2),
(i8 imm:$src3))))], d, IIC_SSE_SHUFP>,
AVX-512: Added TB prefix to all instructions without prefixes, otherwise encoding fails after the last change in X86MCCodeEmitter.cpp. llvm-svn: 191812
2013-10-02 14:39:07 +08:00
EVEX_4V, Sched<[WriteShuffleLd, ReadAfterLd]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
def rri : AVX512PIi8<0xC6, MRMSrcReg, (outs RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins RC:$src1, RC:$src2, u8imm:$src3),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $src1, $dst|$dst, $src1, $src2, $src3}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[(set RC:$dst, (vt (X86Shufp RC:$src1, RC:$src2,
(i8 imm:$src3))))], d, IIC_SSE_SHUFP>,
AVX-512: Added TB prefix to all instructions without prefixes, otherwise encoding fails after the last change in X86MCCodeEmitter.cpp. llvm-svn: 191812
2013-10-02 14:39:07 +08:00
EVEX_4V, Sched<[WriteShuffle]>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VSHUFPSZ : avx512_shufp<VR512, f512mem, v16f32, "vshufps", loadv16f32,
Add an x86 prefix encoding for instructions that would decode to a different instruction with 0xf2/f3/66 were in front of them, but don't themselves have a prefix. For now this doesn't change any bbehavior, but plan to use it to fix some bugs in the disassembler. llvm-svn: 201538
2014-02-18 08:21:49 +08:00
SSEPackedSingle>, PS, EVEX_V512, EVEX_CD8<32, CD8VF>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
defm VSHUFPDZ : avx512_shufp<VR512, f512mem, v8f64, "vshufpd", loadv8f64,
Separate the concept of 16-bit/32-bit operand size controlled by 0x66 prefix and the current mode from the concept of SSE instructions using 0x66 prefix as part of their encoding without being affected by the mode. This should allow SSE instructions to be encoded correctly in 16-bit mode which r198586 probably broke. llvm-svn: 199193
2014-01-14 15:41:20 +08:00
SSEPackedDouble>, PD, VEX_W, EVEX_V512, EVEX_CD8<64, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: fixed shuffle lowering in case of BLEND and added VSHUFPS patterns. llvm-svn: 192055
2013-10-06 14:11:18 +08:00
def : Pat<(v16i32 (X86Shufp VR512:$src1, VR512:$src2, (i8 imm:$imm))),
(VSHUFPSZrri VR512:$src1, VR512:$src2, imm:$imm)>;
def : Pat<(v16i32 (X86Shufp VR512:$src1,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(loadv16i32 addr:$src2), (i8 imm:$imm))),
AVX-512: fixed shuffle lowering in case of BLEND and added VSHUFPS patterns. llvm-svn: 192055
2013-10-06 14:11:18 +08:00
(VSHUFPSZrmi VR512:$src1, addr:$src2, imm:$imm)>;
def : Pat<(v8i64 (X86Shufp VR512:$src1, VR512:$src2, (i8 imm:$imm))),
(VSHUFPDZrri VR512:$src1, VR512:$src2, imm:$imm)>;
def : Pat<(v8i64 (X86Shufp VR512:$src1,
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
(loadv8i64 addr:$src2), (i8 imm:$imm))),
AVX-512: fixed shuffle lowering in case of BLEND and added VSHUFPS patterns. llvm-svn: 192055
2013-10-06 14:11:18 +08:00
(VSHUFPDZrmi VR512:$src1, addr:$src2, imm:$imm)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
multiclass avx512_valign<X86VectorVTInfo _> {
[AVX512] Rename AVX512_masking* to AVX512_maskable* No functional change. This is the current AVX512_maskable multiclass hierarchy: maskable_custom / \ / \ maskable_common maskable_in_asm / \ / \ maskable maskable_3src llvm-svn: 219363
2014-10-09 07:25:39 +08:00
defm rri : AVX512_maskable<0x03, MRMSrcReg, _, (outs _.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins _.RC:$src1, _.RC:$src2, u8imm:$src3),
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
"valign"##_.Suffix,
[AVX512] Generate masking instruction variants with tablegen After adding the masking variants to several instructions, I have decided to experiment with generating these from the non-masking/unconditional variant. This will hopefully reduce the amount repetition that we currently have in order to define an instruction with all its variants (for a reg/mem instruction this would be 6 instruction defs and 2 Pat<> for the intrinsic). The patch is the first cut that is currently only applied to valignd/q to make the patch small. A few notes on the approach: * In order to stitch together the dag for both the conditional and the unconditional patterns I pass the RHS of the set rather than the full pattern (set dest, RHS). * Rather than subclassing each instruction base class (e.g. AVX512AIi8), with a masking variant which wouldn't scale, I derived the masking instructions from a new base class AVX512 (this is just I<> with Requires<HasAVX512>). The instructions derive from this now, plus a new set of classes that add the format bits and everything else that instruction base class provided (i.e. AVX512AIi8 vs. AVX512AIi8Base). I hope we can go incrementally from here. I expect that: * We will need different variants of the masking class. One example is instructions requiring three vector sources. In this case we tie one of the source operands to dest rather than a new implicit source operand ($src0) * Add the zero-masking variant * Add more AVX512*Base classes as new uses are added I've looked at X86.td.expanded before and after to make sure that nothing got lost for valignd/q. llvm-svn: 215125
2014-08-08 01:53:55 +08:00
"$src3, $src2, $src1", "$src1, $src2, $src3",
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
(_.VT (X86VAlign _.RC:$src2, _.RC:$src1,
[AVX512] Use X86VectorVTInfo in the masking helper classes and the FMAs No functionality change. Makes the code more compact (see the FMA part). This needs a new type attribute MemOpFrag in X86VectorVTInfo. For now I only defined this in the simple cases. See the commment before the attribute. Diff of X86.td.expanded before and after is empty except for the appearance of the new attribute. llvm-svn: 218637
2014-09-30 06:54:41 +08:00
(i8 imm:$src3)))>,
[AVX512] Generate masking instruction variants with tablegen After adding the masking variants to several instructions, I have decided to experiment with generating these from the non-masking/unconditional variant. This will hopefully reduce the amount repetition that we currently have in order to define an instruction with all its variants (for a reg/mem instruction this would be 6 instruction defs and 2 Pat<> for the intrinsic). The patch is the first cut that is currently only applied to valignd/q to make the patch small. A few notes on the approach: * In order to stitch together the dag for both the conditional and the unconditional patterns I pass the RHS of the set rather than the full pattern (set dest, RHS). * Rather than subclassing each instruction base class (e.g. AVX512AIi8), with a masking variant which wouldn't scale, I derived the masking instructions from a new base class AVX512 (this is just I<> with Requires<HasAVX512>). The instructions derive from this now, plus a new set of classes that add the format bits and everything else that instruction base class provided (i.e. AVX512AIi8 vs. AVX512AIi8Base). I hope we can go incrementally from here. I expect that: * We will need different variants of the masking class. One example is instructions requiring three vector sources. In this case we tie one of the source operands to dest rather than a new implicit source operand ($src0) * Add the zero-masking variant * Add more AVX512*Base classes as new uses are added I've looked at X86.td.expanded before and after to make sure that nothing got lost for valignd/q. llvm-svn: 215125
2014-08-08 01:53:55 +08:00
AVX512AIi8Base, EVEX_4V;
[AVX512] Add masking variant and intrinsics for valignd/q This is similar to what I did with the two-source permutation recently. (It's almost too similar so that we should consider generating the masking variants with some tablegen help.) Both encoding and intrinsic tests are added as well. For the latter, this is what the IR that the intrinsic test on the clang side generates. Part of <rdar://problem/17688758> llvm-svn: 214890
2014-08-06 01:23:04 +08:00
[AVX512] Pull everything alignr-related into the multiclass The packed integer pattern becomes the DAG pattern for rri and the packed float, another Pat<> inside the multiclass. No functional change. llvm-svn: 214885
2014-08-06 01:22:50 +08:00
// Also match valign of packed floats.
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
def : Pat<(_.FloatVT (X86VAlign _.RC:$src1, _.RC:$src2, (i8 imm:$imm))),
(!cast<Instruction>(NAME##rri) _.RC:$src2, _.RC:$src1, imm:$imm)>;
[AVX512] Pull everything alignr-related into the multiclass The packed integer pattern becomes the DAG pattern for rri and the packed float, another Pat<> inside the multiclass. No functional change. llvm-svn: 214885
2014-08-06 01:22:50 +08:00
AVX-512: Handled extractelement from mask vector; Added VMOSHDUP/VMOVSLDUP shuffle instructions. llvm-svn: 194691
2013-11-14 19:29:27 +08:00
let mayLoad = 1 in
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
def rmi : AVX512AIi8<0x03, MRMSrcMem, (outs _.RC:$dst),
[X86] Convert all the i8imm used by AVX512 and MMX instructions to u8imm. llvm-svn: 226646
2015-01-21 16:43:49 +08:00
(ins _.RC:$src1, _.MemOp:$src2, u8imm:$src3),
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
!strconcat("valign"##_.Suffix,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src3, $src2, $src1, $dst|"
Wrap long lines llvm-svn: 214884
2014-08-06 01:22:47 +08:00
"$dst, $src1, $src2, $src3}"),
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
[]>, EVEX_4V;
}
[AVX512] Add class to group common template arguments related to vector type We discussed the issue of generality vs. readability of the AVX512 classes recently. I proposed this approach to try to hide and centralize the mappings we commonly perform based on the vector type. A new class X86VectorVTInfo captures these. The idea is to pass an instance of this class to classes/multiclasses instead of the corresponding ValueType. Then the class/multiclass can use its field for things that derive from the type rather than passing all those as separate arguments. I modified avx512_valign to demonstrate this new approach. As you can see instead of 7 related template parameters we now have one. The downside is that we have to refer to fields for the derived values. I named the argument '_' in order to make this as invisible as possible. Please let me know if you absolutely hate this. (Also once we allow local initializations in multiclasses we can recover the original version by assigning the fields to local variables.) Another possible use-case for this class is to directly map things, e.g.: RegisterClass KRC = X86VectorVTInfo<32, i16>.KRC llvm-svn: 216209
2014-08-22 03:50:07 +08:00
defm VALIGND : avx512_valign<v16i32_info>, EVEX_V512, EVEX_CD8<32, CD8VF>;
defm VALIGNQ : avx512_valign<v8i64_info>, VEX_W, EVEX_V512, EVEX_CD8<64, CD8VF>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
// Helper fragments to match sext vXi1 to vXiY.
def v16i1sextv16i32 : PatLeaf<(v16i32 (X86vsrai VR512:$src, (i8 31)))>;
def v8i1sextv8i64 : PatLeaf<(v8i64 (X86vsrai VR512:$src, (i8 63)))>;
multiclass avx512_vpabs<bits<8> opc, string OpcodeStr, ValueType OpVT,
RegisterClass KRC, RegisterClass RC,
X86MemOperand x86memop, X86MemOperand x86scalar_mop,
string BrdcstStr> {
def rr : AVX5128I<opc, MRMSrcReg, (outs RC:$dst), (ins RC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[]>, EVEX;
def rrk : AVX5128I<opc, MRMSrcReg, (outs RC:$dst), (ins KRC:$mask, RC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst {${mask}}|$dst {${mask}}, $src}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[]>, EVEX, EVEX_K;
def rrkz : AVX5128I<opc, MRMSrcReg, (outs RC:$dst), (ins KRC:$mask, RC:$src),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, $dst {${mask}} {z}|$dst {${mask}} {z}, $src}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[]>, EVEX, EVEX_KZ;
let mayLoad = 1 in {
def rm : AVX5128I<opc, MRMSrcMem, (outs VR512:$dst),
(ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, $dst|$dst, $src}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[]>, EVEX;
def rmk : AVX5128I<opc, MRMSrcMem, (outs VR512:$dst),
(ins KRC:$mask, x86memop:$src),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, $dst {${mask}}|$dst {${mask}}, $src}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[]>, EVEX, EVEX_K;
def rmkz : AVX5128I<opc, MRMSrcMem, (outs VR512:$dst),
(ins KRC:$mask, x86memop:$src),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, $dst {${mask}} {z}|$dst {${mask}} {z}, $src}"),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
[]>, EVEX, EVEX_KZ;
def rmb : AVX5128I<opc, MRMSrcMem, (outs VR512:$dst),
(ins x86scalar_mop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src}", BrdcstStr,
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
", $dst|$dst, ${src}", BrdcstStr, "}"),
[]>, EVEX, EVEX_B;
def rmbk : AVX5128I<opc, MRMSrcMem, (outs VR512:$dst),
(ins KRC:$mask, x86scalar_mop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src}", BrdcstStr,
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
", $dst {${mask}}|$dst {${mask}}, ${src}", BrdcstStr, "}"),
[]>, EVEX, EVEX_B, EVEX_K;
def rmbkz : AVX5128I<opc, MRMSrcMem, (outs VR512:$dst),
(ins KRC:$mask, x86scalar_mop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src}", BrdcstStr,
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
", $dst {${mask}} {z}|$dst {${mask}} {z}, ${src}",
BrdcstStr, "}"),
[]>, EVEX, EVEX_B, EVEX_KZ;
}
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
}
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
defm VPABSDZ : avx512_vpabs<0x1E, "vpabsd", v16i32, VK16WM, VR512,
i512mem, i32mem, "{1to16}">, EVEX_V512,
EVEX_CD8<32, CD8VF>;
defm VPABSQZ : avx512_vpabs<0x1F, "vpabsq", v8i64, VK8WM, VR512,
i512mem, i64mem, "{1to8}">, EVEX_V512, VEX_W,
EVEX_CD8<64, CD8VF>;
def : Pat<(xor
(bc_v16i32 (v16i1sextv16i32)),
(bc_v16i32 (add (v16i32 VR512:$src), (v16i1sextv16i32)))),
(VPABSDZrr VR512:$src)>;
def : Pat<(xor
(bc_v8i64 (v8i1sextv8i64)),
(bc_v8i64 (add (v8i64 VR512:$src), (v8i1sextv8i64)))),
(VPABSQZrr VR512:$src)>;
AVX-512: Converted to Unix style llvm-svn: 190851
2013-09-17 15:34:34 +08:00
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
def : Pat<(v16i32 (int_x86_avx512_mask_pabs_d_512 (v16i32 VR512:$src),
(v16i32 immAllZerosV), (i16 -1))),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
(VPABSDZrr VR512:$src)>;
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
def : Pat<(v8i64 (int_x86_avx512_mask_pabs_q_512 (v8i64 VR512:$src),
(bc_v8i64 (v16i32 immAllZerosV)), (i8 -1))),
AVX-512: Implemented masking for integer arithmetic & logic instructions. By Robert Khasanov rob.khasanov@gmail.com llvm-svn: 204906
2014-03-27 17:45:08 +08:00
(VPABSQZrr VR512:$src)>;
AVX-512: Added more intrinsics for pmin/pmax, pabs, blend, pmuldq. llvm-svn: 198745
2014-01-08 18:54:22 +08:00
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
multiclass avx512_conflict<bits<8> opc, string OpcodeStr,
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
RegisterClass RC, RegisterClass KRC,
X86MemOperand x86memop,
X86MemOperand x86scalar_mop, string BrdcstStr> {
[X86] Add IntrNoMem to the AVX512 conflict intrinsics. llvm-svn: 226897
2015-01-23 14:11:45 +08:00
let hasSideEffects = 0 in {
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
def rr : AVX5128I<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, ${dst} |${dst}, $src}"),
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
[]>, EVEX;
[X86] Add IntrNoMem to the AVX512 conflict intrinsics. llvm-svn: 226897
2015-01-23 14:11:45 +08:00
let mayLoad = 1 in
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
def rm : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins x86memop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{$src, ${dst}|${dst}, $src}"),
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
[]>, EVEX;
[X86] Add IntrNoMem to the AVX512 conflict intrinsics. llvm-svn: 226897
2015-01-23 14:11:45 +08:00
let mayLoad = 1 in
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
def rmb : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins x86scalar_mop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src}", BrdcstStr,
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
", ${dst}|${dst}, ${src}", BrdcstStr, "}"),
[]>, EVEX, EVEX_B;
def rrkz : AVX5128I<opc, MRMSrcReg, (outs RC:$dst),
(ins KRC:$mask, RC:$src),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src}"),
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
[]>, EVEX, EVEX_KZ;
[X86] Add IntrNoMem to the AVX512 conflict intrinsics. llvm-svn: 226897
2015-01-23 14:11:45 +08:00
let mayLoad = 1 in
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
def rmkz : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins KRC:$mask, x86memop:$src),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src, ${dst} {${mask}} {z}|${dst} {${mask}} {z}, $src}"),
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
[]>, EVEX, EVEX_KZ;
[X86] Add IntrNoMem to the AVX512 conflict intrinsics. llvm-svn: 226897
2015-01-23 14:11:45 +08:00
let mayLoad = 1 in
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
def rmbkz : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins KRC:$mask, x86scalar_mop:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src}", BrdcstStr,
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
", ${dst} {${mask}} {z}|${dst} {${mask}} {z}, ${src}",
BrdcstStr, "}"),
[]>, EVEX, EVEX_KZ, EVEX_B;
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
let Constraints = "$src1 = $dst" in {
def rrk : AVX5128I<opc, MRMSrcReg, (outs RC:$dst),
(ins RC:$src1, KRC:$mask, RC:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, ${dst} {${mask}}|${dst} {${mask}}, $src2}"),
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
[]>, EVEX, EVEX_K;
[X86] Add IntrNoMem to the AVX512 conflict intrinsics. llvm-svn: 226897
2015-01-23 14:11:45 +08:00
let mayLoad = 1 in
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
def rmk : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, KRC:$mask, x86memop:$src2),
!strconcat(OpcodeStr,
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
"\t{$src2, ${dst} {${mask}}|${dst} {${mask}}, $src2}"),
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
[]>, EVEX, EVEX_K;
[X86] Add IntrNoMem to the AVX512 conflict intrinsics. llvm-svn: 226897
2015-01-23 14:11:45 +08:00
let mayLoad = 1 in
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
def rmbk : AVX5128I<opc, MRMSrcMem, (outs RC:$dst),
(ins RC:$src1, KRC:$mask, x86scalar_mop:$src2),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr, "\t{${src2}", BrdcstStr,
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
", ${dst} {${mask}}|${dst} {${mask}}, ${src2}", BrdcstStr, "}"),
[]>, EVEX, EVEX_K, EVEX_B;
[X86] Add IntrNoMem to the AVX512 conflict intrinsics. llvm-svn: 226897
2015-01-23 14:11:45 +08:00
}
}
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
}
let Predicates = [HasCDI] in {
defm VPCONFLICTD : avx512_conflict<0xC4, "vpconflictd", VR512, VK16WM,
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
i512mem, i32mem, "{1to16}">,
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
EVEX_V512, EVEX_CD8<32, CD8VF>;
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
defm VPCONFLICTQ : avx512_conflict<0xC4, "vpconflictq", VR512, VK8WM,
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
i512mem, i64mem, "{1to8}">,
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
AVX-512: added VPCONFLICT instruction and intrinsics, added EVEX_KZ to tablegen llvm-svn: 193959
2013-11-03 21:46:31 +08:00
}
AVX-512: Changed intrinsics of VPCONFLICT to match GCC builtin form llvm-svn: 196914
2013-12-10 19:58:35 +08:00
def : Pat<(int_x86_avx512_mask_conflict_d_512 VR512:$src2, VR512:$src1,
GR16:$mask),
(VPCONFLICTDrrk VR512:$src1,
(v16i1 (COPY_TO_REGCLASS GR16:$mask, VK16WM)), VR512:$src2)>;
def : Pat<(int_x86_avx512_mask_conflict_q_512 VR512:$src2, VR512:$src1,
GR8:$mask),
(VPCONFLICTQrrk VR512:$src1,
(v8i1 (COPY_TO_REGCLASS GR8:$mask, VK8WM)), VR512:$src2)>;
AVX-512: insert element to mask vector; store i1 data Implemented INSERT_VECTOR_ELT operation for v16i1 and v8i1 vectors; Implemented "store" for i1 type llvm-svn: 205850
2014-04-09 20:37:50 +08:00
Add AVX512 masked leadz instrinsic support. llvm-svn: 210652
2014-06-11 20:54:45 +08:00
let Predicates = [HasCDI] in {
defm VPLZCNTD : avx512_conflict<0x44, "vplzcntd", VR512, VK16WM,
i512mem, i32mem, "{1to16}">,
EVEX_V512, EVEX_CD8<32, CD8VF>;
defm VPLZCNTQ : avx512_conflict<0x44, "vplzcntq", VR512, VK8WM,
i512mem, i64mem, "{1to8}">,
EVEX_V512, VEX_W, EVEX_CD8<64, CD8VF>;
}
def : Pat<(int_x86_avx512_mask_lzcnt_d_512 VR512:$src2, VR512:$src1,
GR16:$mask),
(VPLZCNTDrrk VR512:$src1,
(v16i1 (COPY_TO_REGCLASS GR16:$mask, VK16WM)), VR512:$src2)>;
def : Pat<(int_x86_avx512_mask_lzcnt_q_512 VR512:$src2, VR512:$src1,
GR8:$mask),
(VPLZCNTQrrk VR512:$src1,
(v8i1 (COPY_TO_REGCLASS GR8:$mask, VK8WM)), VR512:$src2)>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
def : Pat<(v16i32 (ctlz (loadv16i32 addr:$src))),
Hook up vector int_ctlz for AVX512. llvm-svn: 211024
2014-06-16 22:12:28 +08:00
(VPLZCNTDrm addr:$src)>;
def : Pat<(v16i32 (ctlz (v16i32 VR512:$src))),
(VPLZCNTDrr VR512:$src)>;
[X86] Remove 'memop' uses from AVX512. Use 'load' instead. llvm-svn: 228562
2015-02-09 12:04:50 +08:00
def : Pat<(v8i64 (ctlz (loadv8i64 addr:$src))),
Hook up vector int_ctlz for AVX512. llvm-svn: 211024
2014-06-16 22:12:28 +08:00
(VPLZCNTQrm addr:$src)>;
def : Pat<(v8i64 (ctlz (v8i64 VR512:$src))),
(VPLZCNTQrr VR512:$src)>;
AVX-512: insert element to mask vector; store i1 data Implemented INSERT_VECTOR_ELT operation for v16i1 and v8i1 vectors; Implemented "store" for i1 type llvm-svn: 205850
2014-04-09 20:37:50 +08:00
def : Pat<(store (i1 -1), addr:$dst), (MOV8mi addr:$dst, (i8 1))>;
def : Pat<(store (i1 1), addr:$dst), (MOV8mi addr:$dst, (i8 1))>;
def : Pat<(store (i1 0), addr:$dst), (MOV8mi addr:$dst, (i8 0))>;
AVX-512: store and truncstore for i1 values llvm-svn: 206897
2014-04-22 22:13:10 +08:00
def : Pat<(store VK1:$src, addr:$dst),
AVX-512: Changes in operations on masks registers for KNL and SKX - Added KSHIFTB/D/Q for skx - Added KORTESTB/D/Q for skx - Fixed store operation for v8i1 type for KNL - Store size of v8i1, v4i1 and v2i1 are changed to 8 bits llvm-svn: 227043
2015-01-25 20:47:15 +08:00
(MOV8mr addr:$dst,
(EXTRACT_SUBREG (KMOVWrk (COPY_TO_REGCLASS VK1:$src, VK16)),
sub_8bit))>, Requires<[HasAVX512, NoDQI]>;
def : Pat<(store VK8:$src, addr:$dst),
(MOV8mr addr:$dst,
(EXTRACT_SUBREG (KMOVWrk (COPY_TO_REGCLASS VK8:$src, VK16)),
sub_8bit))>, Requires<[HasAVX512, NoDQI]>;
AVX-512: store and truncstore for i1 values llvm-svn: 206897
2014-04-22 22:13:10 +08:00
def truncstorei1 : PatFrag<(ops node:$val, node:$ptr),
(truncstore node:$val, node:$ptr), [{
return cast<StoreSDNode>(N)->getMemoryVT() == MVT::i1;
}]>;
def : Pat<(truncstorei1 GR8:$src, addr:$dst),
(MOV8mr addr:$dst, GR8:$src)>;
AVX-512-SKX: Added instruction VPMOVM2B/W/D/Q. This instruction allows to broadacst mask vector to data vector. llvm-svn: 219083
2014-10-05 22:11:08 +08:00
multiclass cvt_by_vec_width<bits<8> opc, X86VectorVTInfo Vec, string OpcodeStr > {
def rr : AVX512XS8I<opc, MRMDestReg, (outs Vec.RC:$dst), (ins Vec.KRC:$src),
Remove space before tab in all AVX512 mnemonic strings. llvm-svn: 222778
2014-11-26 04:11:23 +08:00
!strconcat(OpcodeStr##Vec.Suffix, "\t{$src, $dst|$dst, $src}"),
AVX-512-SKX: Added instruction VPMOVM2B/W/D/Q. This instruction allows to broadacst mask vector to data vector. llvm-svn: 219083
2014-10-05 22:11:08 +08:00
[(set Vec.RC:$dst, (Vec.VT (X86vsext Vec.KRC:$src)))]>, EVEX;
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512-SKX: Added instruction VPMOVM2B/W/D/Q. This instruction allows to broadacst mask vector to data vector. llvm-svn: 219083
2014-10-05 22:11:08 +08:00
multiclass cvt_mask_by_elt_width<bits<8> opc, AVX512VLVectorVTInfo VTInfo,
string OpcodeStr, Predicate prd> {
let Predicates = [prd] in
defm Z : cvt_by_vec_width<opc, VTInfo.info512, OpcodeStr>, EVEX_V512;
let Predicates = [prd, HasVLX] in {
defm Z256 : cvt_by_vec_width<opc, VTInfo.info256, OpcodeStr>, EVEX_V256;
defm Z128 : cvt_by_vec_width<opc, VTInfo.info128, OpcodeStr>, EVEX_V128;
}
}
multiclass avx512_convert_mask_to_vector<string OpcodeStr> {
defm NAME##B : cvt_mask_by_elt_width<0x28, avx512vl_i8_info, OpcodeStr,
HasBWI>;
defm NAME##W : cvt_mask_by_elt_width<0x28, avx512vl_i16_info, OpcodeStr,
HasBWI>, VEX_W;
defm NAME##D : cvt_mask_by_elt_width<0x38, avx512vl_i32_info, OpcodeStr,
HasDQI>;
defm NAME##Q : cvt_mask_by_elt_width<0x38, avx512vl_i64_info, OpcodeStr,
HasDQI>, VEX_W;
}
[X86] Clean up whitespace as well as minor coding style llvm-svn: 223339
2014-12-04 13:20:33 +08:00
AVX-512-SKX: Added instruction VPMOVM2B/W/D/Q. This instruction allows to broadacst mask vector to data vector. llvm-svn: 219083
2014-10-05 22:11:08 +08:00
defm VPMOVM2 : avx512_convert_mask_to_vector<"vpmovm2">;
AVX-512: Added all forms of COMPRESS instruction + intrinsics + tests llvm-svn: 224019
2014-12-11 23:02:24 +08:00
//===----------------------------------------------------------------------===//
// AVX-512 - COMPRESS and EXPAND
//
multiclass compress_by_vec_width<bits<8> opc, X86VectorVTInfo _,
string OpcodeStr> {
def rrkz : AVX5128I<opc, MRMDestReg, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src),
OpcodeStr # "\t{$src, $dst {${mask}} {z}|$dst {${mask}} {z}, $src}",
[(set _.RC:$dst, (_.VT (X86compress _.KRCWM:$mask, _.RC:$src,
_.ImmAllZerosV)))]>, EVEX_KZ;
let Constraints = "$src0 = $dst" in
def rrk : AVX5128I<opc, MRMDestReg, (outs _.RC:$dst),
(ins _.RC:$src0, _.KRCWM:$mask, _.RC:$src),
OpcodeStr # "\t{$src, $dst {${mask}} |$dst {${mask}}, $src}",
[(set _.RC:$dst, (_.VT (X86compress _.KRCWM:$mask, _.RC:$src,
_.RC:$src0)))]>, EVEX_K;
let mayStore = 1 in {
def mrk : AVX5128I<opc, MRMDestMem, (outs),
(ins _.MemOp:$dst, _.KRCWM:$mask, _.RC:$src),
OpcodeStr # "\t{$src, $dst {${mask}} |$dst {${mask}}, $src}",
[(store (_.VT (X86compress _.KRCWM:$mask, _.RC:$src, undef)),
addr:$dst)]>,
EVEX_K, EVEX_CD8<_.EltSize, CD8VT1>;
}
}
multiclass compress_by_elt_width<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
defm Z : compress_by_vec_width<opc, VTInfo.info512, OpcodeStr>, EVEX_V512;
let Predicates = [HasVLX] in {
defm Z256 : compress_by_vec_width<opc, VTInfo.info256, OpcodeStr>, EVEX_V256;
defm Z128 : compress_by_vec_width<opc, VTInfo.info128, OpcodeStr>, EVEX_V128;
}
}
defm VPCOMPRESSD : compress_by_elt_width <0x8B, "vpcompressd", avx512vl_i32_info>,
EVEX;
defm VPCOMPRESSQ : compress_by_elt_width <0x8B, "vpcompressq", avx512vl_i64_info>,
EVEX, VEX_W;
defm VCOMPRESSPS : compress_by_elt_width <0x8A, "vcompressps", avx512vl_f32_info>,
EVEX;
defm VCOMPRESSPD : compress_by_elt_width <0x8A, "vcompresspd", avx512vl_f64_info>,
EVEX, VEX_W;
AVX-512: Added EXPAND instructions and intrinsics. llvm-svn: 224241
2014-12-15 18:03:52 +08:00
// expand
multiclass expand_by_vec_width<bits<8> opc, X86VectorVTInfo _,
string OpcodeStr> {
def rrkz : AVX5128I<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.RC:$src),
OpcodeStr # "\t{$src, $dst {${mask}} {z}|$dst {${mask}} {z}, $src}",
[(set _.RC:$dst, (_.VT (X86expand _.KRCWM:$mask, (_.VT _.RC:$src),
_.ImmAllZerosV)))]>, EVEX_KZ;
let Constraints = "$src0 = $dst" in
def rrk : AVX5128I<opc, MRMSrcReg, (outs _.RC:$dst),
(ins _.RC:$src0, _.KRCWM:$mask, _.RC:$src),
OpcodeStr # "\t{$src, $dst {${mask}} |$dst {${mask}}, $src}",
[(set _.RC:$dst, (_.VT (X86expand _.KRCWM:$mask,
(_.VT _.RC:$src), _.RC:$src0)))]>, EVEX_K;
let mayLoad = 1, Constraints = "$src0 = $dst" in
def rmk : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.RC:$src0, _.KRCWM:$mask, _.MemOp:$src),
OpcodeStr # "\t{$src, $dst {${mask}} |$dst {${mask}}, $src}",
[(set _.RC:$dst, (_.VT (X86expand _.KRCWM:$mask,
(_.VT (bitconvert
(_.LdFrag addr:$src))),
_.RC:$src0)))]>,
EVEX_K, EVEX_CD8<_.EltSize, CD8VT1>;
let mayLoad = 1 in
def rmkz : AVX5128I<opc, MRMSrcMem, (outs _.RC:$dst),
(ins _.KRCWM:$mask, _.MemOp:$src),
OpcodeStr # "\t{$src, $dst {${mask}} {z}|$dst {${mask}} {z}, $src}",
[(set _.RC:$dst, (_.VT (X86expand _.KRCWM:$mask,
(_.VT (bitconvert (_.LdFrag addr:$src))),
_.ImmAllZerosV)))]>,
EVEX_KZ, EVEX_CD8<_.EltSize, CD8VT1>;
}
multiclass expand_by_elt_width<bits<8> opc, string OpcodeStr,
AVX512VLVectorVTInfo VTInfo> {
defm Z : expand_by_vec_width<opc, VTInfo.info512, OpcodeStr>, EVEX_V512;
let Predicates = [HasVLX] in {
defm Z256 : expand_by_vec_width<opc, VTInfo.info256, OpcodeStr>, EVEX_V256;
defm Z128 : expand_by_vec_width<opc, VTInfo.info128, OpcodeStr>, EVEX_V128;
}
}
defm VPEXPANDD : expand_by_elt_width <0x89, "vpexpandd", avx512vl_i32_info>,
EVEX;
defm VPEXPANDQ : expand_by_elt_width <0x89, "vpexpandq", avx512vl_i64_info>,
EVEX, VEX_W;
defm VEXPANDPS : expand_by_elt_width <0x88, "vexpandps", avx512vl_f32_info>,
EVEX;
defm VEXPANDPD : expand_by_elt_width <0x88, "vexpandpd", avx512vl_f64_info>,
EVEX, VEX_W;