maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/IR/AutoUpgrade.cpp

1983 lines
87 KiB
C++
Raw Normal View History

This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
//===-- AutoUpgrade.cpp - Implement auto-upgrade helper functions ---------===//
//
// The LLVM Compiler Infrastructure
//
Remove attribution from file headers, per discussion on llvmdev. llvm-svn: 45418
2007-12-30 04:36:04 +08:00
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
//
//===----------------------------------------------------------------------===//
//
[X86, AVX] replace vinsertf128 intrinsics with generic shuffles We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is the sibling patch for the Clang half of this change: http://reviews.llvm.org/D8088 Differential Revision: http://reviews.llvm.org/D8086 llvm-svn: 231794
2015-03-11 00:08:36 +08:00
// This file implements the auto-upgrade helper functions.
// This is where deprecated IR intrinsics and other IR features are updated to
// current specifications.
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
//
//===----------------------------------------------------------------------===//
[Layering] Move AutoUpgrade.h into the IR library where its implementation already lives. llvm-svn: 202961
2014-03-05 18:34:14 +08:00
#include "llvm/IR/AutoUpgrade.h"
[Modules] Move CFG.h to the IR library as it defines graph traits over IR types. llvm-svn: 202827
2014-03-04 19:45:46 +08:00
#include "llvm/IR/CFG.h"
[Modules] Move CallSite into the IR library where it belogs. It is abstracting between a CallInst and an InvokeInst, both of which are IR concepts. llvm-svn: 202816
2014-03-04 19:01:28 +08:00
#include "llvm/IR/CallSite.h"
Move all of the header files which are involved in modelling the LLVM IR into their new header subdirectory: include/llvm/IR. This matches the directory structure of lib, and begins to correct a long standing point of file layout clutter in LLVM. There are still more header files to move here, but I wanted to handle them in separate commits to make tracking what files make sense at each layer easier. The only really questionable files here are the target intrinsic tablegen files. But that's a battle I'd rather not fight today. I've updated both CMake and Makefile build systems (I think, and my tests think, but I may have missed something). I've also re-sorted the includes throughout the project. I'll be committing updates to Clang, DragonEgg, and Polly momentarily. llvm-svn: 171366
2013-01-02 19:36:10 +08:00
#include "llvm/IR/Constants.h"
[cleanup] Re-sort all the #include lines in LLVM using utils/sort_includes.py. I clearly haven't done this in a while, so more changed than usual. This even uncovered a missing include from the InstrProf library that I've added. No functionality changed here, just mechanical cleanup of the include order. llvm-svn: 225974
2015-01-14 19:23:27 +08:00
#include "llvm/IR/DIBuilder.h"
[Layering] Move DebugInfo.h into the IR library where its implementation already lives. llvm-svn: 203046
2014-03-06 08:46:21 +08:00
#include "llvm/IR/DebugInfo.h"
Report a warning when dropping outdated debug info metadata. Use DiagnosticInfo to emit the warning. llvm-svn: 199346
2014-01-16 09:51:12 +08:00
#include "llvm/IR/DiagnosticInfo.h"
Move all of the header files which are involved in modelling the LLVM IR into their new header subdirectory: include/llvm/IR. This matches the directory structure of lib, and begins to correct a long standing point of file layout clutter in LLVM. There are still more header files to move here, but I wanted to handle them in separate commits to make tracking what files make sense at each layer easier. The only really questionable files here are the target intrinsic tablegen files. But that's a battle I'd rather not fight today. I've updated both CMake and Makefile build systems (I think, and my tests think, but I may have missed something). I've also re-sorted the includes throughout the project. I'll be committing updates to Clang, DragonEgg, and Polly momentarily. llvm-svn: 171366
2013-01-02 19:36:10 +08:00
#include "llvm/IR/Function.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Instruction.h"
#include "llvm/IR/IntrinsicInst.h"
#include "llvm/IR/LLVMContext.h"
#include "llvm/IR/Module.h"
assert(0) -> LLVM_UNREACHABLE. Make llvm_unreachable take an optional string, thus moving the cerr<< out of line. LLVM_UNREACHABLE is now a simple wrapper that makes the message go away for NDEBUG builds. llvm-svn: 75379
2009-07-12 04:10:48 +08:00
#include "llvm/Support/ErrorHandling.h"
[ARM][NEON] Use address space in vld([1234]|[234]lane) and vst([1234]|[234]lane) instructions This commit changes the interface of the vld[1234], vld[234]lane, and vst[1234], vst[234]lane ARM neon intrinsics and associates an address space with the pointer that these intrinsics take. This changes, e.g., <2 x i32> @llvm.arm.neon.vld1.v2i32(i8*, i32) to <2 x i32> @llvm.arm.neon.vld1.v2i32.p0i8(i8*, i32) This change ensures that address spaces are fully taken into account in the ARM target during lowering of interleaved loads and stores. Differential Revision: http://reviews.llvm.org/D12985 llvm-svn: 248887
2015-09-30 18:56:37 +08:00
#include "llvm/Support/Regex.h"
Unbreak build with gcc 4.3: provide missed includes and silence most annoying warnings. llvm-svn: 47367
2008-02-20 19:08:44 +08:00
#include <cstring>
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
using namespace llvm;
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
static void rename(GlobalValue *GV) { GV->setName(GV->getName() + ".old"); }
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
// Upgrade the declarations of the SSE4.1 functions whose arguments have
// changed their type from v4f32 to v2i64.
static bool UpgradeSSE41Function(Function* F, Intrinsic::ID IID,
Function *&NewFn) {
// Check whether this is an old version of the function, which received
// v4f32 arguments.
Type *Arg0Type = F->getFunctionType()->getParamType(0);
if (Arg0Type != VectorType::get(Type::getFloatTy(F->getContext()), 4))
return false;
// Yes, it's old, replace it with new version.
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(), IID);
return true;
}
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
// Upgrade the declarations of intrinsic functions whose 8-bit immediate mask
// arguments have changed their type from i32 to i8.
static bool UpgradeX86IntrinsicsWith8BitMask(Function *F, Intrinsic::ID IID,
Function *&NewFn) {
// Check that the last argument is an i32.
Type *LastArgType = F->getFunctionType()->getParamType(
F->getFunctionType()->getNumParams() - 1);
if (!LastArgType->isIntegerTy(32))
return false;
// Move this function aside and map down.
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(), IID);
return true;
}
Bring back int_x86_sse2_movl_dq intrinsic for backward compatibility. Make sure it's auto-upgraded to a shufflevector instruction. llvm-svn: 45131
2007-12-18 06:33:23 +08:00
static bool UpgradeIntrinsicFunction1(Function *F, Function *&NewFn) {
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
assert(F && "Illegal to upgrade a non-existent Function.");
// Quickly eliminate it, if it's not a candidate.
Move setName after accessing Name llvm-svn: 274862
2016-07-09 00:09:51 +08:00
StringRef Name = F->getName();
rework the remaining autoupgrade logic to use a StringRef instead of creating a temporary std::string for every function being checked. llvm-svn: 133355
2011-06-19 02:56:39 +08:00
if (Name.size() <= 8 || !Name.startswith("llvm."))
Bring back int_x86_sse2_movl_dq intrinsic for backward compatibility. Make sure it's auto-upgraded to a shufflevector instruction. llvm-svn: 45131
2007-12-18 06:33:23 +08:00
return false;
rework the remaining autoupgrade logic to use a StringRef instead of creating a temporary std::string for every function being checked. llvm-svn: 133355
2011-06-19 02:56:39 +08:00
Name = Name.substr(5); // Strip off "llvm."
Eli managed to kill off llvm.membarrier in llvm 3.0 also, this means that mainline needs no autoupgrade logic for intrinsics yet, woohoo! llvm-svn: 145178
2011-11-27 16:42:07 +08:00
rework the remaining autoupgrade logic to use a StringRef instead of creating a temporary std::string for every function being checked. llvm-svn: 133355
2011-06-19 02:56:39 +08:00
switch (Name[0]) {
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
default: break;
This is one of the first steps at moving to replace target-dependent intrinsics with target-indepdent intrinsics. The first instruction(s) to be handled are the vector versions of count leading zeros (ctlz). The changes here are to clang so that it generates a target independent vector ctlz when it sees an ARM dependent vector ctlz. The changes in llvm are to match the target independent vector ctlz and in VMCore/AutoUpgrade.cpp to update any existing bc files containing ARM dependent vector ctlzs with target-independent ctlzs. There are also changes to an existing test case in llvm for ARM vector count instructions and a new test for the bitcode upgrade. <rdar://problem/11831778> There is deliberately no test for the change to clang, as so far as I know, no consensus has been reached regarding how to test neon instructions in clang; q.v. <rdar://problem/8762292> llvm-svn: 160200
2012-07-14 07:25:25 +08:00
case 'a': {
if (Name.startswith("arm.neon.vclz")) {
Type* args[2] = {
Remove trailing whitespace, fix file path in comment llvm-svn: 186766
2013-07-21 01:46:00 +08:00
F->arg_begin()->getType(),
This is one of the first steps at moving to replace target-dependent intrinsics with target-indepdent intrinsics. The first instruction(s) to be handled are the vector versions of count leading zeros (ctlz). The changes here are to clang so that it generates a target independent vector ctlz when it sees an ARM dependent vector ctlz. The changes in llvm are to match the target independent vector ctlz and in VMCore/AutoUpgrade.cpp to update any existing bc files containing ARM dependent vector ctlzs with target-independent ctlzs. There are also changes to an existing test case in llvm for ARM vector count instructions and a new test for the bitcode upgrade. <rdar://problem/11831778> There is deliberately no test for the change to clang, as so far as I know, no consensus has been reached regarding how to test neon instructions in clang; q.v. <rdar://problem/8762292> llvm-svn: 160200
2012-07-14 07:25:25 +08:00
Type::getInt1Ty(F->getContext())
};
// Can't use Intrinsic::getDeclaration here as it adds a ".i1" to
// the end of the name. Change name from llvm.arm.neon.vclz.* to
// llvm.ctlz.*
FunctionType* fType = FunctionType::get(F->getReturnType(), args, false);
Remove trailing whitespace, fix file path in comment llvm-svn: 186766
2013-07-21 01:46:00 +08:00
NewFn = Function::Create(fType, F->getLinkage(),
This is one of the first steps at moving to replace target-dependent intrinsics with target-indepdent intrinsics. The first instruction(s) to be handled are the vector versions of count leading zeros (ctlz). The changes here are to clang so that it generates a target independent vector ctlz when it sees an ARM dependent vector ctlz. The changes in llvm are to match the target independent vector ctlz and in VMCore/AutoUpgrade.cpp to update any existing bc files containing ARM dependent vector ctlzs with target-independent ctlzs. There are also changes to an existing test case in llvm for ARM vector count instructions and a new test for the bitcode upgrade. <rdar://problem/11831778> There is deliberately no test for the change to clang, as so far as I know, no consensus has been reached regarding how to test neon instructions in clang; q.v. <rdar://problem/8762292> llvm-svn: 160200
2012-07-14 07:25:25 +08:00
"llvm.ctlz." + Name.substr(14), F->getParent());
return true;
}
More replacing of target-dependent intrinsics with target-indepdent intrinsics. The second instruction(s) to be handled are the vector versions of count set bits (ctpop). The changes here are to clang so that it generates a target independent vector ctpop when it sees an ARM dependent vector bits set count. The changes in llvm are to match the target independent vector ctpop and in VMCore/AutoUpgrade.cpp to update any existing bc files containing ARM dependent vector pop counts with target-independent ctpops. There are also changes to an existing test case in llvm for ARM vector count instructions and to a test for the bitcode upgrade. <rdar://problem/11892519> There is deliberately no test for the change to clang, as so far as I know, no consensus has been reached regarding how to test neon instructions in clang; q.v. <rdar://problem/8762292> llvm-svn: 160410
2012-07-18 08:02:16 +08:00
if (Name.startswith("arm.neon.vcnt")) {
NewFn = Intrinsic::getDeclaration(F->getParent(), Intrinsic::ctpop,
F->arg_begin()->getType());
return true;
}
[ARM][NEON] Use address space in vld([1234]|[234]lane) and vst([1234]|[234]lane) instructions This commit changes the interface of the vld[1234], vld[234]lane, and vst[1234], vst[234]lane ARM neon intrinsics and associates an address space with the pointer that these intrinsics take. This changes, e.g., <2 x i32> @llvm.arm.neon.vld1.v2i32(i8*, i32) to <2 x i32> @llvm.arm.neon.vld1.v2i32.p0i8(i8*, i32) This change ensures that address spaces are fully taken into account in the ARM target during lowering of interleaved loads and stores. Differential Revision: http://reviews.llvm.org/D12985 llvm-svn: 248887
2015-09-30 18:56:37 +08:00
Regex vldRegex("^arm\\.neon\\.vld([1234]|[234]lane)\\.v[a-z0-9]*$");
if (vldRegex.match(Name)) {
auto fArgs = F->getFunctionType()->params();
SmallVector<Type *, 4> Tys(fArgs.begin(), fArgs.end());
// Can't use Intrinsic::getDeclaration here as the return types might
// then only be structurally equal.
FunctionType* fType = FunctionType::get(F->getReturnType(), Tys, false);
NewFn = Function::Create(fType, F->getLinkage(),
"llvm." + Name + ".p0i8", F->getParent());
return true;
}
Regex vstRegex("^arm\\.neon\\.vst([1234]|[234]lane)\\.v[a-z0-9]*$");
if (vstRegex.match(Name)) {
Make a bunch of static arrays const. llvm-svn: 250642
2015-10-18 13:15:34 +08:00
static const Intrinsic::ID StoreInts[] = {Intrinsic::arm_neon_vst1,
Intrinsic::arm_neon_vst2,
Intrinsic::arm_neon_vst3,
Intrinsic::arm_neon_vst4};
[ARM][NEON] Use address space in vld([1234]|[234]lane) and vst([1234]|[234]lane) instructions This commit changes the interface of the vld[1234], vld[234]lane, and vst[1234], vst[234]lane ARM neon intrinsics and associates an address space with the pointer that these intrinsics take. This changes, e.g., <2 x i32> @llvm.arm.neon.vld1.v2i32(i8*, i32) to <2 x i32> @llvm.arm.neon.vld1.v2i32.p0i8(i8*, i32) This change ensures that address spaces are fully taken into account in the ARM target during lowering of interleaved loads and stores. Differential Revision: http://reviews.llvm.org/D12985 llvm-svn: 248887
2015-09-30 18:56:37 +08:00
Make a bunch of static arrays const. llvm-svn: 250642
2015-10-18 13:15:34 +08:00
static const Intrinsic::ID StoreLaneInts[] = {
Intrinsic::arm_neon_vst2lane, Intrinsic::arm_neon_vst3lane,
Intrinsic::arm_neon_vst4lane
};
[ARM][NEON] Use address space in vld([1234]|[234]lane) and vst([1234]|[234]lane) instructions This commit changes the interface of the vld[1234], vld[234]lane, and vst[1234], vst[234]lane ARM neon intrinsics and associates an address space with the pointer that these intrinsics take. This changes, e.g., <2 x i32> @llvm.arm.neon.vld1.v2i32(i8*, i32) to <2 x i32> @llvm.arm.neon.vld1.v2i32.p0i8(i8*, i32) This change ensures that address spaces are fully taken into account in the ARM target during lowering of interleaved loads and stores. Differential Revision: http://reviews.llvm.org/D12985 llvm-svn: 248887
2015-09-30 18:56:37 +08:00
auto fArgs = F->getFunctionType()->params();
Type *Tys[] = {fArgs[0], fArgs[1]};
if (Name.find("lane") == StringRef::npos)
NewFn = Intrinsic::getDeclaration(F->getParent(),
StoreInts[fArgs.size() - 3], Tys);
else
NewFn = Intrinsic::getDeclaration(F->getParent(),
StoreLaneInts[fArgs.size() - 5], Tys);
return true;
}
[AArch64] [ARM] Make a target-independent llvm.thread.pointer intrinsic. Both AArch64 and ARM support llvm.<arch>.thread.pointer intrinsics that just return the thread pointer. I have a pending patch that does the same for SystemZ (D19054), and there are many more targets that could benefit from one. This patch merges the ARM and AArch64 intrinsics into a single target independent one that will also be used by subsequent targets. Differential Revision: http://reviews.llvm.org/D19098 llvm-svn: 266818
2016-04-20 04:51:05 +08:00
if (Name == "aarch64.thread.pointer" || Name == "arm.thread.pointer") {
NewFn = Intrinsic::getDeclaration(F->getParent(), Intrinsic::thread_pointer);
return true;
}
This is one of the first steps at moving to replace target-dependent intrinsics with target-indepdent intrinsics. The first instruction(s) to be handled are the vector versions of count leading zeros (ctlz). The changes here are to clang so that it generates a target independent vector ctlz when it sees an ARM dependent vector ctlz. The changes in llvm are to match the target independent vector ctlz and in VMCore/AutoUpgrade.cpp to update any existing bc files containing ARM dependent vector ctlzs with target-independent ctlzs. There are also changes to an existing test case in llvm for ARM vector count instructions and a new test for the bitcode upgrade. <rdar://problem/11831778> There is deliberately no test for the change to clang, as so far as I know, no consensus has been reached regarding how to test neon instructions in clang; q.v. <rdar://problem/8762292> llvm-svn: 160200
2012-07-14 07:25:25 +08:00
break;
}
[ARM][NEON] Use address space in vld([1234]|[234]lane) and vst([1234]|[234]lane) instructions This commit changes the interface of the vld[1234], vld[234]lane, and vst[1234], vst[234]lane ARM neon intrinsics and associates an address space with the pointer that these intrinsics take. This changes, e.g., <2 x i32> @llvm.arm.neon.vld1.v2i32(i8*, i32) to <2 x i32> @llvm.arm.neon.vld1.v2i32.p0i8(i8*, i32) This change ensures that address spaces are fully taken into account in the ARM target during lowering of interleaved loads and stores. Differential Revision: http://reviews.llvm.org/D12985 llvm-svn: 248887
2015-09-30 18:56:37 +08:00
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
case 'c': {
if (Name.startswith("ctlz.") && F->arg_size() == 1) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Don't rely in there being one argument before we've actually identified a function to upgrade. Also, simplify the code a bit at the expense of one line. llvm-svn: 146368
2011-12-12 18:57:20 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(), Intrinsic::ctlz,
F->arg_begin()->getType());
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
return true;
}
if (Name.startswith("cttz.") && F->arg_size() == 1) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Don't rely in there being one argument before we've actually identified a function to upgrade. Also, simplify the code a bit at the expense of one line. llvm-svn: 146368
2011-12-12 18:57:20 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(), Intrinsic::cttz,
F->arg_begin()->getType());
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
return true;
}
break;
}
Revert "Revert "Invariant start/end intrinsics overloaded for address space"" This reverts commit 32fc6488e48eafc0ca1bac1bd9cbf0008224d530. llvm-svn: 278609
2016-08-14 07:31:24 +08:00
case 'i': {
if (Name.startswith("invariant.start")) {
auto Args = F->getFunctionType()->params();
Type* ObjectPtr[1] = {Args[1]};
if (F->getName() !=
Intrinsic::getName(Intrinsic::invariant_start, ObjectPtr)) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Revert "Revert "Invariant start/end intrinsics overloaded for address space"" This reverts commit 32fc6488e48eafc0ca1bac1bd9cbf0008224d530. llvm-svn: 278609
2016-08-14 07:31:24 +08:00
NewFn = Intrinsic::getDeclaration(
F->getParent(), Intrinsic::invariant_start, ObjectPtr);
return true;
}
}
if (Name.startswith("invariant.end")) {
auto Args = F->getFunctionType()->params();
Type* ObjectPtr[1] = {Args[2]};
if (F->getName() !=
Intrinsic::getName(Intrinsic::invariant_end, ObjectPtr)) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Revert "Revert "Invariant start/end intrinsics overloaded for address space"" This reverts commit 32fc6488e48eafc0ca1bac1bd9cbf0008224d530. llvm-svn: 278609
2016-08-14 07:31:24 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(),
Intrinsic::invariant_end, ObjectPtr);
return true;
}
}
break;
}
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
case 'm': {
if (Name.startswith("masked.load.")) {
Type *Tys[] = { F->getReturnType(), F->arg_begin()->getType() };
if (F->getName() != Intrinsic::getName(Intrinsic::masked_load, Tys)) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(),
Intrinsic::masked_load,
Tys);
return true;
}
}
if (Name.startswith("masked.store.")) {
auto Args = F->getFunctionType()->params();
Type *Tys[] = { Args[0], Args[1] };
if (F->getName() != Intrinsic::getName(Intrinsic::masked_store, Tys)) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(),
Intrinsic::masked_store,
Tys);
return true;
}
}
break;
}
Change objectsize intrinsic to accept different address spaces. Bitcasting everything to i8* won't work. Autoupgrade the old intrinsic declarations to use the new mangling. llvm-svn: 192117
2013-10-08 02:06:48 +08:00
case 'o':
// We only need to change the name to match the mangling including the
// address space.
if (F->arg_size() == 2 && Name.startswith("objectsize.")) {
Type *Tys[2] = { F->getReturnType(), F->arg_begin()->getType() };
if (F->getName() != Intrinsic::getName(Intrinsic::objectsize, Tys)) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Change objectsize intrinsic to accept different address spaces. Bitcasting everything to i8* won't work. Autoupgrade the old intrinsic declarations to use the new mangling. llvm-svn: 192117
2013-10-08 02:06:48 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(),
Intrinsic::objectsize, Tys);
return true;
}
}
break;
[SSP] Remove llvm.stackprotectorcheck. This is a cleanup patch for SSP support in LLVM. There is no functional change. llvm.stackprotectorcheck is not needed, because SelectionDAG isn't actually lowering it in SelectBasicBlock; rather, it adds check code in FinishBasicBlock, ignoring the position where the intrinsic is inserted (See FindSplitPointForStackProtector()). llvm-svn: 265851
2016-04-09 05:26:31 +08:00
case 's':
if (Name == "stackprotectorcheck") {
NewFn = nullptr;
return true;
}
Add a missing break statement. NFC. llvm-svn: 286203
2016-11-08 12:01:50 +08:00
break;
[SSP] Remove llvm.stackprotectorcheck. This is a cleanup patch for SSP support in LLVM. There is no functional change. llvm.stackprotectorcheck is not needed, because SelectionDAG isn't actually lowering it in SelectBasicBlock; rather, it adds check code in FinishBasicBlock, ignoring the position where the intrinsic is inserted (See FindSplitPointForStackProtector()). llvm-svn: 265851
2016-04-09 05:26:31 +08:00
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
case 'x': {
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
bool IsX86 = Name.startswith("x86.");
if (IsX86)
Name = Name.substr(4);
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
// All of the intrinsics matches below should be marked with which llvm
// version started autoupgrading them. At some point in the future we would
// like to use this information to remove upgrade code for some older
// intrinsics. It is currently undecided how we will determine that future
// point.
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
if (IsX86 &&
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
(Name.startswith("sse2.pcmpeq.") || // Added in 3.1
Name.startswith("sse2.pcmpgt.") || // Added in 3.1
Name.startswith("avx2.pcmpeq.") || // Added in 3.1
Name.startswith("avx2.pcmpgt.") || // Added in 3.1
Name.startswith("avx512.mask.pcmpeq.") || // Added in 3.9
Name.startswith("avx512.mask.pcmpgt.") || // Added in 3.9
[X86] Remove the scalar intrinsics for fadd/fsub/fdiv/fmul Summary: These intrinsics have been unused for clang for a while. This patch removes them. We auto upgrade them to extractelements, a scalar operation and then an insertelement. This matches the sequence used by clangs intrinsic file. Reviewers: zvi, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26660 llvm-svn: 287083
2016-11-16 13:24:10 +08:00
Name == "sse.add.ss" || // Added in 4.0
Name == "sse2.add.sd" || // Added in 4.0
Name == "sse.sub.ss" || // Added in 4.0
Name == "sse2.sub.sd" || // Added in 4.0
Name == "sse.mul.ss" || // Added in 4.0
Name == "sse2.mul.sd" || // Added in 4.0
Name == "sse.div.ss" || // Added in 4.0
Name == "sse2.div.sd" || // Added in 4.0
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
Name == "sse41.pmaxsb" || // Added in 3.9
Name == "sse2.pmaxs.w" || // Added in 3.9
Name == "sse41.pmaxsd" || // Added in 3.9
Name == "sse2.pmaxu.b" || // Added in 3.9
Name == "sse41.pmaxuw" || // Added in 3.9
Name == "sse41.pmaxud" || // Added in 3.9
Name == "sse41.pminsb" || // Added in 3.9
Name == "sse2.pmins.w" || // Added in 3.9
Name == "sse41.pminsd" || // Added in 3.9
Name == "sse2.pminu.b" || // Added in 3.9
Name == "sse41.pminuw" || // Added in 3.9
Name == "sse41.pminud" || // Added in 3.9
Name == "avx512.mask.pshuf.b.128" || // Added in 4.0
Name == "avx512.mask.pshuf.b.256" || // Added in 4.0
Name.startswith("avx2.pmax") || // Added in 3.9
Name.startswith("avx2.pmin") || // Added in 3.9
Name.startswith("avx512.mask.pmax") || // Added in 4.0
Name.startswith("avx512.mask.pmin") || // Added in 4.0
Name.startswith("avx2.vbroadcast") || // Added in 3.8
Name.startswith("avx2.pbroadcast") || // Added in 3.8
Name.startswith("avx.vpermil.") || // Added in 3.1
Name.startswith("sse2.pshuf") || // Added in 3.9
Name.startswith("avx512.pbroadcast") || // Added in 3.9
Name.startswith("avx512.mask.broadcast.s") || // Added in 3.9
Name.startswith("avx512.mask.movddup") || // Added in 3.9
Name.startswith("avx512.mask.movshdup") || // Added in 3.9
Name.startswith("avx512.mask.movsldup") || // Added in 3.9
Name.startswith("avx512.mask.pshuf.d.") || // Added in 3.9
Name.startswith("avx512.mask.pshufl.w.") || // Added in 3.9
Name.startswith("avx512.mask.pshufh.w.") || // Added in 3.9
Name.startswith("avx512.mask.shuf.p") || // Added in 4.0
Name.startswith("avx512.mask.vpermil.p") || // Added in 3.9
Name.startswith("avx512.mask.perm.df.") || // Added in 3.9
Name.startswith("avx512.mask.perm.di.") || // Added in 3.9
Name.startswith("avx512.mask.punpckl") || // Added in 3.9
Name.startswith("avx512.mask.punpckh") || // Added in 3.9
Name.startswith("avx512.mask.unpckl.") || // Added in 3.9
Name.startswith("avx512.mask.unpckh.") || // Added in 3.9
Name.startswith("avx512.mask.pand.") || // Added in 3.9
Name.startswith("avx512.mask.pandn.") || // Added in 3.9
Name.startswith("avx512.mask.por.") || // Added in 3.9
Name.startswith("avx512.mask.pxor.") || // Added in 3.9
Name.startswith("avx512.mask.and.") || // Added in 3.9
Name.startswith("avx512.mask.andn.") || // Added in 3.9
Name.startswith("avx512.mask.or.") || // Added in 3.9
Name.startswith("avx512.mask.xor.") || // Added in 3.9
Name.startswith("avx512.mask.padd.") || // Added in 4.0
Name.startswith("avx512.mask.psub.") || // Added in 4.0
Name.startswith("avx512.mask.pmull.") || // Added in 4.0
[X86][AVX512] Autoupgrade lossless i32/u32 to f64 conversion intrinsics with generic IR Both the (V)CVTDQ2PD (i32 to f64) and (V)CVTUDQ2PD (u32 to f64) conversion instructions are lossless and can be safely represented as generic SINT_TO_FP/UINT_TO_FP calls instead of x86 intrinsics without affecting final codegen. LLVM counterpart to D26686 Differential Revision: https://reviews.llvm.org/D26736 llvm-svn: 287108
2016-11-16 22:48:32 +08:00
Name.startswith("avx512.mask.cvtdq2pd.") || // Added in 4.0
Name.startswith("avx512.mask.cvtudq2pd.") || // Added in 4.0
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
Name == "avx512.mask.add.pd.128" || // Added in 4.0
Name == "avx512.mask.add.pd.256" || // Added in 4.0
Name == "avx512.mask.add.ps.128" || // Added in 4.0
Name == "avx512.mask.add.ps.256" || // Added in 4.0
Name == "avx512.mask.div.pd.128" || // Added in 4.0
Name == "avx512.mask.div.pd.256" || // Added in 4.0
Name == "avx512.mask.div.ps.128" || // Added in 4.0
Name == "avx512.mask.div.ps.256" || // Added in 4.0
Name == "avx512.mask.mul.pd.128" || // Added in 4.0
Name == "avx512.mask.mul.pd.256" || // Added in 4.0
Name == "avx512.mask.mul.ps.128" || // Added in 4.0
Name == "avx512.mask.mul.ps.256" || // Added in 4.0
Name == "avx512.mask.sub.pd.128" || // Added in 4.0
Name == "avx512.mask.sub.pd.256" || // Added in 4.0
Name == "avx512.mask.sub.ps.128" || // Added in 4.0
Name == "avx512.mask.sub.ps.256" || // Added in 4.0
Name.startswith("avx512.mask.psll.d") || // Added in 4.0
Name.startswith("avx512.mask.psll.q") || // Added in 4.0
Name.startswith("avx512.mask.psll.w") || // Added in 4.0
Name.startswith("avx512.mask.psra.d") || // Added in 4.0
Name.startswith("avx512.mask.psra.q") || // Added in 4.0
Name.startswith("avx512.mask.psra.w") || // Added in 4.0
Name.startswith("avx512.mask.psrl.d") || // Added in 4.0
Name.startswith("avx512.mask.psrl.q") || // Added in 4.0
Name.startswith("avx512.mask.psrl.w") || // Added in 4.0
Name.startswith("avx512.mask.pslli") || // Added in 4.0
Name.startswith("avx512.mask.psrai") || // Added in 4.0
Name.startswith("avx512.mask.psrli") || // Added in 4.0
[AVX-512] Replace masked 16-bit element variable shift intrinsics with new unmasked versions and selects. The same thing was done to 32-bit and 64-bit element sizes previously. This will allow us to support these shuffls in InstCombineCalls along with the other variable shift intrinsics. llvm-svn: 287312
2016-11-18 13:04:44 +08:00
Name.startswith("avx512.mask.psllv") || // Added in 4.0
Name.startswith("avx512.mask.psrav") || // Added in 4.0
Name.startswith("avx512.mask.psrlv") || // Added in 4.0
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
Name.startswith("sse41.pmovsx") || // Added in 3.8
Name.startswith("sse41.pmovzx") || // Added in 3.9
Name.startswith("avx2.pmovsx") || // Added in 3.9
Name.startswith("avx2.pmovzx") || // Added in 3.9
Name.startswith("avx512.mask.pmovsx") || // Added in 4.0
Name.startswith("avx512.mask.pmovzx") || // Added in 4.0
Name == "sse2.cvtdq2pd" || // Added in 3.9
Name == "sse2.cvtps2pd" || // Added in 3.9
Name == "avx.cvtdq2.pd.256" || // Added in 3.9
Name == "avx.cvt.ps2.pd.256" || // Added in 3.9
Name.startswith("avx.vinsertf128.") || // Added in 3.7
Name == "avx2.vinserti128" || // Added in 3.7
Name.startswith("avx.vextractf128.") || // Added in 3.7
Name == "avx2.vextracti128" || // Added in 3.7
Name.startswith("sse4a.movnt.") || // Added in 3.9
Name.startswith("avx.movnt.") || // Added in 3.2
Name.startswith("avx512.storent.") || // Added in 3.9
Name == "sse2.storel.dq" || // Added in 3.9
Name.startswith("sse.storeu.") || // Added in 3.9
Name.startswith("sse2.storeu.") || // Added in 3.9
Name.startswith("avx.storeu.") || // Added in 3.9
Name.startswith("avx512.mask.storeu.") || // Added in 3.9
Name.startswith("avx512.mask.store.p") || // Added in 3.9
Name.startswith("avx512.mask.store.b.") || // Added in 3.9
Name.startswith("avx512.mask.store.w.") || // Added in 3.9
Name.startswith("avx512.mask.store.d.") || // Added in 3.9
Name.startswith("avx512.mask.store.q.") || // Added in 3.9
Name.startswith("avx512.mask.loadu.") || // Added in 3.9
Name.startswith("avx512.mask.load.") || // Added in 3.9
Name == "sse42.crc32.64.8" || // Added in 3.4
Name.startswith("avx.vbroadcast.s") || // Added in 3.5
Name.startswith("avx512.mask.palignr.") || // Added in 3.9
[AVX-512] Remove intrinsics for valignd/q and autoupgrade them to native shuffles. llvm-svn: 287744
2016-11-23 14:54:55 +08:00
Name.startswith("avx512.mask.valign.") || // Added in 4.0
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
Name.startswith("sse2.psll.dq") || // Added in 3.7
Name.startswith("sse2.psrl.dq") || // Added in 3.7
Name.startswith("avx2.psll.dq") || // Added in 3.7
Name.startswith("avx2.psrl.dq") || // Added in 3.7
Name.startswith("avx512.psll.dq") || // Added in 3.9
Name.startswith("avx512.psrl.dq") || // Added in 3.9
Name == "sse41.pblendw" || // Added in 3.7
Name.startswith("sse41.blendp") || // Added in 3.7
Name.startswith("avx.blend.p") || // Added in 3.7
Name == "avx2.pblendw" || // Added in 3.7
Name.startswith("avx2.pblendd.") || // Added in 3.7
Name.startswith("avx.vbroadcastf128") || // Added in 4.0
Name == "avx2.vbroadcasti128" || // Added in 3.7
Name == "xop.vpcmov" || // Added in 3.8
[X86][AVX512] Removing llvm x86 intrinsics for _mm_mask_move_{ss|sd} intrinsics. Differential Revision: https://reviews.llvm.org/D26128 llvm-svn: 287087
2016-11-16 17:00:28 +08:00
Name.startswith("avx512.mask.move.s") || // Added in 4.0
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
(Name.startswith("xop.vpcom") && // Added in 3.2
F->arg_size() == 2))) {
[C++11] More 'nullptr' conversion or in some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 205831
2014-04-09 14:08:46 +08:00
NewFn = nullptr;
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
return true;
}
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
// SSE4.1 ptest functions may have an old signature.
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
if (IsX86 && Name.startswith("sse41.ptest")) { // Added in 3.2
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
if (Name.substr(11) == "c")
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
return UpgradeSSE41Function(F, Intrinsic::x86_sse41_ptestc, NewFn);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
if (Name.substr(11) == "z")
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
return UpgradeSSE41Function(F, Intrinsic::x86_sse41_ptestz, NewFn);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
if (Name.substr(11) == "nzc")
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
return UpgradeSSE41Function(F, Intrinsic::x86_sse41_ptestnzc, NewFn);
}
fix typo; NFC llvm-svn: 230876
2015-03-01 06:25:06 +08:00
// Several blend and other instructions with masks used the wrong number of
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
// bits.
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
if (IsX86 && Name == "sse41.insertps") // Added in 3.6
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
return UpgradeX86IntrinsicsWith8BitMask(F, Intrinsic::x86_sse41_insertps,
NewFn);
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
if (IsX86 && Name == "sse41.dppd") // Added in 3.6
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
return UpgradeX86IntrinsicsWith8BitMask(F, Intrinsic::x86_sse41_dppd,
NewFn);
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
if (IsX86 && Name == "sse41.dpps") // Added in 3.6
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
return UpgradeX86IntrinsicsWith8BitMask(F, Intrinsic::x86_sse41_dpps,
NewFn);
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
if (IsX86 && Name == "sse41.mpsadbw") // Added in 3.6
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
return UpgradeX86IntrinsicsWith8BitMask(F, Intrinsic::x86_sse41_mpsadbw,
NewFn);
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
if (IsX86 && Name == "avx.dp.ps.256") // Added in 3.6
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
return UpgradeX86IntrinsicsWith8BitMask(F, Intrinsic::x86_avx_dp_ps_256,
NewFn);
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
if (IsX86 && Name == "avx2.mpsadbw") // Added in 3.6
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
return UpgradeX86IntrinsicsWith8BitMask(F, Intrinsic::x86_avx2_mpsadbw,
NewFn);
[X86] Use i8 immediate for comparison type on AVX512 packed integer instructions. This matches floating point equivalents. Includes autoupgrade support to convert old code. llvm-svn: 227063
2015-01-26 07:26:02 +08:00
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
// frcz.ss/sd may need to have an argument dropped. Added in 3.2
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
if (IsX86 && Name.startswith("xop.vfrcz.ss") && F->arg_size() == 2) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Fix intrinsics for XOP frczss/sd instructions. These instructions only take one source register and zero the upper bits of the destination rather than preserving them. llvm-svn: 158396
2012-06-13 15:18:53 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(),
Intrinsic::x86_xop_vfrcz_ss);
return true;
}
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
if (IsX86 && Name.startswith("xop.vfrcz.sd") && F->arg_size() == 2) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
Fix intrinsics for XOP frczss/sd instructions. These instructions only take one source register and zero the upper bits of the destination rather than preserving them. llvm-svn: 158396
2012-06-13 15:18:53 +08:00
NewFn = Intrinsic::getDeclaration(F->getParent(),
Intrinsic::x86_xop_vfrcz_sd);
return true;
}
[X86][XOP] Support for VPERMIL2PD/VPERMIL2PS 2-input shuffle instructions This patch begins adding support for lowering to the XOP VPERMIL2PD/VPERMIL2PS shuffle instructions - adding the X86ISD::VPERMIL2 opcode and cleaning up the usage. The internal llvm intrinsics were assuming the shuffle mask operand was the same type as the float/double input operands (I guess to simplify the intrinsic definitions in X86InstrXOP.td to a single value type). These needed changing to integer types (matching the clang builtin and the AMD intrinsics definitions), an auto upgrade path is added to convert old calls. Mask decoding/target shuffle support will be added in future patches. Differential Revision: http://reviews.llvm.org/D20049 llvm-svn: 271633
2016-06-03 16:06:03 +08:00
// Upgrade any XOP PERMIL2 index operand still using a float/double vector.
[X86] Add LLVM version number for each intrinsic handled by auto upgrade for age tracking. One day we'd like to remove some of this autoupgrade support and it will be easier if we know how long some of it has been around. Differential Revision: https://reviews.llvm.org/D26321 llvm-svn: 286933
2016-11-15 13:04:51 +08:00
if (IsX86 && Name.startswith("xop.vpermil2")) { // Added in 3.9
[X86][XOP] Support for VPERMIL2PD/VPERMIL2PS 2-input shuffle instructions This patch begins adding support for lowering to the XOP VPERMIL2PD/VPERMIL2PS shuffle instructions - adding the X86ISD::VPERMIL2 opcode and cleaning up the usage. The internal llvm intrinsics were assuming the shuffle mask operand was the same type as the float/double input operands (I guess to simplify the intrinsic definitions in X86InstrXOP.td to a single value type). These needed changing to integer types (matching the clang builtin and the AMD intrinsics definitions), an auto upgrade path is added to convert old calls. Mask decoding/target shuffle support will be added in future patches. Differential Revision: http://reviews.llvm.org/D20049 llvm-svn: 271633
2016-06-03 16:06:03 +08:00
auto Params = F->getFunctionType()->params();
auto Idx = Params[2];
if (Idx->getScalarType()->isFloatingPointTy()) {
Don't drop the llvm. prefix when renaming. If the llvm. prefix is dropped other parts of llvm don't see this as an intrinsic. This means that the number of regular symbols depends on the context the module is loaded into, which causes LTO to abort. Fixes PR30509. llvm-svn: 283117
2016-10-03 23:51:42 +08:00
rename(F);
[X86][XOP] Support for VPERMIL2PD/VPERMIL2PS 2-input shuffle instructions This patch begins adding support for lowering to the XOP VPERMIL2PD/VPERMIL2PS shuffle instructions - adding the X86ISD::VPERMIL2 opcode and cleaning up the usage. The internal llvm intrinsics were assuming the shuffle mask operand was the same type as the float/double input operands (I guess to simplify the intrinsic definitions in X86InstrXOP.td to a single value type). These needed changing to integer types (matching the clang builtin and the AMD intrinsics definitions), an auto upgrade path is added to convert old calls. Mask decoding/target shuffle support will be added in future patches. Differential Revision: http://reviews.llvm.org/D20049 llvm-svn: 271633
2016-06-03 16:06:03 +08:00
unsigned IdxSize = Idx->getPrimitiveSizeInBits();
unsigned EltSize = Idx->getScalarSizeInBits();
Intrinsic::ID Permil2ID;
if (EltSize == 64 && IdxSize == 128)
Permil2ID = Intrinsic::x86_xop_vpermil2pd;
else if (EltSize == 32 && IdxSize == 128)
Permil2ID = Intrinsic::x86_xop_vpermil2ps;
else if (EltSize == 64 && IdxSize == 256)
Permil2ID = Intrinsic::x86_xop_vpermil2pd_256;
else
Permil2ID = Intrinsic::x86_xop_vpermil2ps_256;
NewFn = Intrinsic::getDeclaration(F->getParent(), Permil2ID);
return true;
}
}
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
break;
}
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
}
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
// This may not belong here. This function is effectively being overloaded
// to both detect an intrinsic which needs upgrading, and to provide the
// upgraded form of the intrinsic. We should perhaps have two separate
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
// functions for this.
Bring back int_x86_sse2_movl_dq intrinsic for backward compatibility. Make sure it's auto-upgraded to a shufflevector instruction. llvm-svn: 45131
2007-12-18 06:33:23 +08:00
return false;
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
}
Bring back int_x86_sse2_movl_dq intrinsic for backward compatibility. Make sure it's auto-upgraded to a shufflevector instruction. llvm-svn: 45131
2007-12-18 06:33:23 +08:00
bool llvm::UpgradeIntrinsicFunction(Function *F, Function *&NewFn) {
[C++11] More 'nullptr' conversion or in some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 205831
2014-04-09 14:08:46 +08:00
NewFn = nullptr;
Bring back int_x86_sse2_movl_dq intrinsic for backward compatibility. Make sure it's auto-upgraded to a shufflevector instruction. llvm-svn: 45131
2007-12-18 06:33:23 +08:00
bool Upgraded = UpgradeIntrinsicFunction1(F, NewFn);
Remove always-true comparison, NFC. Summary: Looking at r241279, I noticed that UpgradedIntrinsics only gets written to in the following code: if (UpgradeIntrinsicFunction(&F, NewFn)) UpgradedIntrinsics[&F] = NewFn; Looking through UpgradeIntrinsicFunction, we always return false OR NewFn will be set to a different function from our source. This patch pulls the F != NewFn into UpgradeIntrinsicFunction as an assert, and removes the check from callers of UpgradeIntrinsicFunction. Reviewers: rafael, chandlerc Subscribers: llvm-commits-list Differential Revision: http://reviews.llvm.org/D10915 llvm-svn: 241369
2015-07-04 04:12:01 +08:00
assert(F != NewFn && "Intrinsic function upgraded to the same function");
Rather than having special rules like "intrinsics cannot throw exceptions", just mark intrinsics with the nounwind attribute. Likewise, mark intrinsics as readnone/readonly and get rid of special aliasing logic (which didn't use anything more than this anyway). llvm-svn: 44544
2007-12-04 04:06:50 +08:00
// Upgrade intrinsic attributes. This does not change the function.
Bring back int_x86_sse2_movl_dq intrinsic for backward compatibility. Make sure it's auto-upgraded to a shufflevector instruction. llvm-svn: 45131
2007-12-18 06:33:23 +08:00
if (NewFn)
F = NewFn;
Change Function::getIntrinsicID() to return an Intrinsic::ID. NFC. Now that Intrinsic::ID is a typed enum, we can forward declare it and so return it from this method. This updates all users which were either using an unsigned to store it, or had a now unnecessary cast. llvm-svn: 237810
2015-05-21 01:16:39 +08:00
if (Intrinsic::ID id = F->getIntrinsicID())
F->setAttributes(Intrinsic::getAttributes(F->getContext(), id));
Rather than having special rules like "intrinsics cannot throw exceptions", just mark intrinsics with the nounwind attribute. Likewise, mark intrinsics as readnone/readonly and get rid of special aliasing logic (which didn't use anything more than this anyway). llvm-svn: 44544
2007-12-04 04:06:50 +08:00
return Upgraded;
}
Auto-upgrade the magic ".llvm.eh.catch.all.value" global to "llvm.eh.catch.all.value". Only the name needs to be changed. llvm-svn: 113600
2010-09-11 02:51:56 +08:00
bool llvm::UpgradeGlobalVariable(GlobalVariable *GV) {
rip out a ton of intrinsic modernization logic from AutoUpgrade.cpp, which is for pre-2.9 bitcode files. We keep x86 unaligned loads, movnt, crc32, and the target indep prefetch change. As usual, updating the testsuite is a PITA. llvm-svn: 133337
2011-06-18 14:05:24 +08:00
// Nothing to do yet.
Auto-upgrade the magic ".llvm.eh.catch.all.value" global to "llvm.eh.catch.all.value". Only the name needs to be changed. llvm-svn: 113600
2010-09-11 02:51:56 +08:00
return false;
}
[X86][AVX512] Dropped avx512 VPSLLDQ/VPSRLDQ intrinsics Auto-upgrade to generic shuffles like sse/avx2 implementations now that we can lower to VPSLLDQ/VPSRLDQ llvm-svn: 272308
2016-06-10 05:09:03 +08:00
// Handles upgrading SSE2/AVX2/AVX512BW PSLLDQ intrinsics by converting them
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
// to byte shuffles.
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
static Value *UpgradeX86PSLLDQIntrinsics(IRBuilder<> &Builder,
[X86] Simplify some of the autoupgrade code. NFC llvm-svn: 271174
2016-05-29 14:37:33 +08:00
Value *Op, unsigned Shift) {
Type *ResultTy = Op->getType();
unsigned NumElts = ResultTy->getVectorNumElements() * 8;
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
// Bitcast from a 64-bit element type to a byte element type.
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
Type *VecTy = VectorType::get(Builder.getInt8Ty(), NumElts);
[X86] Simplify some of the autoupgrade code. NFC llvm-svn: 271174
2016-05-29 14:37:33 +08:00
Op = Builder.CreateBitCast(Op, VecTy, "cast");
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
// We'll be shuffling in zeroes.
[X86] Simplify some of the autoupgrade code. NFC llvm-svn: 271174
2016-05-29 14:37:33 +08:00
Value *Res = Constant::getNullValue(VecTy);
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
// If shift is less than 16, emit a shuffle to move the bytes. Otherwise,
// we'll just return the zero vector.
if (Shift < 16) {
[IR] Require ArrayRef of 'uint32_t' instead of 'int' for the mask argument for one of the signatures of CreateShuffleVector. This better emphasises that you can't use it for the -1 as undef behavior. llvm-svn: 272491
2016-06-12 08:41:19 +08:00
uint32_t Idxs[64];
[X86][AVX512] Dropped avx512 VPSLLDQ/VPSRLDQ intrinsics Auto-upgrade to generic shuffles like sse/avx2 implementations now that we can lower to VPSLLDQ/VPSRLDQ llvm-svn: 272308
2016-06-10 05:09:03 +08:00
// 256/512-bit version is split into 2/4 16-byte lanes.
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
for (unsigned l = 0; l != NumElts; l += 16)
for (unsigned i = 0; i != 16; ++i) {
unsigned Idx = NumElts + i - Shift;
if (Idx < NumElts)
Idx -= NumElts - 16; // end of lane, switch operand.
[X86] Simplify some of the autoupgrade code. NFC llvm-svn: 271174
2016-05-29 14:37:33 +08:00
Idxs[l + i] = Idx + l;
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
}
[X86] Simplify some of the autoupgrade code. NFC llvm-svn: 271174
2016-05-29 14:37:33 +08:00
Res = Builder.CreateShuffleVector(Res, Op, makeArrayRef(Idxs, NumElts));
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
}
// Bitcast back to a 64-bit element type.
[X86] Simplify some of the autoupgrade code. NFC llvm-svn: 271174
2016-05-29 14:37:33 +08:00
return Builder.CreateBitCast(Res, ResultTy, "cast");
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
}
[X86] Refactor some of the X86 autoupgrade code to split mask vector and select generation into routines that can be reused for future intrinsic upgrades. NFC llvm-svn: 272526
2016-06-13 10:36:42 +08:00
// Handles upgrading SSE2/AVX2/AVX512BW PSRLDQ intrinsics by converting them
// to byte shuffles.
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
static Value *UpgradeX86PSRLDQIntrinsics(IRBuilder<> &Builder, Value *Op,
[X86] Refactor some of the X86 autoupgrade code to split mask vector and select generation into routines that can be reused for future intrinsic upgrades. NFC llvm-svn: 272526
2016-06-13 10:36:42 +08:00
unsigned Shift) {
Type *ResultTy = Op->getType();
unsigned NumElts = ResultTy->getVectorNumElements() * 8;
// Bitcast from a 64-bit element type to a byte element type.
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
Type *VecTy = VectorType::get(Builder.getInt8Ty(), NumElts);
[X86] Refactor some of the X86 autoupgrade code to split mask vector and select generation into routines that can be reused for future intrinsic upgrades. NFC llvm-svn: 272526
2016-06-13 10:36:42 +08:00
Op = Builder.CreateBitCast(Op, VecTy, "cast");
// We'll be shuffling in zeroes.
Value *Res = Constant::getNullValue(VecTy);
// If shift is less than 16, emit a shuffle to move the bytes. Otherwise,
// we'll just return the zero vector.
if (Shift < 16) {
uint32_t Idxs[64];
// 256/512-bit version is split into 2/4 16-byte lanes.
for (unsigned l = 0; l != NumElts; l += 16)
for (unsigned i = 0; i != 16; ++i) {
unsigned Idx = i + Shift;
if (Idx >= 16)
Idx += NumElts - 16; // end of lane, switch operand.
Idxs[l + i] = Idx + l;
}
Res = Builder.CreateShuffleVector(Op, Res, makeArrayRef(Idxs, NumElts));
}
// Bitcast back to a 64-bit element type.
return Builder.CreateBitCast(Res, ResultTy, "cast");
}
static Value *getX86MaskVec(IRBuilder<> &Builder, Value *Mask,
unsigned NumElts) {
llvm::VectorType *MaskTy = llvm::VectorType::get(Builder.getInt1Ty(),
cast<IntegerType>(Mask->getType())->getBitWidth());
Mask = Builder.CreateBitCast(Mask, MaskTy);
// If we have less than 8 elements, then the starting mask was an i8 and
// we need to extract down to the right number of elements.
if (NumElts < 8) {
uint32_t Indices[4];
for (unsigned i = 0; i != NumElts; ++i)
Indices[i] = i;
Mask = Builder.CreateShuffleVector(Mask, Mask,
makeArrayRef(Indices, NumElts),
"extract");
}
return Mask;
}
static Value *EmitX86Select(IRBuilder<> &Builder, Value *Mask,
Value *Op0, Value *Op1) {
// If the mask is all ones just emit the align operation.
if (const auto *C = dyn_cast<Constant>(Mask))
if (C->isAllOnesValue())
return Op0;
Mask = getX86MaskVec(Builder, Mask, Op0->getType()->getVectorNumElements());
return Builder.CreateSelect(Mask, Op0, Op1);
}
[AVX-512] Remove intrinsics for valignd/q and autoupgrade them to native shuffles. llvm-svn: 287744
2016-11-23 14:54:55 +08:00
// Handle autoupgrade for masked PALIGNR and VALIGND/Q intrinsics.
// PALIGNR handles large immediates by shifting while VALIGN masks the immediate
// so we need to handle both cases. VALIGN also doesn't have 128-bit lanes.
static Value *UpgradeX86ALIGNIntrinsics(IRBuilder<> &Builder, Value *Op0,
Value *Op1, Value *Shift,
Value *Passthru, Value *Mask,
bool IsVALIGN) {
[AVX512] Remove masked palignr intrinsics and auto-upgrade them to native IR of vector shuffle and select. llvm-svn: 271872
2016-06-06 14:12:54 +08:00
unsigned ShiftVal = cast<llvm::ConstantInt>(Shift)->getZExtValue();
unsigned NumElts = Op0->getType()->getVectorNumElements();
[AVX-512] Remove intrinsics for valignd/q and autoupgrade them to native shuffles. llvm-svn: 287744
2016-11-23 14:54:55 +08:00
assert((IsVALIGN || NumElts % 16 == 0) && "Illegal NumElts for PALIGNR!");
assert((!IsVALIGN || NumElts <= 16) && "NumElts too large for VALIGN!");
assert(isPowerOf2_32(NumElts) && "NumElts not a power of 2!");
// Mask the immediate for VALIGN.
if (IsVALIGN)
ShiftVal &= (NumElts - 1);
[AVX512] Remove masked palignr intrinsics and auto-upgrade them to native IR of vector shuffle and select. llvm-svn: 271872
2016-06-06 14:12:54 +08:00
// If palignr is shifting the pair of vectors more than the size of two
// lanes, emit zero.
if (ShiftVal >= 32)
return llvm::Constant::getNullValue(Op0->getType());
// If palignr is shifting the pair of input vectors more than one lane,
// but less than two lanes, convert to shifting in zeroes.
if (ShiftVal > 16) {
ShiftVal -= 16;
Op1 = Op0;
Op0 = llvm::Constant::getNullValue(Op0->getType());
}
[IR] Require ArrayRef of 'uint32_t' instead of 'int' for the mask argument for one of the signatures of CreateShuffleVector. This better emphasises that you can't use it for the -1 as undef behavior. llvm-svn: 272491
2016-06-12 08:41:19 +08:00
uint32_t Indices[64];
[AVX512] Remove masked palignr intrinsics and auto-upgrade them to native IR of vector shuffle and select. llvm-svn: 271872
2016-06-06 14:12:54 +08:00
// 256-bit palignr operates on 128-bit lanes so we need to handle that
[AVX-512] Remove intrinsics for valignd/q and autoupgrade them to native shuffles. llvm-svn: 287744
2016-11-23 14:54:55 +08:00
for (unsigned l = 0; l < NumElts; l += 16) {
[AVX512] Remove masked palignr intrinsics and auto-upgrade them to native IR of vector shuffle and select. llvm-svn: 271872
2016-06-06 14:12:54 +08:00
for (unsigned i = 0; i != 16; ++i) {
unsigned Idx = ShiftVal + i;
[AVX-512] Remove intrinsics for valignd/q and autoupgrade them to native shuffles. llvm-svn: 287744
2016-11-23 14:54:55 +08:00
if (!IsVALIGN && Idx >= 16) // Disable wrap for VALIGN.
[AVX512] Remove masked palignr intrinsics and auto-upgrade them to native IR of vector shuffle and select. llvm-svn: 271872
2016-06-06 14:12:54 +08:00
Idx += NumElts - 16; // End of lane, switch operand.
Indices[l + i] = Idx + l;
}
}
Value *Align = Builder.CreateShuffleVector(Op1, Op0,
makeArrayRef(Indices, NumElts),
"palignr");
[X86] Refactor some of the X86 autoupgrade code to split mask vector and select generation into routines that can be reused for future intrinsic upgrades. NFC llvm-svn: 272526
2016-06-13 10:36:42 +08:00
return EmitX86Select(Builder, Mask, Align, Passthru);
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
}
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
static Value *UpgradeMaskedStore(IRBuilder<> &Builder,
[AVX512] Remove masked store intrinsics. Clang now emits generic masked store intrinsics instead. The intrinsics will be autoupgraded to the same generic masked stores. llvm-svn: 271245
2016-05-31 09:50:02 +08:00
Value *Ptr, Value *Data, Value *Mask,
bool Aligned) {
// Cast the pointer to the right type.
Ptr = Builder.CreateBitCast(Ptr,
llvm::PointerType::getUnqual(Data->getType()));
unsigned Align =
Aligned ? cast<VectorType>(Data->getType())->getBitWidth() / 8 : 1;
// If the mask is all ones just emit a regular store.
if (const auto *C = dyn_cast<Constant>(Mask))
if (C->isAllOnesValue())
return Builder.CreateAlignedStore(Data, Ptr, Align);
// Convert the mask from an integer type to a vector of i1.
unsigned NumElts = Data->getType()->getVectorNumElements();
[X86] Refactor some of the X86 autoupgrade code to split mask vector and select generation into routines that can be reused for future intrinsic upgrades. NFC llvm-svn: 272526
2016-06-13 10:36:42 +08:00
Mask = getX86MaskVec(Builder, Mask, NumElts);
[AVX512] Remove masked store intrinsics. Clang now emits generic masked store intrinsics instead. The intrinsics will be autoupgraded to the same generic masked stores. llvm-svn: 271245
2016-05-31 09:50:02 +08:00
return Builder.CreateMaskedStore(Data, Ptr, Align, Mask);
}
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
static Value *UpgradeMaskedLoad(IRBuilder<> &Builder,
[AVX512] Remove masked load intrinsics. Clang now emits generic masked load intrinsics instead. The intrinsics will be autoupgraded to the same generic masked loads. llvm-svn: 271478
2016-06-02 12:19:36 +08:00
Value *Ptr, Value *Passthru, Value *Mask,
bool Aligned) {
// Cast the pointer to the right type.
Ptr = Builder.CreateBitCast(Ptr,
llvm::PointerType::getUnqual(Passthru->getType()));
unsigned Align =
Aligned ? cast<VectorType>(Passthru->getType())->getBitWidth() / 8 : 1;
// If the mask is all ones just emit a regular store.
if (const auto *C = dyn_cast<Constant>(Mask))
if (C->isAllOnesValue())
return Builder.CreateAlignedLoad(Ptr, Align);
// Convert the mask from an integer type to a vector of i1.
unsigned NumElts = Passthru->getType()->getVectorNumElements();
[X86] Refactor some of the X86 autoupgrade code to split mask vector and select generation into routines that can be reused for future intrinsic upgrades. NFC llvm-svn: 272526
2016-06-13 10:36:42 +08:00
Mask = getX86MaskVec(Builder, Mask, NumElts);
[AVX512] Remove masked load intrinsics. Clang now emits generic masked load intrinsics instead. The intrinsics will be autoupgraded to the same generic masked loads. llvm-svn: 271478
2016-06-02 12:19:36 +08:00
return Builder.CreateMaskedLoad(Ptr, Align, Mask, Passthru);
}
[x86] autoupgrade and remove SSE2/SSE41 integer min/max intrinsics Follow-up to: http://reviews.llvm.org/rL272806 http://reviews.llvm.org/rL272807 llvm-svn: 272907
2016-06-16 23:48:30 +08:00
static Value *upgradeIntMinMax(IRBuilder<> &Builder, CallInst &CI,
ICmpInst::Predicate Pred) {
Value *Op0 = CI.getArgOperand(0);
Value *Op1 = CI.getArgOperand(1);
Value *Cmp = Builder.CreateICmp(Pred, Op0, Op1);
[AVX-512] Remove masked pmin/pmax intrinsics and autoupgrade to native IR. Clang patch to replace 512-bit vector and 64-bit element versions with native IR will follow. llvm-svn: 284955
2016-10-24 12:04:16 +08:00
Value *Res = Builder.CreateSelect(Cmp, Op0, Op1);
if (CI.getNumArgOperands() == 4)
Res = EmitX86Select(Builder, CI.getArgOperand(3), Res, CI.getArgOperand(2));
return Res;
[x86] autoupgrade and remove SSE2/SSE41 integer min/max intrinsics Follow-up to: http://reviews.llvm.org/rL272806 http://reviews.llvm.org/rL272807 llvm-svn: 272907
2016-06-16 23:48:30 +08:00
}
[AVX512] Remove the masked vpcmpeq/vcmpgt intrinsics and autoupgrade them to native icmps. llvm-svn: 273240
2016-06-21 11:53:24 +08:00
static Value *upgradeMaskedCompare(IRBuilder<> &Builder, CallInst &CI,
ICmpInst::Predicate Pred) {
Value *Op0 = CI.getArgOperand(0);
unsigned NumElts = Op0->getType()->getVectorNumElements();
Value *Cmp = Builder.CreateICmp(Pred, Op0, CI.getArgOperand(1));
Value *Mask = CI.getArgOperand(2);
const auto *C = dyn_cast<Constant>(Mask);
if (!C || !C->isAllOnesValue())
Cmp = Builder.CreateAnd(Cmp, getX86MaskVec(Builder, Mask, NumElts));
if (NumElts < 8) {
uint32_t Indices[8];
for (unsigned i = 0; i != NumElts; ++i)
Indices[i] = i;
for (unsigned i = NumElts; i != 8; ++i)
[AVX512] Zero extend the result of vpcmpeq/vpcmpgt and similar intrinsics in the autoupgrade code. This currently results in worse codegen but is needed for correctness. llvm-svn: 274736
2016-07-07 14:11:07 +08:00
Indices[i] = NumElts + i % NumElts;
Cmp = Builder.CreateShuffleVector(Cmp,
Constant::getNullValue(Cmp->getType()),
[AVX512] Remove the masked vpcmpeq/vcmpgt intrinsics and autoupgrade them to native icmps. llvm-svn: 273240
2016-06-21 11:53:24 +08:00
Indices);
}
return Builder.CreateBitCast(Cmp, IntegerType::get(CI.getContext(),
std::max(NumElts, 8U)));
}
[AVX-512] Remove intrinsics for 128/256-bit masked shift by single element in xmm. Instead upgrade them to a select and the older SSE/AVX2 intrinsic. llvm-svn: 286070
2016-11-07 00:29:08 +08:00
// Replace a masked intrinsic with an older unmasked intrinsic.
static Value *UpgradeX86MaskedShift(IRBuilder<> &Builder, CallInst &CI,
Intrinsic::ID IID) {
Function *F = CI.getCalledFunction();
Function *Intrin = Intrinsic::getDeclaration(F->getParent(), IID);
Value *Rep = Builder.CreateCall(Intrin,
{ CI.getArgOperand(0), CI.getArgOperand(1) });
return EmitX86Select(Builder, CI.getArgOperand(3), Rep, CI.getArgOperand(2));
}
[X86][AVX512] Removing llvm x86 intrinsics for _mm_mask_move_{ss|sd} intrinsics. Differential Revision: https://reviews.llvm.org/D26128 llvm-svn: 287087
2016-11-16 17:00:28 +08:00
static Value* upgradeMaskedMove(IRBuilder<> &Builder, CallInst &CI) {
Value* A = CI.getArgOperand(0);
Value* B = CI.getArgOperand(1);
Value* Src = CI.getArgOperand(2);
Value* Mask = CI.getArgOperand(3);
Value* AndNode = Builder.CreateAnd(Mask, APInt(8, 1));
Value* Cmp = Builder.CreateIsNotNull(AndNode);
Value* Extract1 = Builder.CreateExtractElement(B, (uint64_t)0);
Value* Extract2 = Builder.CreateExtractElement(Src, (uint64_t)0);
Value* Select = Builder.CreateSelect(Cmp, Extract1, Extract2);
return Builder.CreateInsertElement(A, Select, (uint64_t)0);
}
[AVX-512] Remove intrinsics for 128/256-bit masked shift by single element in xmm. Instead upgrade them to a select and the older SSE/AVX2 intrinsic. llvm-svn: 286070
2016-11-07 00:29:08 +08:00
fix comments; NFC llvm-svn: 272848
2016-06-16 06:01:28 +08:00
/// Upgrade a call to an old intrinsic. All argument and return casting must be
/// provided to seamlessly integrate with existing context.
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
void llvm::UpgradeIntrinsicCall(CallInst *CI, Function *NewFn) {
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
Function *F = CI->getCalledFunction();
Fix unused value warning for value used only in assert. llvm-svn: 146440
2011-12-13 06:59:34 +08:00
LLVMContext &C = CI->getContext();
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
IRBuilder<> Builder(C);
IR: Remove implicit iterator conversions from lib/IR, NFC Stop converting implicitly between iterators and pointers/references in lib/IR. For convenience, I've added a `getIterator()` accessor to `ilist_node` so that callers don't need to know how to spell the iterator class (i.e., they can use `X.getIterator()` instead of `Function::iterator(X)`). I'll eventually disallow these implicit conversions entirely, but there's a lot of code, so it doesn't make sense to do it all in one patch. One library or so at a time. Why? To root out cases of `getNextNode()` and `getPrevNode()` being used in iterator logic. The design of `ilist` makes that invalid when the current node could be at the back of the list, but it happens to "work" right now because of a bug where those functions never return `nullptr` if you're using a half-node sentinel. Before I can fix the function, I have to remove uses of it that rely on it misbehaving. (Maybe the function should just be deleted anyway? But I don't want deleting it -- potentially a huge project -- to block fixing ilist/iplist.) llvm-svn: 249782
2015-10-09 07:49:46 +08:00
Builder.SetInsertPoint(CI->getParent(), CI->getIterator());
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
assert(F && "Intrinsic call is not direct?");
if (!NewFn) {
// Get the Function's name.
StringRef Name = F->getName();
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
assert(Name.startswith("llvm.") && "Intrinsic doesn't start with 'llvm.'");
Name = Name.substr(5);
bool IsX86 = Name.startswith("x86.");
if (IsX86)
Name = Name.substr(4);
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
Value *Rep;
fix comments; NFC llvm-svn: 272848
2016-06-16 06:01:28 +08:00
// Upgrade packed integer vector compare intrinsics to compare instructions.
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
if (IsX86 && (Name.startswith("sse2.pcmpeq.") ||
Name.startswith("avx2.pcmpeq."))) {
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
Rep = Builder.CreateICmpEQ(CI->getArgOperand(0), CI->getArgOperand(1),
"pcmpeq");
Rep = Builder.CreateSExt(Rep, CI->getType(), "");
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("sse2.pcmpgt.") ||
Name.startswith("avx2.pcmpgt."))) {
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
Rep = Builder.CreateICmpSGT(CI->getArgOperand(0), CI->getArgOperand(1),
"pcmpgt");
Rep = Builder.CreateSExt(Rep, CI->getType(), "");
[X86] Remove the scalar intrinsics for fadd/fsub/fdiv/fmul Summary: These intrinsics have been unused for clang for a while. This patch removes them. We auto upgrade them to extractelements, a scalar operation and then an insertelement. This matches the sequence used by clangs intrinsic file. Reviewers: zvi, delena, RKSimon Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D26660 llvm-svn: 287083
2016-11-16 13:24:10 +08:00
} else if (IsX86 && (Name == "sse.add.ss" || Name == "sse2.add.sd")) {
Type *I32Ty = Type::getInt32Ty(C);
Value *Elt0 = Builder.CreateExtractElement(CI->getArgOperand(0),
ConstantInt::get(I32Ty, 0));
Value *Elt1 = Builder.CreateExtractElement(CI->getArgOperand(1),
ConstantInt::get(I32Ty, 0));
Rep = Builder.CreateInsertElement(CI->getArgOperand(0),
Builder.CreateFAdd(Elt0, Elt1),
ConstantInt::get(I32Ty, 0));
} else if (IsX86 && (Name == "sse.sub.ss" || Name == "sse2.sub.sd")) {
Type *I32Ty = Type::getInt32Ty(C);
Value *Elt0 = Builder.CreateExtractElement(CI->getArgOperand(0),
ConstantInt::get(I32Ty, 0));
Value *Elt1 = Builder.CreateExtractElement(CI->getArgOperand(1),
ConstantInt::get(I32Ty, 0));
Rep = Builder.CreateInsertElement(CI->getArgOperand(0),
Builder.CreateFSub(Elt0, Elt1),
ConstantInt::get(I32Ty, 0));
} else if (IsX86 && (Name == "sse.mul.ss" || Name == "sse2.mul.sd")) {
Type *I32Ty = Type::getInt32Ty(C);
Value *Elt0 = Builder.CreateExtractElement(CI->getArgOperand(0),
ConstantInt::get(I32Ty, 0));
Value *Elt1 = Builder.CreateExtractElement(CI->getArgOperand(1),
ConstantInt::get(I32Ty, 0));
Rep = Builder.CreateInsertElement(CI->getArgOperand(0),
Builder.CreateFMul(Elt0, Elt1),
ConstantInt::get(I32Ty, 0));
} else if (IsX86 && (Name == "sse.div.ss" || Name == "sse2.div.sd")) {
Type *I32Ty = Type::getInt32Ty(C);
Value *Elt0 = Builder.CreateExtractElement(CI->getArgOperand(0),
ConstantInt::get(I32Ty, 0));
Value *Elt1 = Builder.CreateExtractElement(CI->getArgOperand(1),
ConstantInt::get(I32Ty, 0));
Rep = Builder.CreateInsertElement(CI->getArgOperand(0),
Builder.CreateFDiv(Elt0, Elt1),
ConstantInt::get(I32Ty, 0));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.pcmpeq.")) {
[AVX512] Remove the masked vpcmpeq/vcmpgt intrinsics and autoupgrade them to native icmps. llvm-svn: 273240
2016-06-21 11:53:24 +08:00
Rep = upgradeMaskedCompare(Builder, *CI, ICmpInst::ICMP_EQ);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.pcmpgt.")) {
[AVX512] Remove the masked vpcmpeq/vcmpgt intrinsics and autoupgrade them to native icmps. llvm-svn: 273240
2016-06-21 11:53:24 +08:00
Rep = upgradeMaskedCompare(Builder, *CI, ICmpInst::ICMP_SGT);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse41.pmaxsb" ||
Name == "sse2.pmaxs.w" ||
Name == "sse41.pmaxsd" ||
[AVX-512] Remove masked pmin/pmax intrinsics and autoupgrade to native IR. Clang patch to replace 512-bit vector and 64-bit element versions with native IR will follow. llvm-svn: 284955
2016-10-24 12:04:16 +08:00
Name.startswith("avx2.pmaxs") ||
Name.startswith("avx512.mask.pmaxs"))) {
[x86] autoupgrade and remove SSE2/SSE41 integer min/max intrinsics Follow-up to: http://reviews.llvm.org/rL272806 http://reviews.llvm.org/rL272807 llvm-svn: 272907
2016-06-16 23:48:30 +08:00
Rep = upgradeIntMinMax(Builder, *CI, ICmpInst::ICMP_SGT);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.pmaxu.b" ||
Name == "sse41.pmaxuw" ||
Name == "sse41.pmaxud" ||
[AVX-512] Remove masked pmin/pmax intrinsics and autoupgrade to native IR. Clang patch to replace 512-bit vector and 64-bit element versions with native IR will follow. llvm-svn: 284955
2016-10-24 12:04:16 +08:00
Name.startswith("avx2.pmaxu") ||
Name.startswith("avx512.mask.pmaxu"))) {
[x86] autoupgrade and remove SSE2/SSE41 integer min/max intrinsics Follow-up to: http://reviews.llvm.org/rL272806 http://reviews.llvm.org/rL272807 llvm-svn: 272907
2016-06-16 23:48:30 +08:00
Rep = upgradeIntMinMax(Builder, *CI, ICmpInst::ICMP_UGT);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse41.pminsb" ||
Name == "sse2.pmins.w" ||
Name == "sse41.pminsd" ||
[AVX-512] Remove masked pmin/pmax intrinsics and autoupgrade to native IR. Clang patch to replace 512-bit vector and 64-bit element versions with native IR will follow. llvm-svn: 284955
2016-10-24 12:04:16 +08:00
Name.startswith("avx2.pmins") ||
Name.startswith("avx512.mask.pmins"))) {
[x86] autoupgrade and remove SSE2/SSE41 integer min/max intrinsics Follow-up to: http://reviews.llvm.org/rL272806 http://reviews.llvm.org/rL272807 llvm-svn: 272907
2016-06-16 23:48:30 +08:00
Rep = upgradeIntMinMax(Builder, *CI, ICmpInst::ICMP_SLT);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.pminu.b" ||
Name == "sse41.pminuw" ||
Name == "sse41.pminud" ||
[AVX-512] Remove masked pmin/pmax intrinsics and autoupgrade to native IR. Clang patch to replace 512-bit vector and 64-bit element versions with native IR will follow. llvm-svn: 284955
2016-10-24 12:04:16 +08:00
Name.startswith("avx2.pminu") ||
Name.startswith("avx512.mask.pminu"))) {
[x86] autoupgrade and remove SSE2/SSE41 integer min/max intrinsics Follow-up to: http://reviews.llvm.org/rL272806 http://reviews.llvm.org/rL272807 llvm-svn: 272907
2016-06-16 23:48:30 +08:00
Rep = upgradeIntMinMax(Builder, *CI, ICmpInst::ICMP_ULT);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.cvtdq2pd" ||
Name == "sse2.cvtps2pd" ||
Name == "avx.cvtdq2.pd.256" ||
[X86][AVX512] Autoupgrade lossless i32/u32 to f64 conversion intrinsics with generic IR Both the (V)CVTDQ2PD (i32 to f64) and (V)CVTUDQ2PD (u32 to f64) conversion instructions are lossless and can be safely represented as generic SINT_TO_FP/UINT_TO_FP calls instead of x86 intrinsics without affecting final codegen. LLVM counterpart to D26686 Differential Revision: https://reviews.llvm.org/D26736 llvm-svn: 287108
2016-11-16 22:48:32 +08:00
Name == "avx.cvt.ps2.pd.256" ||
Name.startswith("avx512.mask.cvtdq2pd.") ||
Name.startswith("avx512.mask.cvtudq2pd."))) {
[X86][SSE] Replace (V)CVTDQ2PD(Y) and (V)CVTPS2PD(Y) lossless conversion intrinsics with generic IR Followup to D20528 clang patch, this removes the (V)CVTDQ2PD(Y) and (V)CVTPS2PD(Y) llvm intrinsics and auto-upgrades to sitofp/fpext instead. Differential Revision: http://reviews.llvm.org/D20568 llvm-svn: 270678
2016-05-25 16:59:18 +08:00
// Lossless i32/float to double conversion.
// Extract the bottom elements if necessary and convert to double vector.
Value *Src = CI->getArgOperand(0);
VectorType *SrcTy = cast<VectorType>(Src->getType());
VectorType *DstTy = cast<VectorType>(CI->getType());
Rep = CI->getArgOperand(0);
unsigned NumDstElts = DstTy->getNumElements();
if (NumDstElts < SrcTy->getNumElements()) {
assert(NumDstElts == 2 && "Unexpected vector size");
[IR] Require ArrayRef of 'uint32_t' instead of 'int' for the mask argument for one of the signatures of CreateShuffleVector. This better emphasises that you can't use it for the -1 as undef behavior. llvm-svn: 272491
2016-06-12 08:41:19 +08:00
uint32_t ShuffleMask[2] = { 0, 1 };
Rep = Builder.CreateShuffleVector(Rep, UndefValue::get(SrcTy),
ShuffleMask);
[X86][SSE] Replace (V)CVTDQ2PD(Y) and (V)CVTPS2PD(Y) lossless conversion intrinsics with generic IR Followup to D20528 clang patch, this removes the (V)CVTDQ2PD(Y) and (V)CVTPS2PD(Y) llvm intrinsics and auto-upgrades to sitofp/fpext instead. Differential Revision: http://reviews.llvm.org/D20568 llvm-svn: 270678
2016-05-25 16:59:18 +08:00
}
[X86][AVX512] Autoupgrade lossless i32/u32 to f64 conversion intrinsics with generic IR Both the (V)CVTDQ2PD (i32 to f64) and (V)CVTUDQ2PD (u32 to f64) conversion instructions are lossless and can be safely represented as generic SINT_TO_FP/UINT_TO_FP calls instead of x86 intrinsics without affecting final codegen. LLVM counterpart to D26686 Differential Revision: https://reviews.llvm.org/D26736 llvm-svn: 287108
2016-11-16 22:48:32 +08:00
bool SInt2Double = (StringRef::npos != Name.find("cvtdq2"));
bool UInt2Double = (StringRef::npos != Name.find("cvtudq2"));
if (SInt2Double)
[X86][SSE] Replace (V)CVTDQ2PD(Y) and (V)CVTPS2PD(Y) lossless conversion intrinsics with generic IR Followup to D20528 clang patch, this removes the (V)CVTDQ2PD(Y) and (V)CVTPS2PD(Y) llvm intrinsics and auto-upgrades to sitofp/fpext instead. Differential Revision: http://reviews.llvm.org/D20568 llvm-svn: 270678
2016-05-25 16:59:18 +08:00
Rep = Builder.CreateSIToFP(Rep, DstTy, "cvtdq2pd");
[X86][AVX512] Autoupgrade lossless i32/u32 to f64 conversion intrinsics with generic IR Both the (V)CVTDQ2PD (i32 to f64) and (V)CVTUDQ2PD (u32 to f64) conversion instructions are lossless and can be safely represented as generic SINT_TO_FP/UINT_TO_FP calls instead of x86 intrinsics without affecting final codegen. LLVM counterpart to D26686 Differential Revision: https://reviews.llvm.org/D26736 llvm-svn: 287108
2016-11-16 22:48:32 +08:00
else if (UInt2Double)
Rep = Builder.CreateUIToFP(Rep, DstTy, "cvtudq2pd");
[X86][SSE] Replace (V)CVTDQ2PD(Y) and (V)CVTPS2PD(Y) lossless conversion intrinsics with generic IR Followup to D20528 clang patch, this removes the (V)CVTDQ2PD(Y) and (V)CVTPS2PD(Y) llvm intrinsics and auto-upgrades to sitofp/fpext instead. Differential Revision: http://reviews.llvm.org/D20568 llvm-svn: 270678
2016-05-25 16:59:18 +08:00
else
Rep = Builder.CreateFPExt(Rep, DstTy, "cvtps2pd");
[X86][AVX512] Autoupgrade lossless i32/u32 to f64 conversion intrinsics with generic IR Both the (V)CVTDQ2PD (i32 to f64) and (V)CVTUDQ2PD (u32 to f64) conversion instructions are lossless and can be safely represented as generic SINT_TO_FP/UINT_TO_FP calls instead of x86 intrinsics without affecting final codegen. LLVM counterpart to D26686 Differential Revision: https://reviews.llvm.org/D26736 llvm-svn: 287108
2016-11-16 22:48:32 +08:00
if (CI->getNumArgOperands() == 3)
Rep = EmitX86Select(Builder, CI->getArgOperand(2), Rep,
CI->getArgOperand(1));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && Name.startswith("sse4a.movnt.")) {
[X86][SSE4A] Autoupgrade and remove MOVNTSD/MOVNTSS intrinsics Required better annotation of the instruction defs upon removal of the builtin intrinsic pattern. llvm-svn: 273077
2016-06-18 10:38:26 +08:00
Module *M = F->getParent();
SmallVector<Metadata *, 1> Elts;
Elts.push_back(
ConstantAsMetadata::get(ConstantInt::get(Type::getInt32Ty(C), 1)));
MDNode *Node = MDNode::get(C, Elts);
Value *Arg0 = CI->getArgOperand(0);
Value *Arg1 = CI->getArgOperand(1);
// Nontemporal (unaligned) store of the 0'th element of the float/double
// vector.
Type *SrcEltTy = cast<VectorType>(Arg1->getType())->getElementType();
PointerType *EltPtrTy = PointerType::getUnqual(SrcEltTy);
Value *Addr = Builder.CreateBitCast(Arg0, EltPtrTy, "cast");
Value *Extract =
Builder.CreateExtractElement(Arg1, (uint64_t)0, "extractelement");
StoreInst *SI = Builder.CreateAlignedStore(Extract, Addr, 1);
SI->setMetadata(M->getMDKindID("nontemporal"), Node);
// Remove intrinsic.
CI->eraseFromParent();
return;
[X86] Remove and autoupgrade 512-bit non-temporal store intrinsics. llvm-svn: 274966
2016-07-09 12:38:27 +08:00
} else if (IsX86 && (Name.startswith("avx.movnt.") ||
Name.startswith("avx512.storent."))) {
Remove 256-bit AVX non-temporal store intrinsics. Similar was previously done for 128-bit. llvm-svn: 156375
2012-05-08 14:58:15 +08:00
Module *M = F->getParent();
IR: Split Metadata from Value Split `Metadata` away from the `Value` class hierarchy, as part of PR21532. Assembly and bitcode changes are in the wings, but this is the bulk of the change for the IR C++ API. I have a follow-up patch prepared for `clang`. If this breaks other sub-projects, I apologize in advance :(. Help me compile it on Darwin I'll try to fix it. FWIW, the errors should be easy to fix, so it may be simpler to just fix it yourself. This breaks the build for all metadata-related code that's out-of-tree. Rest assured the transition is mechanical and the compiler should catch almost all of the problems. Here's a quick guide for updating your code: - `Metadata` is the root of a class hierarchy with three main classes: `MDNode`, `MDString`, and `ValueAsMetadata`. It is distinct from the `Value` class hierarchy. It is typeless -- i.e., instances do *not* have a `Type`. - `MDNode`'s operands are all `Metadata *` (instead of `Value *`). - `TrackingVH<MDNode>` and `WeakVH` referring to metadata can be replaced with `TrackingMDNodeRef` and `TrackingMDRef`, respectively. If you're referring solely to resolved `MDNode`s -- post graph construction -- just use `MDNode*`. - `MDNode` (and the rest of `Metadata`) have only limited support for `replaceAllUsesWith()`. As long as an `MDNode` is pointing at a forward declaration -- the result of `MDNode::getTemporary()` -- it maintains a side map of its uses and can RAUW itself. Once the forward declarations are fully resolved RAUW support is dropped on the ground. This means that uniquing collisions on changing operands cause nodes to become "distinct". (This already happened fairly commonly, whenever an operand went to null.) If you're constructing complex (non self-reference) `MDNode` cycles, you need to call `MDNode::resolveCycles()` on each node (or on a top-level node that somehow references all of the nodes). Also, don't do that. Metadata cycles (and the RAUW machinery needed to construct them) are expensive. - An `MDNode` can only refer to a `Constant` through a bridge called `ConstantAsMetadata` (one of the subclasses of `ValueAsMetadata`). As a side effect, accessing an operand of an `MDNode` that is known to be, e.g., `ConstantInt`, takes three steps: first, cast from `Metadata` to `ConstantAsMetadata`; second, extract the `Constant`; third, cast down to `ConstantInt`. The eventual goal is to introduce `MDInt`/`MDFloat`/etc. and have metadata schema owners transition away from using `Constant`s when the type isn't important (and they don't care about referring to `GlobalValue`s). In the meantime, I've added transitional API to the `mdconst` namespace that matches semantics with the old code, in order to avoid adding the error-prone three-step equivalent to every call site. If your old code was: MDNode *N = foo(); bar(isa <ConstantInt>(N->getOperand(0))); baz(cast <ConstantInt>(N->getOperand(1))); bak(cast_or_null <ConstantInt>(N->getOperand(2))); bat(dyn_cast <ConstantInt>(N->getOperand(3))); bay(dyn_cast_or_null<ConstantInt>(N->getOperand(4))); you can trivially match its semantics with: MDNode *N = foo(); bar(mdconst::hasa <ConstantInt>(N->getOperand(0))); baz(mdconst::extract <ConstantInt>(N->getOperand(1))); bak(mdconst::extract_or_null <ConstantInt>(N->getOperand(2))); bat(mdconst::dyn_extract <ConstantInt>(N->getOperand(3))); bay(mdconst::dyn_extract_or_null<ConstantInt>(N->getOperand(4))); and when you transition your metadata schema to `MDInt`: MDNode *N = foo(); bar(isa <MDInt>(N->getOperand(0))); baz(cast <MDInt>(N->getOperand(1))); bak(cast_or_null <MDInt>(N->getOperand(2))); bat(dyn_cast <MDInt>(N->getOperand(3))); bay(dyn_cast_or_null<MDInt>(N->getOperand(4))); - A `CallInst` -- specifically, intrinsic instructions -- can refer to metadata through a bridge called `MetadataAsValue`. This is a subclass of `Value` where `getType()->isMetadataTy()`. `MetadataAsValue` is the *only* class that can legally refer to a `LocalAsMetadata`, which is a bridged form of non-`Constant` values like `Argument` and `Instruction`. It can also refer to any other `Metadata` subclass. (I'll break all your testcases in a follow-up commit, when I propagate this change to assembly.) llvm-svn: 223802
2014-12-10 02:38:53 +08:00
SmallVector<Metadata *, 1> Elts;
Elts.push_back(
ConstantAsMetadata::get(ConstantInt::get(Type::getInt32Ty(C), 1)));
Remove 256-bit AVX non-temporal store intrinsics. Similar was previously done for 128-bit. llvm-svn: 156375
2012-05-08 14:58:15 +08:00
MDNode *Node = MDNode::get(C, Elts);
Value *Arg0 = CI->getArgOperand(0);
Value *Arg1 = CI->getArgOperand(1);
// Convert the type of the pointer to a pointer to the stored type.
Value *BC = Builder.CreateBitCast(Arg0,
PointerType::getUnqual(Arg1->getType()),
"cast");
[X86] Remove and autoupgrade 512-bit non-temporal store intrinsics. llvm-svn: 274966
2016-07-09 12:38:27 +08:00
VectorType *VTy = cast<VectorType>(Arg1->getType());
StoreInst *SI = Builder.CreateAlignedStore(Arg1, BC,
VTy->getBitWidth() / 8);
Remove 256-bit AVX non-temporal store intrinsics. Similar was previously done for 128-bit. llvm-svn: 156375
2012-05-08 14:58:15 +08:00
SI->setMetadata(M->getMDKindID("nontemporal"), Node);
[X86] Remove the llvm.x86.sse2.storel.dq intrinsic. It hasn't been used in a long time. llvm-svn: 270677
2016-05-25 14:56:32 +08:00
// Remove intrinsic.
CI->eraseFromParent();
return;
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && Name == "sse2.storel.dq") {
[X86] Remove the llvm.x86.sse2.storel.dq intrinsic. It hasn't been used in a long time. llvm-svn: 270677
2016-05-25 14:56:32 +08:00
Value *Arg0 = CI->getArgOperand(0);
Value *Arg1 = CI->getArgOperand(1);
Type *NewVecTy = VectorType::get(Type::getInt64Ty(C), 2);
Value *BC0 = Builder.CreateBitCast(Arg1, NewVecTy, "cast");
Value *Elt = Builder.CreateExtractElement(BC0, (uint64_t)0);
Value *BC = Builder.CreateBitCast(Arg0,
PointerType::getUnqual(Elt->getType()),
"cast");
[X86] Simplify and remove some unnecessary auto-upgrade code. llvm-svn: 271233
2016-05-31 06:54:12 +08:00
Builder.CreateAlignedStore(Elt, BC, 1);
[X86] Remove the llvm.x86.sse2.storel.dq intrinsic. It hasn't been used in a long time. llvm-svn: 270677
2016-05-25 14:56:32 +08:00
[X86] Remove SSE/AVX unaligned store intrinsics as clang no longer uses them. Auto upgrade to native unaligned store instructions. llvm-svn: 271236
2016-05-31 07:15:56 +08:00
// Remove intrinsic.
CI->eraseFromParent();
return;
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("sse.storeu.") ||
Name.startswith("sse2.storeu.") ||
Name.startswith("avx.storeu."))) {
[X86] Remove SSE/AVX unaligned store intrinsics as clang no longer uses them. Auto upgrade to native unaligned store instructions. llvm-svn: 271236
2016-05-31 07:15:56 +08:00
Value *Arg0 = CI->getArgOperand(0);
Value *Arg1 = CI->getArgOperand(1);
Arg0 = Builder.CreateBitCast(Arg0,
PointerType::getUnqual(Arg1->getType()),
"cast");
Builder.CreateAlignedStore(Arg1, Arg0, 1);
[AVX512] Remove masked store intrinsics. Clang now emits generic masked store intrinsics instead. The intrinsics will be autoupgraded to the same generic masked stores. llvm-svn: 271245
2016-05-31 09:50:02 +08:00
// Remove intrinsic.
CI->eraseFromParent();
return;
[X86] Combine some of the strings in autoupgrade code. llvm-svn: 280603
2016-09-04 07:55:13 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.storeu."))) {
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
UpgradeMaskedStore(Builder, CI->getArgOperand(0), CI->getArgOperand(1),
[AVX512] Remove masked store intrinsics. Clang now emits generic masked store intrinsics instead. The intrinsics will be autoupgraded to the same generic masked stores. llvm-svn: 271245
2016-05-31 09:50:02 +08:00
CI->getArgOperand(2), /*Aligned*/false);
// Remove intrinsic.
CI->eraseFromParent();
return;
[X86] Combine some of the strings in autoupgrade code. llvm-svn: 280603
2016-09-04 07:55:13 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.store."))) {
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
UpgradeMaskedStore(Builder, CI->getArgOperand(0), CI->getArgOperand(1),
[AVX512] Remove masked store intrinsics. Clang now emits generic masked store intrinsics instead. The intrinsics will be autoupgraded to the same generic masked stores. llvm-svn: 271245
2016-05-31 09:50:02 +08:00
CI->getArgOperand(2), /*Aligned*/true);
Remove 256-bit AVX non-temporal store intrinsics. Similar was previously done for 128-bit. llvm-svn: 156375
2012-05-08 14:58:15 +08:00
// Remove intrinsic.
CI->eraseFromParent();
return;
[X86] Combine some of the strings in autoupgrade code. llvm-svn: 280603
2016-09-04 07:55:13 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.loadu."))) {
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
Rep = UpgradeMaskedLoad(Builder, CI->getArgOperand(0),
[AVX512] Remove masked load intrinsics. Clang now emits generic masked load intrinsics instead. The intrinsics will be autoupgraded to the same generic masked loads. llvm-svn: 271478
2016-06-02 12:19:36 +08:00
CI->getArgOperand(1), CI->getArgOperand(2),
/*Aligned*/false);
[X86] Combine some of the strings in autoupgrade code. llvm-svn: 280603
2016-09-04 07:55:13 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.load."))) {
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
Rep = UpgradeMaskedLoad(Builder, CI->getArgOperand(0),
[AVX512] Remove masked load intrinsics. Clang now emits generic masked load intrinsics instead. The intrinsics will be autoupgraded to the same generic masked loads. llvm-svn: 271478
2016-06-02 12:19:36 +08:00
CI->getArgOperand(1),CI->getArgOperand(2),
/*Aligned*/true);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && Name.startswith("xop.vpcom")) {
Replace XOP vpcom intrinsics with fewer intrinsics that take the immediate as an argument. llvm-svn: 158278
2012-06-10 00:46:13 +08:00
Intrinsic::ID intID;
if (Name.endswith("ub"))
intID = Intrinsic::x86_xop_vpcomub;
else if (Name.endswith("uw"))
intID = Intrinsic::x86_xop_vpcomuw;
else if (Name.endswith("ud"))
intID = Intrinsic::x86_xop_vpcomud;
else if (Name.endswith("uq"))
intID = Intrinsic::x86_xop_vpcomuq;
else if (Name.endswith("b"))
intID = Intrinsic::x86_xop_vpcomb;
else if (Name.endswith("w"))
intID = Intrinsic::x86_xop_vpcomw;
else if (Name.endswith("d"))
intID = Intrinsic::x86_xop_vpcomd;
else if (Name.endswith("q"))
intID = Intrinsic::x86_xop_vpcomq;
else
llvm_unreachable("Unknown suffix");
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
Name = Name.substr(9); // strip off "xop.vpcom"
Replace XOP vpcom intrinsics with fewer intrinsics that take the immediate as an argument. llvm-svn: 158278
2012-06-10 00:46:13 +08:00
unsigned Imm;
if (Name.startswith("lt"))
Imm = 0;
else if (Name.startswith("le"))
Imm = 1;
else if (Name.startswith("gt"))
Imm = 2;
else if (Name.startswith("ge"))
Imm = 3;
else if (Name.startswith("eq"))
Imm = 4;
else if (Name.startswith("ne"))
Imm = 5;
else if (Name.startswith("false"))
[X86] Fix XOP vpcom intrinsic autoupgrade to map 'true' and 'false' to the correct immediates. Seems they were swapped. llvm-svn: 229077
2015-02-13 15:42:15 +08:00
Imm = 6;
else if (Name.startswith("true"))
Replace XOP vpcom intrinsics with fewer intrinsics that take the immediate as an argument. llvm-svn: 158278
2012-06-10 00:46:13 +08:00
Imm = 7;
else
llvm_unreachable("Unknown condition");
Function *VPCOM = Intrinsic::getDeclaration(F->getParent(), intID);
Simplify IRBuilder::CreateCall* by using ArrayRef+initializer_list/braced init only llvm-svn: 237624
2015-05-19 06:13:54 +08:00
Rep =
Builder.CreateCall(VPCOM, {CI->getArgOperand(0), CI->getArgOperand(1),
Builder.getInt8(Imm)});
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && Name == "xop.vpcmov") {
[X86][XOP] Add support for the matching of the VPCMOV bit select instruction XOP has the VPCMOV instruction that performs the common vector bit select operation OR( AND( SRC1, SRC3 ), AND( SRC2, ~SRC3 ) ) This patch adds tablegen pattern matching for this instruction. Differential Revision: http://reviews.llvm.org/D8841 llvm-svn: 251975
2015-11-04 04:27:01 +08:00
Value *Arg0 = CI->getArgOperand(0);
Value *Arg1 = CI->getArgOperand(1);
Value *Sel = CI->getArgOperand(2);
unsigned NumElts = CI->getType()->getVectorNumElements();
Constant *MinusOne = ConstantVector::getSplat(NumElts, Builder.getInt64(-1));
Value *NotSel = Builder.CreateXor(Sel, MinusOne);
Value *Sel0 = Builder.CreateAnd(Arg0, Sel);
Value *Sel1 = Builder.CreateAnd(Arg1, NotSel);
Rep = Builder.CreateOr(Sel0, Sel1);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && Name == "sse42.crc32.64.8") {
Remove x86_sse42_crc32_64_8 intrinsic. It has no functional difference from x86_sse42_crc32_32_8 and was not mapped to a clang builtin. I'm not even sure why this form of the instruction is even called out explicitly in the docs. Also add AutoUpgrade support to convert it into the other intrinsic with appropriate trunc and zext. llvm-svn: 192672
2013-10-15 13:20:47 +08:00
Function *CRC32 = Intrinsic::getDeclaration(F->getParent(),
Intrinsic::x86_sse42_crc32_32_8);
Value *Trunc0 = Builder.CreateTrunc(CI->getArgOperand(0), Type::getInt32Ty(C));
Simplify IRBuilder::CreateCall* by using ArrayRef+initializer_list/braced init only llvm-svn: 237624
2015-05-19 06:13:54 +08:00
Rep = Builder.CreateCall(CRC32, {Trunc0, CI->getArgOperand(1)});
Remove x86_sse42_crc32_64_8 intrinsic. It has no functional difference from x86_sse42_crc32_32_8 and was not mapped to a clang builtin. I'm not even sure why this form of the instruction is even called out explicitly in the docs. Also add AutoUpgrade support to convert it into the other intrinsic with appropriate trunc and zext. llvm-svn: 192672
2013-10-15 13:20:47 +08:00
Rep = Builder.CreateZExt(Rep, CI->getType(), "");
[X86][AVX] Added support for lowering to VBROADCASTF128/VBROADCASTI128 (reapplied) As reported on PR26235, we don't currently make use of the VBROADCASTF128/VBROADCASTI128 instructions (or the AVX512 equivalents) to load+splat a 128-bit vector to both lanes of a 256-bit vector. This patch enables lowering from subvector insertion/concatenation patterns and auto-upgrades the llvm.x86.avx.vbroadcastf128.pd.256 / llvm.x86.avx.vbroadcastf128.ps.256 intrinsics to match. We could possibly investigate using VBROADCASTF128/VBROADCASTI128 to load repeated constants as well (similar to how we already do for scalar broadcasts). Reapplied with fix for PR28657 - removed intrinsic definitions (clang companion patch to be be submitted shortly). Differential Revision: https://reviews.llvm.org/D22460 llvm-svn: 276416
2016-07-22 21:58:44 +08:00
} else if (IsX86 && Name.startswith("avx.vbroadcast.s")) {
[X86] Auto-upgrade AVX1 vbroadcast intrinsics They are replaced with the same IR that is generated for the vector-initializers in avxintrin.h. The test verifies that we get back the original instruction. I haven't seen this approach to be used in other auto-upgrade tests (i.e. llc + FileCheck) but I think it's the most direct way to test this case. I believe this should work because llc upgrades calls during parsing. (Other tests mostly check that assembling and disassembling yields the upgraded IR.) llvm-svn: 209863
2014-05-30 07:35:33 +08:00
// Replace broadcasts with a series of insertelements.
Type *VecTy = CI->getType();
Type *EltTy = VecTy->getVectorElementType();
unsigned EltNum = VecTy->getVectorNumElements();
Value *Cast = Builder.CreateBitCast(CI->getArgOperand(0),
EltTy->getPointerTo());
[opaque pointer type] Pass explicit type to Load instruction creation in AutoUpgrade llvm-svn: 237838
2015-05-21 05:46:30 +08:00
Value *Load = Builder.CreateLoad(EltTy, Cast);
[X86] Auto-upgrade AVX1 vbroadcast intrinsics They are replaced with the same IR that is generated for the vector-initializers in avxintrin.h. The test verifies that we get back the original instruction. I haven't seen this approach to be used in other auto-upgrade tests (i.e. llc + FileCheck) but I think it's the most direct way to test this case. I believe this should work because llc upgrades calls during parsing. (Other tests mostly check that assembling and disassembling yields the upgraded IR.) llvm-svn: 209863
2014-05-30 07:35:33 +08:00
Type *I32Ty = Type::getInt32Ty(C);
Rep = UndefValue::get(VecTy);
for (unsigned I = 0; I < EltNum; ++I)
Rep = Builder.CreateInsertElement(Rep, Load,
ConstantInt::get(I32Ty, I));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("sse41.pmovsx") ||
Name.startswith("sse41.pmovzx") ||
Name.startswith("avx2.pmovsx") ||
[AVX-512] Remove masked pmovzx/pmovsx builtins and autoupgrade them to selects and native zext/sext. This mostly reuses earlier autoupgrade support for the sse and avx equivalents. Just needed to add the code to add the select. llvm-svn: 286092
2016-11-07 10:12:57 +08:00
Name.startswith("avx2.pmovzx") ||
Name.startswith("avx512.mask.pmovsx") ||
Name.startswith("avx512.mask.pmovzx"))) {
[X86][SSE] Replace 128-bit SSE41 PMOVSX intrinsics with native IR This patches removes the x86.sse41.pmovsx* intrinsics, provides a suitable upgrade path and updates relevant tests to sign extend a subvector instead. LLVM counterpart to D12835 Differential Revision: http://reviews.llvm.org/D13002 llvm-svn: 248368
2015-09-23 16:48:33 +08:00
VectorType *SrcTy = cast<VectorType>(CI->getArgOperand(0)->getType());
VectorType *DstTy = cast<VectorType>(CI->getType());
unsigned NumDstElts = DstTy->getNumElements();
[X86][SSE] (Reapplied) Replace (V)PMOVSX and (V)PMOVZX integer extension intrinsics with generic IR (llvm) This patch removes the llvm intrinsics VPMOVSX and (V)PMOVZX sign/zero extension intrinsics and auto-upgrades to SEXT/ZEXT calls instead. We already did this for SSE41 PMOVSX sometime ago so much of that implementation can be reused. Reapplied now that the the companion patch (D20684) removes/auto-upgrade the clang intrinsics has been committed. Differential Revision: http://reviews.llvm.org/D20686 llvm-svn: 271131
2016-05-29 02:03:41 +08:00
// Extract a subvector of the first NumDstElts lanes and sign/zero extend.
[X86] Pre-allocate some of the shuffle mask SmallVectors in the auto upgrade code instead of calling push_back in a loop. This removes the need to check if the vector needs to grow on each iteration. llvm-svn: 272501
2016-06-12 12:48:00 +08:00
SmallVector<uint32_t, 8> ShuffleMask(NumDstElts);
[IR] Require ArrayRef of 'uint32_t' instead of 'int' for the mask argument for one of the signatures of CreateShuffleVector. This better emphasises that you can't use it for the -1 as undef behavior. llvm-svn: 272491
2016-06-12 08:41:19 +08:00
for (unsigned i = 0; i != NumDstElts; ++i)
[X86] Pre-allocate some of the shuffle mask SmallVectors in the auto upgrade code instead of calling push_back in a loop. This removes the need to check if the vector needs to grow on each iteration. llvm-svn: 272501
2016-06-12 12:48:00 +08:00
ShuffleMask[i] = i;
[X86][SSE] Replace 128-bit SSE41 PMOVSX intrinsics with native IR This patches removes the x86.sse41.pmovsx* intrinsics, provides a suitable upgrade path and updates relevant tests to sign extend a subvector instead. LLVM counterpart to D12835 Differential Revision: http://reviews.llvm.org/D13002 llvm-svn: 248368
2015-09-23 16:48:33 +08:00
Value *SV = Builder.CreateShuffleVector(
CI->getArgOperand(0), UndefValue::get(SrcTy), ShuffleMask);
[X86][SSE] (Reapplied) Replace (V)PMOVSX and (V)PMOVZX integer extension intrinsics with generic IR (llvm) This patch removes the llvm intrinsics VPMOVSX and (V)PMOVZX sign/zero extension intrinsics and auto-upgrades to SEXT/ZEXT calls instead. We already did this for SSE41 PMOVSX sometime ago so much of that implementation can be reused. Reapplied now that the the companion patch (D20684) removes/auto-upgrade the clang intrinsics has been committed. Differential Revision: http://reviews.llvm.org/D20686 llvm-svn: 271131
2016-05-29 02:03:41 +08:00
bool DoSext = (StringRef::npos != Name.find("pmovsx"));
Rep = DoSext ? Builder.CreateSExt(SV, DstTy)
: Builder.CreateZExt(SV, DstTy);
[AVX-512] Remove masked pmovzx/pmovsx builtins and autoupgrade them to selects and native zext/sext. This mostly reuses earlier autoupgrade support for the sse and avx equivalents. Just needed to add the code to add the select. llvm-svn: 286092
2016-11-07 10:12:57 +08:00
// If there are 3 arguments, it's a masked intrinsic so we need a select.
if (CI->getNumArgOperands() == 3)
Rep = EmitX86Select(Builder, CI->getArgOperand(2), Rep,
CI->getArgOperand(1));
[X86][AVX] Added support for lowering to VBROADCASTF128/VBROADCASTI128 (reapplied) As reported on PR26235, we don't currently make use of the VBROADCASTF128/VBROADCASTI128 instructions (or the AVX512 equivalents) to load+splat a 128-bit vector to both lanes of a 256-bit vector. This patch enables lowering from subvector insertion/concatenation patterns and auto-upgrades the llvm.x86.avx.vbroadcastf128.pd.256 / llvm.x86.avx.vbroadcastf128.ps.256 intrinsics to match. We could possibly investigate using VBROADCASTF128/VBROADCASTI128 to load repeated constants as well (similar to how we already do for scalar broadcasts). Reapplied with fix for PR28657 - removed intrinsic definitions (clang companion patch to be be submitted shortly). Differential Revision: https://reviews.llvm.org/D22460 llvm-svn: 276416
2016-07-22 21:58:44 +08:00
} else if (IsX86 && (Name.startswith("avx.vbroadcastf128") ||
Name == "avx2.vbroadcasti128")) {
// Replace vbroadcastf128/vbroadcasti128 with a vector load+shuffle.
Type *EltTy = CI->getType()->getVectorElementType();
unsigned NumSrcElts = 128 / EltTy->getPrimitiveSizeInBits();
Type *VT = VectorType::get(EltTy, NumSrcElts);
[opaque pointer type] Pass explicit type to Load instruction creation in AutoUpgrade llvm-svn: 237838
2015-05-21 05:46:30 +08:00
Value *Op = Builder.CreatePointerCast(CI->getArgOperand(0),
PointerType::getUnqual(VT));
[x86] Fix a bug in the auto-upgrade from r276416 where we failed to give a sufficiently low alignment for the IR load created. There is no test case because we don't have any test cases for the *IR* produced by the autoupgrade, only the x86 assembly, and it happens that the x86 assembly for this intrinsic as it is tested in the autoupgrade path just happens to not produce a separate load instruction where we might have observed the alignment. I'm going to follow up on the original commit to suggest getting IR-level testing in addition to the asm level testing here so that we can see and test these kinds of issues. We might never get an x86 instruction out with an alignment constraint, but we could stil miscompile code by folding against the alignment marked on (or inferred for in this case) the load. llvm-svn: 278203
2016-08-10 15:41:26 +08:00
Value *Load = Builder.CreateAlignedLoad(Op, 1);
[X86][AVX] Added support for lowering to VBROADCASTF128/VBROADCASTI128 (reapplied) As reported on PR26235, we don't currently make use of the VBROADCASTF128/VBROADCASTI128 instructions (or the AVX512 equivalents) to load+splat a 128-bit vector to both lanes of a 256-bit vector. This patch enables lowering from subvector insertion/concatenation patterns and auto-upgrades the llvm.x86.avx.vbroadcastf128.pd.256 / llvm.x86.avx.vbroadcastf128.ps.256 intrinsics to match. We could possibly investigate using VBROADCASTF128/VBROADCASTI128 to load repeated constants as well (similar to how we already do for scalar broadcasts). Reapplied with fix for PR28657 - removed intrinsic definitions (clang companion patch to be be submitted shortly). Differential Revision: https://reviews.llvm.org/D22460 llvm-svn: 276416
2016-07-22 21:58:44 +08:00
if (NumSrcElts == 2)
Rep = Builder.CreateShuffleVector(Load, UndefValue::get(Load->getType()),
{ 0, 1, 0, 1 });
else
Rep = Builder.CreateShuffleVector(Load, UndefValue::get(Load->getType()),
{ 0, 1, 2, 3, 0, 1, 2, 3 });
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("avx2.pbroadcast") ||
[X86][AVX512] Autoupgrade the BROADCAST intrinsics llvm-svn: 274550
2016-07-05 21:58:47 +08:00
Name.startswith("avx2.vbroadcast") ||
Name.startswith("avx512.pbroadcast") ||
Name.startswith("avx512.mask.broadcast.s"))) {
[X86] Replace avx2 broadcast intrinsics with native IR. Since r245605, the clang headers don't use these anymore. r245165 updated some of the tests already; update the others, add an autoupgrade, remove the intrinsics, and cleanup the definitions. Differential Revision: http://reviews.llvm.org/D10555 llvm-svn: 245606
2015-08-21 04:36:19 +08:00
// Replace vp?broadcasts with a vector shuffle.
Value *Op = CI->getArgOperand(0);
unsigned NumElts = CI->getType()->getVectorNumElements();
Type *MaskTy = VectorType::get(Type::getInt32Ty(C), NumElts);
Rep = Builder.CreateShuffleVector(Op, UndefValue::get(Op->getType()),
Constant::getNullValue(MaskTy));
[X86][AVX512] Autoupgrade the BROADCAST intrinsics llvm-svn: 274550
2016-07-05 21:58:47 +08:00
if (CI->getNumArgOperands() == 3)
Rep = EmitX86Select(Builder, CI->getArgOperand(2), Rep,
CI->getArgOperand(1));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.palignr.")) {
[AVX-512] Remove intrinsics for valignd/q and autoupgrade them to native shuffles. llvm-svn: 287744
2016-11-23 14:54:55 +08:00
Rep = UpgradeX86ALIGNIntrinsics(Builder, CI->getArgOperand(0),
CI->getArgOperand(1),
CI->getArgOperand(2),
CI->getArgOperand(3),
CI->getArgOperand(4),
false);
} else if (IsX86 && Name.startswith("avx512.mask.valign.")) {
Rep = UpgradeX86ALIGNIntrinsics(Builder, CI->getArgOperand(0),
CI->getArgOperand(1),
CI->getArgOperand(2),
CI->getArgOperand(3),
CI->getArgOperand(4),
true);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.psll.dq" ||
Name == "avx2.psll.dq")) {
[X86] Simplify some of the autoupgrade code. NFC llvm-svn: 271174
2016-05-29 14:37:33 +08:00
// 128/256-bit shift left specified in bits.
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
unsigned Shift = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
Rep = UpgradeX86PSLLDQIntrinsics(Builder, CI->getArgOperand(0),
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
Shift / 8); // Shift is in bits.
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.psrl.dq" ||
Name == "avx2.psrl.dq")) {
[X86] Simplify some of the autoupgrade code. NFC llvm-svn: 271174
2016-05-29 14:37:33 +08:00
// 128/256-bit shift right specified in bits.
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
unsigned Shift = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
Rep = UpgradeX86PSRLDQIntrinsics(Builder, CI->getArgOperand(0),
[X86] Remove AVX2 and SSE2 pslldq and psrldq intrinsics. We can represent them in IR with vector shuffles now. All their uses have been removed from clang in favor of shuffles. llvm-svn: 229640
2015-02-18 14:24:44 +08:00
Shift / 8); // Shift is in bits.
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.psll.dq.bs" ||
Name == "avx2.psll.dq.bs" ||
Name == "avx512.psll.dq.512")) {
[X86][AVX512] Dropped avx512 VPSLLDQ/VPSRLDQ intrinsics Auto-upgrade to generic shuffles like sse/avx2 implementations now that we can lower to VPSLLDQ/VPSRLDQ llvm-svn: 272308
2016-06-10 05:09:03 +08:00
// 128/256/512-bit shift left specified in bytes.
[X86] Remove x86.avx2.psll.dq.bs and x86.avx2.psrl.dq.bs intrinsics. llvm-svn: 229430
2015-02-17 04:51:59 +08:00
unsigned Shift = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
Rep = UpgradeX86PSLLDQIntrinsics(Builder, CI->getArgOperand(0), Shift);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.psrl.dq.bs" ||
Name == "avx2.psrl.dq.bs" ||
Name == "avx512.psrl.dq.512")) {
[X86][AVX512] Dropped avx512 VPSLLDQ/VPSRLDQ intrinsics Auto-upgrade to generic shuffles like sse/avx2 implementations now that we can lower to VPSLLDQ/VPSRLDQ llvm-svn: 272308
2016-06-10 05:09:03 +08:00
// 128/256/512-bit shift right specified in bytes.
[X86] Remove x86.avx2.psll.dq.bs and x86.avx2.psrl.dq.bs intrinsics. llvm-svn: 229430
2015-02-17 04:51:59 +08:00
unsigned Shift = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
[X86,IR] Remove unnecessary or unused LLVMContext parameter from some of the X86 intrinsic upgrade functions. llvm-svn: 275138
2016-07-12 09:42:33 +08:00
Rep = UpgradeX86PSRLDQIntrinsics(Builder, CI->getArgOperand(0), Shift);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse41.pblendw" ||
Name.startswith("sse41.blendp") ||
Name.startswith("avx.blend.p") ||
Name == "avx2.pblendw" ||
Name.startswith("avx2.pblendd."))) {
[X86] Remove the blendpd/blendps/pblendw/pblendd intrinsics. They can represented by shuffle_vector instructions. llvm-svn: 230860
2015-03-01 03:33:17 +08:00
Value *Op0 = CI->getArgOperand(0);
Value *Op1 = CI->getArgOperand(1);
unsigned Imm = cast <ConstantInt>(CI->getArgOperand(2))->getZExtValue();
VectorType *VecTy = cast<VectorType>(CI->getType());
unsigned NumElts = VecTy->getNumElements();
[X86] Pre-allocate some of the shuffle mask SmallVectors in the auto upgrade code instead of calling push_back in a loop. This removes the need to check if the vector needs to grow on each iteration. llvm-svn: 272501
2016-06-12 12:48:00 +08:00
SmallVector<uint32_t, 16> Idxs(NumElts);
for (unsigned i = 0; i != NumElts; ++i)
Idxs[i] = ((Imm >> (i%8)) & 1) ? i + NumElts : i;
[X86] Remove the blendpd/blendps/pblendw/pblendd intrinsics. They can represented by shuffle_vector instructions. llvm-svn: 230860
2015-03-01 03:33:17 +08:00
[X86,IR] Make use of the CreateShuffleVector form that takes an ArrayRef<uint32_t> to avoid the need to manually create a bunch of Constants and a ConstantVector. NFC llvm-svn: 272493
2016-06-12 09:05:59 +08:00
Rep = Builder.CreateShuffleVector(Op0, Op1, Idxs);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("avx.vinsertf128.") ||
Name == "avx2.vinserti128")) {
[X86, AVX] replace vinsertf128 intrinsics with generic shuffles We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is the sibling patch for the Clang half of this change: http://reviews.llvm.org/D8088 Differential Revision: http://reviews.llvm.org/D8086 llvm-svn: 231794
2015-03-11 00:08:36 +08:00
Value *Op0 = CI->getArgOperand(0);
Value *Op1 = CI->getArgOperand(1);
unsigned Imm = cast<ConstantInt>(CI->getArgOperand(2))->getZExtValue();
VectorType *VecTy = cast<VectorType>(CI->getType());
unsigned NumElts = VecTy->getNumElements();
[X86][SSE] Replace 128-bit SSE41 PMOVSX intrinsics with native IR This patches removes the x86.sse41.pmovsx* intrinsics, provides a suitable upgrade path and updates relevant tests to sign extend a subvector instead. LLVM counterpart to D12835 Differential Revision: http://reviews.llvm.org/D13002 llvm-svn: 248368
2015-09-23 16:48:33 +08:00
[X86, AVX] replace vinsertf128 intrinsics with generic shuffles We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is the sibling patch for the Clang half of this change: http://reviews.llvm.org/D8088 Differential Revision: http://reviews.llvm.org/D8086 llvm-svn: 231794
2015-03-11 00:08:36 +08:00
// Mask off the high bits of the immediate value; hardware ignores those.
Imm = Imm & 1;
[X86][SSE] Replace 128-bit SSE41 PMOVSX intrinsics with native IR This patches removes the x86.sse41.pmovsx* intrinsics, provides a suitable upgrade path and updates relevant tests to sign extend a subvector instead. LLVM counterpart to D12835 Differential Revision: http://reviews.llvm.org/D13002 llvm-svn: 248368
2015-09-23 16:48:33 +08:00
[X86, AVX] replace vinsertf128 intrinsics with generic shuffles We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is the sibling patch for the Clang half of this change: http://reviews.llvm.org/D8088 Differential Revision: http://reviews.llvm.org/D8086 llvm-svn: 231794
2015-03-11 00:08:36 +08:00
// Extend the second operand into a vector that is twice as big.
Value *UndefV = UndefValue::get(Op1->getType());
[X86] Pre-allocate some of the shuffle mask SmallVectors in the auto upgrade code instead of calling push_back in a loop. This removes the need to check if the vector needs to grow on each iteration. llvm-svn: 272501
2016-06-12 12:48:00 +08:00
SmallVector<uint32_t, 8> Idxs(NumElts);
for (unsigned i = 0; i != NumElts; ++i)
Idxs[i] = i;
[X86,IR] Make use of the CreateShuffleVector form that takes an ArrayRef<uint32_t> to avoid the need to manually create a bunch of Constants and a ConstantVector. NFC llvm-svn: 272493
2016-06-12 09:05:59 +08:00
Rep = Builder.CreateShuffleVector(Op1, UndefV, Idxs);
[X86, AVX] replace vinsertf128 intrinsics with generic shuffles We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is the sibling patch for the Clang half of this change: http://reviews.llvm.org/D8088 Differential Revision: http://reviews.llvm.org/D8086 llvm-svn: 231794
2015-03-11 00:08:36 +08:00
// Insert the second operand into the first operand.
// Note that there is no guarantee that instruction lowering will actually
// produce a vinsertf128 instruction for the created shuffles. In
// particular, the 0 immediate case involves no lane changes, so it can
// be handled as a blend.
// Example of shuffle mask for 32-bit elements:
// Imm = 1 <i32 0, i32 1, i32 2, i32 3, i32 8, i32 9, i32 10, i32 11>
// Imm = 0 <i32 8, i32 9, i32 10, i32 11, i32 4, i32 5, i32 6, i32 7 >
// The low half of the result is either the low half of the 1st operand
// or the low half of the 2nd operand (the inserted vector).
[X86] Pre-allocate some of the shuffle mask SmallVectors in the auto upgrade code instead of calling push_back in a loop. This removes the need to check if the vector needs to grow on each iteration. llvm-svn: 272501
2016-06-12 12:48:00 +08:00
for (unsigned i = 0; i != NumElts / 2; ++i)
Idxs[i] = Imm ? i : (i + NumElts);
[X86, AVX] replace vinsertf128 intrinsics with generic shuffles We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is the sibling patch for the Clang half of this change: http://reviews.llvm.org/D8088 Differential Revision: http://reviews.llvm.org/D8086 llvm-svn: 231794
2015-03-11 00:08:36 +08:00
// The high half of the result is either the low half of the 2nd operand
// (the inserted vector) or the high half of the 1st operand.
[X86] Pre-allocate some of the shuffle mask SmallVectors in the auto upgrade code instead of calling push_back in a loop. This removes the need to check if the vector needs to grow on each iteration. llvm-svn: 272501
2016-06-12 12:48:00 +08:00
for (unsigned i = NumElts / 2; i != NumElts; ++i)
Idxs[i] = Imm ? (i + NumElts / 2) : i;
[X86,IR] Make use of the CreateShuffleVector form that takes an ArrayRef<uint32_t> to avoid the need to manually create a bunch of Constants and a ConstantVector. NFC llvm-svn: 272493
2016-06-12 09:05:59 +08:00
Rep = Builder.CreateShuffleVector(Op0, Rep, Idxs);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("avx.vextractf128.") ||
Name == "avx2.vextracti128")) {
[X86, AVX] replace vextractf128 intrinsics with generic shuffles Now that we've replaced the vinsertf128 intrinsics, do the same for their extract twins. This is very much like D8086 (checked in at r231794): We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is also the LLVM sibling to the cfe D8275 patch. Differential Revision: http://reviews.llvm.org/D8276 llvm-svn: 232045
2015-03-12 23:15:19 +08:00
Value *Op0 = CI->getArgOperand(0);
unsigned Imm = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
VectorType *VecTy = cast<VectorType>(CI->getType());
unsigned NumElts = VecTy->getNumElements();
[X86][SSE] Replace 128-bit SSE41 PMOVSX intrinsics with native IR This patches removes the x86.sse41.pmovsx* intrinsics, provides a suitable upgrade path and updates relevant tests to sign extend a subvector instead. LLVM counterpart to D12835 Differential Revision: http://reviews.llvm.org/D13002 llvm-svn: 248368
2015-09-23 16:48:33 +08:00
[X86, AVX] replace vextractf128 intrinsics with generic shuffles Now that we've replaced the vinsertf128 intrinsics, do the same for their extract twins. This is very much like D8086 (checked in at r231794): We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is also the LLVM sibling to the cfe D8275 patch. Differential Revision: http://reviews.llvm.org/D8276 llvm-svn: 232045
2015-03-12 23:15:19 +08:00
// Mask off the high bits of the immediate value; hardware ignores those.
Imm = Imm & 1;
// Get indexes for either the high half or low half of the input vector.
[X86,IR] Make use of the CreateShuffleVector form that takes an ArrayRef<uint32_t> to avoid the need to manually create a bunch of Constants and a ConstantVector. NFC llvm-svn: 272493
2016-06-12 09:05:59 +08:00
SmallVector<uint32_t, 4> Idxs(NumElts);
[X86, AVX] replace vextractf128 intrinsics with generic shuffles Now that we've replaced the vinsertf128 intrinsics, do the same for their extract twins. This is very much like D8086 (checked in at r231794): We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is also the LLVM sibling to the cfe D8275 patch. Differential Revision: http://reviews.llvm.org/D8276 llvm-svn: 232045
2015-03-12 23:15:19 +08:00
for (unsigned i = 0; i != NumElts; ++i) {
[X86,IR] Make use of the CreateShuffleVector form that takes an ArrayRef<uint32_t> to avoid the need to manually create a bunch of Constants and a ConstantVector. NFC llvm-svn: 272493
2016-06-12 09:05:59 +08:00
Idxs[i] = Imm ? (i + NumElts) : i;
[X86, AVX] replace vextractf128 intrinsics with generic shuffles Now that we've replaced the vinsertf128 intrinsics, do the same for their extract twins. This is very much like D8086 (checked in at r231794): We want to replace as much custom x86 shuffling via intrinsics as possible because pushing the code down the generic shuffle optimization path allows for better codegen and less complexity in LLVM. This is also the LLVM sibling to the cfe D8275 patch. Differential Revision: http://reviews.llvm.org/D8276 llvm-svn: 232045
2015-03-12 23:15:19 +08:00
}
Value *UndefV = UndefValue::get(Op0->getType());
[X86,IR] Make use of the CreateShuffleVector form that takes an ArrayRef<uint32_t> to avoid the need to manually create a bunch of Constants and a ConstantVector. NFC llvm-svn: 272493
2016-06-12 09:05:59 +08:00
Rep = Builder.CreateShuffleVector(Op0, UndefV, Idxs);
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (!IsX86 && Name == "stackprotectorcheck") {
[SSP] Remove llvm.stackprotectorcheck. This is a cleanup patch for SSP support in LLVM. There is no functional change. llvm.stackprotectorcheck is not needed, because SelectionDAG isn't actually lowering it in SelectBasicBlock; rather, it adds check code in FinishBasicBlock, ignoring the position where the intrinsic is inserted (See FindSplitPointForStackProtector()). llvm-svn: 265851
2016-04-09 05:26:31 +08:00
Rep = nullptr;
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.perm.df.") ||
Name.startswith("avx512.mask.perm.di."))) {
[X86][AVX512] Autoupgrade the VPERMPD/VPERMQ intrinsics llvm-svn: 274506
2016-07-04 22:19:05 +08:00
Value *Op0 = CI->getArgOperand(0);
unsigned Imm = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
VectorType *VecTy = cast<VectorType>(CI->getType());
unsigned NumElts = VecTy->getNumElements();
SmallVector<uint32_t, 8> Idxs(NumElts);
for (unsigned i = 0; i != NumElts; ++i)
Idxs[i] = (i & ~0x3) + ((Imm >> (2 * (i & 0x3))) & 3);
Rep = Builder.CreateShuffleVector(Op0, Op0, Idxs);
if (CI->getNumArgOperands() == 4)
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("avx.vpermil.") ||
Name == "sse2.pshuf.d" ||
Name.startswith("avx512.mask.vpermil.p") ||
Name.startswith("avx512.mask.pshuf.d."))) {
[X86] Greatly simplify the llvm.x86.avx.vpermil.* auto-upgrade code. We can fully derive everything using types of the intrinsic arguments rather than writing separate loops for each intrinsic. NFC llvm-svn: 272496
2016-06-12 11:10:47 +08:00
Value *Op0 = CI->getArgOperand(0);
unsigned Imm = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
VectorType *VecTy = cast<VectorType>(CI->getType());
unsigned NumElts = VecTy->getNumElements();
[X86][AVX512] Autoupgrade the VPERMILPD/VPERMILPS intrinsics llvm-svn: 274498
2016-07-04 20:40:54 +08:00
// Calculate the size of each index in the immediate.
[X86] Greatly simplify the llvm.x86.avx.vpermil.* auto-upgrade code. We can fully derive everything using types of the intrinsic arguments rather than writing separate loops for each intrinsic. NFC llvm-svn: 272496
2016-06-12 11:10:47 +08:00
unsigned IdxSize = 64 / VecTy->getScalarSizeInBits();
unsigned IdxMask = ((1 << IdxSize) - 1);
SmallVector<uint32_t, 8> Idxs(NumElts);
// Lookup the bits for this element, wrapping around the immediate every
// 8-bits. Elements are grouped into sets of 2 or 4 elements so we need
// to offset by the first index of each group.
for (unsigned i = 0; i != NumElts; ++i)
Idxs[i] = ((Imm >> ((i * IdxSize) % 8)) & IdxMask) | (i & ~IdxMask);
[X86] Remove sse2 pshufd/pshuflw/pshufhw intrinsics and upgrade them to shufflevector. llvm-svn: 272510
2016-06-12 22:11:32 +08:00
Rep = Builder.CreateShuffleVector(Op0, Op0, Idxs);
[AVX512] Remove maksed pshufd, pshuflw, and phufhw intrinsics and autoupgrade them to selects and shufflevector. llvm-svn: 272527
2016-06-13 10:36:48 +08:00
if (CI->getNumArgOperands() == 4)
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.pshufl.w" ||
Name.startswith("avx512.mask.pshufl.w."))) {
[X86] Remove sse2 pshufd/pshuflw/pshufhw intrinsics and upgrade them to shufflevector. llvm-svn: 272510
2016-06-12 22:11:32 +08:00
Value *Op0 = CI->getArgOperand(0);
unsigned Imm = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
unsigned NumElts = CI->getType()->getVectorNumElements();
SmallVector<uint32_t, 16> Idxs(NumElts);
for (unsigned l = 0; l != NumElts; l += 8) {
for (unsigned i = 0; i != 4; ++i)
Idxs[i + l] = ((Imm >> (2 * i)) & 0x3) + l;
for (unsigned i = 4; i != 8; ++i)
Idxs[i + l] = i + l;
}
Rep = Builder.CreateShuffleVector(Op0, Op0, Idxs);
[AVX512] Remove maksed pshufd, pshuflw, and phufhw intrinsics and autoupgrade them to selects and shufflevector. llvm-svn: 272527
2016-06-13 10:36:48 +08:00
if (CI->getNumArgOperands() == 4)
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name == "sse2.pshufh.w" ||
Name.startswith("avx512.mask.pshufh.w."))) {
[X86] Remove sse2 pshufd/pshuflw/pshufhw intrinsics and upgrade them to shufflevector. llvm-svn: 272510
2016-06-12 22:11:32 +08:00
Value *Op0 = CI->getArgOperand(0);
unsigned Imm = cast<ConstantInt>(CI->getArgOperand(1))->getZExtValue();
unsigned NumElts = CI->getType()->getVectorNumElements();
SmallVector<uint32_t, 16> Idxs(NumElts);
for (unsigned l = 0; l != NumElts; l += 8) {
for (unsigned i = 0; i != 4; ++i)
Idxs[i + l] = i + l;
for (unsigned i = 0; i != 4; ++i)
Idxs[i + l + 4] = ((Imm >> (2 * i)) & 0x3) + 4 + l;
}
[X86] Greatly simplify the llvm.x86.avx.vpermil.* auto-upgrade code. We can fully derive everything using types of the intrinsic arguments rather than writing separate loops for each intrinsic. NFC llvm-svn: 272496
2016-06-12 11:10:47 +08:00
Rep = Builder.CreateShuffleVector(Op0, Op0, Idxs);
[AVX512] Remove maksed pshufd, pshuflw, and phufhw intrinsics and autoupgrade them to selects and shufflevector. llvm-svn: 272527
2016-06-13 10:36:48 +08:00
if (CI->getNumArgOperands() == 4)
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[X86] Remove masked shufpd/shufps intrinsics and autoupgrade to native vector shuffles. They were removed from clang previously but accidentally left in the backend. llvm-svn: 281300
2016-09-13 15:40:53 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.shuf.p")) {
Value *Op0 = CI->getArgOperand(0);
Value *Op1 = CI->getArgOperand(1);
unsigned Imm = cast<ConstantInt>(CI->getArgOperand(2))->getZExtValue();
unsigned NumElts = CI->getType()->getVectorNumElements();
unsigned NumLaneElts = 128/CI->getType()->getScalarSizeInBits();
unsigned HalfLaneElts = NumLaneElts / 2;
SmallVector<uint32_t, 16> Idxs(NumElts);
for (unsigned i = 0; i != NumElts; ++i) {
// Base index is the starting element of the lane.
Idxs[i] = i - (i % NumLaneElts);
// If we are half way through the lane switch to the other source.
if ((i % NumLaneElts) >= HalfLaneElts)
Idxs[i] += NumElts;
// Now select the specific element. By adding HalfLaneElts bits from
// the immediate. Wrapping around the immediate every 8-bits.
Idxs[i] += (Imm >> ((i * HalfLaneElts) % 8)) & ((1 << HalfLaneElts) - 1);
}
Rep = Builder.CreateShuffleVector(Op0, Op1, Idxs);
Rep = EmitX86Select(Builder, CI->getArgOperand(4), Rep,
CI->getArgOperand(3));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.movddup") ||
Name.startswith("avx512.mask.movshdup") ||
Name.startswith("avx512.mask.movsldup"))) {
[X86][AVX512] Autoupgrade the MOVDDUP/MOVSLDUP/MOVSHDUP intrinsics llvm-svn: 274439
2016-07-02 22:42:35 +08:00
Value *Op0 = CI->getArgOperand(0);
unsigned NumElts = CI->getType()->getVectorNumElements();
unsigned NumLaneElts = 128/CI->getType()->getScalarSizeInBits();
unsigned Offset = 0;
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
if (Name.startswith("avx512.mask.movshdup."))
[X86][AVX512] Autoupgrade the MOVDDUP/MOVSLDUP/MOVSHDUP intrinsics llvm-svn: 274439
2016-07-02 22:42:35 +08:00
Offset = 1;
SmallVector<uint32_t, 16> Idxs(NumElts);
for (unsigned l = 0; l != NumElts; l += NumLaneElts)
for (unsigned i = 0; i != NumLaneElts; i += 2) {
Idxs[i + l + 0] = i + l + Offset;
Idxs[i + l + 1] = i + l + Offset;
}
Rep = Builder.CreateShuffleVector(Op0, Op0, Idxs);
Rep = EmitX86Select(Builder, CI->getArgOperand(2), Rep,
CI->getArgOperand(1));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.punpckl") ||
Name.startswith("avx512.mask.unpckl."))) {
[AVX512] Remove masked unpack intrinsics and autoupgrade to vectorshuffle and selects. llvm-svn: 273543
2016-06-23 15:37:33 +08:00
Value *Op0 = CI->getArgOperand(0);
Value *Op1 = CI->getArgOperand(1);
int NumElts = CI->getType()->getVectorNumElements();
int NumLaneElts = 128/CI->getType()->getScalarSizeInBits();
SmallVector<uint32_t, 64> Idxs(NumElts);
for (int l = 0; l != NumElts; l += NumLaneElts)
for (int i = 0; i != NumLaneElts; ++i)
Idxs[i + l] = l + (i / 2) + NumElts * (i % 2);
Rep = Builder.CreateShuffleVector(Op0, Op1, Idxs);
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[IR,X86] Remove some intrinsic prefixes earlier in the auto-upgrade code so we can shorten the length of the comparison strings and avoid repeatedly comparing the common prefix. No functional change intended. llvm-svn: 274522
2016-07-05 04:56:38 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.punpckh") ||
Name.startswith("avx512.mask.unpckh."))) {
[AVX512] Remove masked unpack intrinsics and autoupgrade to vectorshuffle and selects. llvm-svn: 273543
2016-06-23 15:37:33 +08:00
Value *Op0 = CI->getArgOperand(0);
Value *Op1 = CI->getArgOperand(1);
int NumElts = CI->getType()->getVectorNumElements();
int NumLaneElts = 128/CI->getType()->getScalarSizeInBits();
SmallVector<uint32_t, 64> Idxs(NumElts);
for (int l = 0; l != NumElts; l += NumLaneElts)
for (int i = 0; i != NumLaneElts; ++i)
Idxs[i + l] = (NumLaneElts / 2) + l + (i / 2) + NumElts * (i % 2);
Rep = Builder.CreateShuffleVector(Op0, Op1, Idxs);
[AVX512] Remove masked logic op intrinsics and autoupgrade them to native IR. llvm-svn: 275155
2016-07-12 13:27:53 +08:00
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.pand.")) {
Rep = Builder.CreateAnd(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.pandn.")) {
Rep = Builder.CreateAnd(Builder.CreateNot(CI->getArgOperand(0)),
CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.por.")) {
Rep = Builder.CreateOr(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.pxor.")) {
Rep = Builder.CreateXor(CI->getArgOperand(0), CI->getArgOperand(1));
[AVX512] Remove masked unpack intrinsics and autoupgrade to vectorshuffle and selects. llvm-svn: 273543
2016-06-23 15:37:33 +08:00
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[AVX-512] Remove floating point logical operation instrinsics and replace them with native IR. llvm-svn: 280466
2016-09-02 13:29:17 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.and.")) {
VectorType *FTy = cast<VectorType>(CI->getType());
VectorType *ITy = VectorType::getInteger(FTy);
Rep = Builder.CreateAnd(Builder.CreateBitCast(CI->getArgOperand(0), ITy),
Builder.CreateBitCast(CI->getArgOperand(1), ITy));
Rep = Builder.CreateBitCast(Rep, FTy);
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.andn.")) {
VectorType *FTy = cast<VectorType>(CI->getType());
VectorType *ITy = VectorType::getInteger(FTy);
Rep = Builder.CreateNot(Builder.CreateBitCast(CI->getArgOperand(0), ITy));
Rep = Builder.CreateAnd(Rep,
Builder.CreateBitCast(CI->getArgOperand(1), ITy));
Rep = Builder.CreateBitCast(Rep, FTy);
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.or.")) {
VectorType *FTy = cast<VectorType>(CI->getType());
VectorType *ITy = VectorType::getInteger(FTy);
Rep = Builder.CreateOr(Builder.CreateBitCast(CI->getArgOperand(0), ITy),
Builder.CreateBitCast(CI->getArgOperand(1), ITy));
Rep = Builder.CreateBitCast(Rep, FTy);
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.xor.")) {
VectorType *FTy = cast<VectorType>(CI->getType());
VectorType *ITy = VectorType::getInteger(FTy);
Rep = Builder.CreateXor(Builder.CreateBitCast(CI->getArgOperand(0), ITy),
Builder.CreateBitCast(CI->getArgOperand(1), ITy));
Rep = Builder.CreateBitCast(Rep, FTy);
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[AVX-512] Remove masked integer add/sub/mull intrinsics and upgrade to native IR. llvm-svn: 280611
2016-09-04 10:09:53 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.padd.")) {
Rep = Builder.CreateAdd(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.psub.")) {
Rep = Builder.CreateSub(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
} else if (IsX86 && Name.startswith("avx512.mask.pmull.")) {
Rep = Builder.CreateMul(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[AVX-512] Use an equality compare instead of StringRef::startswith in a few places in auto upgrade that were looking for the complete intrinsic name anyway. llvm-svn: 286033
2016-11-05 13:35:23 +08:00
} else if (IsX86 && (Name.startswith("avx512.mask.add.p"))) {
[AVX-512] Remove 128-bit and 256-bit masked floating point add/sub/mul/div intrinsics and upgrade to native IR. llvm-svn: 280633
2016-09-05 02:13:33 +08:00
Rep = Builder.CreateFAdd(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[X86] Use StringRef::startswith to reduce a few compares in the intrinsic autoupgrade code. llvm-svn: 286090
2016-11-07 08:13:42 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.div.p")) {
[AVX-512] Remove 128-bit and 256-bit masked floating point add/sub/mul/div intrinsics and upgrade to native IR. llvm-svn: 280633
2016-09-05 02:13:33 +08:00
Rep = Builder.CreateFDiv(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[X86] Use StringRef::startswith to reduce a few compares in the intrinsic autoupgrade code. llvm-svn: 286090
2016-11-07 08:13:42 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.mul.p")) {
[AVX-512] Remove 128-bit and 256-bit masked floating point add/sub/mul/div intrinsics and upgrade to native IR. llvm-svn: 280633
2016-09-05 02:13:33 +08:00
Rep = Builder.CreateFMul(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[X86] Use StringRef::startswith to reduce a few compares in the intrinsic autoupgrade code. llvm-svn: 286090
2016-11-07 08:13:42 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.sub.p")) {
[AVX-512] Remove 128-bit and 256-bit masked floating point add/sub/mul/div intrinsics and upgrade to native IR. llvm-svn: 280633
2016-09-05 02:13:33 +08:00
Rep = Builder.CreateFSub(CI->getArgOperand(0), CI->getArgOperand(1));
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[AVX-512] Remove 128/256 masked pshufb intrinsics. Autoupgrade them to legacy intrinsics and a select. llvm-svn: 286089
2016-11-07 08:13:39 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.pshuf.b.")) {
VectorType *VecTy = cast<VectorType>(CI->getType());
Intrinsic::ID IID;
if (VecTy->getPrimitiveSizeInBits() == 128)
IID = Intrinsic::x86_ssse3_pshuf_b_128;
else if (VecTy->getPrimitiveSizeInBits() == 256)
IID = Intrinsic::x86_avx2_pshuf_b;
else
llvm_unreachable("Unexpected intrinsic");
Rep = Builder.CreateCall(Intrinsic::getDeclaration(F->getParent(), IID),
{ CI->getArgOperand(0), CI->getArgOperand(1) });
Rep = EmitX86Select(Builder, CI->getArgOperand(3), Rep,
CI->getArgOperand(2));
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.psll")) {
bool IsImmediate = Name[16] == 'i' ||
(Name.size() > 18 && Name[18] == 'i');
bool IsVariable = Name[16] == 'v';
char Size = Name[16] == '.' ? Name[17] :
Name[17] == '.' ? Name[18] :
[AVX-512] Replace masked 16-bit element variable shift intrinsics with new unmasked versions and selects. The same thing was done to 32-bit and 64-bit element sizes previously. This will allow us to support these shuffls in InstCombineCalls along with the other variable shift intrinsics. llvm-svn: 287312
2016-11-18 13:04:44 +08:00
Name[18] == '.' ? Name[19] :
Name[20];
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
Intrinsic::ID IID;
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
if (IsVariable && Name[17] != '.') {
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
if (Size == 'd' && Name[17] == '2') // avx512.mask.psllv2.di
IID = Intrinsic::x86_avx2_psllv_q;
else if (Size == 'd' && Name[17] == '4') // avx512.mask.psllv4.di
IID = Intrinsic::x86_avx2_psllv_q_256;
else if (Size == 's' && Name[17] == '4') // avx512.mask.psllv4.si
IID = Intrinsic::x86_avx2_psllv_d;
else if (Size == 's' && Name[17] == '8') // avx512.mask.psllv8.si
IID = Intrinsic::x86_avx2_psllv_d_256;
[AVX-512] Replace masked 16-bit element variable shift intrinsics with new unmasked versions and selects. The same thing was done to 32-bit and 64-bit element sizes previously. This will allow us to support these shuffls in InstCombineCalls along with the other variable shift intrinsics. llvm-svn: 287312
2016-11-18 13:04:44 +08:00
else if (Size == 'h' && Name[17] == '8') // avx512.mask.psllv8.hi
IID = Intrinsic::x86_avx512_psllv_w_128;
else if (Size == 'h' && Name[17] == '1') // avx512.mask.psllv16.hi
IID = Intrinsic::x86_avx512_psllv_w_256;
else if (Name[17] == '3' && Name[18] == '2') // avx512.mask.psllv32hi
IID = Intrinsic::x86_avx512_psllv_w_512;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else
llvm_unreachable("Unexpected size");
} else if (Name.endswith(".128")) {
if (Size == 'd') // avx512.mask.psll.d.128, avx512.mask.psll.di.128
IID = IsImmediate ? Intrinsic::x86_sse2_pslli_d
: Intrinsic::x86_sse2_psll_d;
else if (Size == 'q') // avx512.mask.psll.q.128, avx512.mask.psll.qi.128
IID = IsImmediate ? Intrinsic::x86_sse2_pslli_q
: Intrinsic::x86_sse2_psll_q;
else if (Size == 'w') // avx512.mask.psll.w.128, avx512.mask.psll.wi.128
IID = IsImmediate ? Intrinsic::x86_sse2_pslli_w
: Intrinsic::x86_sse2_psll_w;
else
llvm_unreachable("Unexpected size");
} else if (Name.endswith(".256")) {
if (Size == 'd') // avx512.mask.psll.d.256, avx512.mask.psll.di.256
IID = IsImmediate ? Intrinsic::x86_avx2_pslli_d
: Intrinsic::x86_avx2_psll_d;
else if (Size == 'q') // avx512.mask.psll.q.256, avx512.mask.psll.qi.256
IID = IsImmediate ? Intrinsic::x86_avx2_pslli_q
: Intrinsic::x86_avx2_psll_q;
else if (Size == 'w') // avx512.mask.psll.w.256, avx512.mask.psll.wi.256
IID = IsImmediate ? Intrinsic::x86_avx2_pslli_w
: Intrinsic::x86_avx2_psll_w;
else
llvm_unreachable("Unexpected size");
} else {
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
if (Size == 'd') // psll.di.512, pslli.d, psll.d, psllv.d.512
IID = IsImmediate ? Intrinsic::x86_avx512_pslli_d_512 :
IsVariable ? Intrinsic::x86_avx512_psllv_d_512 :
Intrinsic::x86_avx512_psll_d_512;
else if (Size == 'q') // psll.qi.512, pslli.q, psll.q, psllv.q.512
IID = IsImmediate ? Intrinsic::x86_avx512_pslli_q_512 :
IsVariable ? Intrinsic::x86_avx512_psllv_q_512 :
Intrinsic::x86_avx512_psll_q_512;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else if (Size == 'w') // psll.wi.512, pslli.w, psll.w
IID = IsImmediate ? Intrinsic::x86_avx512_pslli_w_512
: Intrinsic::x86_avx512_psll_w_512;
else
llvm_unreachable("Unexpected size");
}
Rep = UpgradeX86MaskedShift(Builder, *CI, IID);
} else if (IsX86 && Name.startswith("avx512.mask.psrl")) {
bool IsImmediate = Name[16] == 'i' ||
(Name.size() > 18 && Name[18] == 'i');
bool IsVariable = Name[16] == 'v';
char Size = Name[16] == '.' ? Name[17] :
Name[17] == '.' ? Name[18] :
[AVX-512] Replace masked 16-bit element variable shift intrinsics with new unmasked versions and selects. The same thing was done to 32-bit and 64-bit element sizes previously. This will allow us to support these shuffls in InstCombineCalls along with the other variable shift intrinsics. llvm-svn: 287312
2016-11-18 13:04:44 +08:00
Name[18] == '.' ? Name[19] :
Name[20];
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
Intrinsic::ID IID;
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
if (IsVariable && Name[17] != '.') {
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
if (Size == 'd' && Name[17] == '2') // avx512.mask.psrlv2.di
IID = Intrinsic::x86_avx2_psrlv_q;
else if (Size == 'd' && Name[17] == '4') // avx512.mask.psrlv4.di
IID = Intrinsic::x86_avx2_psrlv_q_256;
else if (Size == 's' && Name[17] == '4') // avx512.mask.psrlv4.si
IID = Intrinsic::x86_avx2_psrlv_d;
else if (Size == 's' && Name[17] == '8') // avx512.mask.psrlv8.si
IID = Intrinsic::x86_avx2_psrlv_d_256;
[AVX-512] Replace masked 16-bit element variable shift intrinsics with new unmasked versions and selects. The same thing was done to 32-bit and 64-bit element sizes previously. This will allow us to support these shuffls in InstCombineCalls along with the other variable shift intrinsics. llvm-svn: 287312
2016-11-18 13:04:44 +08:00
else if (Size == 'h' && Name[17] == '8') // avx512.mask.psrlv8.hi
IID = Intrinsic::x86_avx512_psrlv_w_128;
else if (Size == 'h' && Name[17] == '1') // avx512.mask.psrlv16.hi
IID = Intrinsic::x86_avx512_psrlv_w_256;
else if (Name[17] == '3' && Name[18] == '2') // avx512.mask.psrlv32hi
IID = Intrinsic::x86_avx512_psrlv_w_512;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else
llvm_unreachable("Unexpected size");
} else if (Name.endswith(".128")) {
if (Size == 'd') // avx512.mask.psrl.d.128, avx512.mask.psrl.di.128
IID = IsImmediate ? Intrinsic::x86_sse2_psrli_d
: Intrinsic::x86_sse2_psrl_d;
else if (Size == 'q') // avx512.mask.psrl.q.128, avx512.mask.psrl.qi.128
IID = IsImmediate ? Intrinsic::x86_sse2_psrli_q
: Intrinsic::x86_sse2_psrl_q;
else if (Size == 'w') // avx512.mask.psrl.w.128, avx512.mask.psrl.wi.128
IID = IsImmediate ? Intrinsic::x86_sse2_psrli_w
: Intrinsic::x86_sse2_psrl_w;
else
llvm_unreachable("Unexpected size");
} else if (Name.endswith(".256")) {
if (Size == 'd') // avx512.mask.psrl.d.256, avx512.mask.psrl.di.256
IID = IsImmediate ? Intrinsic::x86_avx2_psrli_d
: Intrinsic::x86_avx2_psrl_d;
else if (Size == 'q') // avx512.mask.psrl.q.256, avx512.mask.psrl.qi.256
IID = IsImmediate ? Intrinsic::x86_avx2_psrli_q
: Intrinsic::x86_avx2_psrl_q;
else if (Size == 'w') // avx512.mask.psrl.w.256, avx512.mask.psrl.wi.256
IID = IsImmediate ? Intrinsic::x86_avx2_psrli_w
: Intrinsic::x86_avx2_psrl_w;
else
llvm_unreachable("Unexpected size");
} else {
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
if (Size == 'd') // psrl.di.512, psrli.d, psrl.d, psrl.d.512
IID = IsImmediate ? Intrinsic::x86_avx512_psrli_d_512 :
IsVariable ? Intrinsic::x86_avx512_psrlv_d_512 :
Intrinsic::x86_avx512_psrl_d_512;
else if (Size == 'q') // psrl.qi.512, psrli.q, psrl.q, psrl.q.512
IID = IsImmediate ? Intrinsic::x86_avx512_psrli_q_512 :
IsVariable ? Intrinsic::x86_avx512_psrlv_q_512 :
Intrinsic::x86_avx512_psrl_q_512;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else if (Size == 'w') // psrl.wi.512, psrli.w, psrl.w)
IID = IsImmediate ? Intrinsic::x86_avx512_psrli_w_512
: Intrinsic::x86_avx512_psrl_w_512;
else
llvm_unreachable("Unexpected size");
}
Rep = UpgradeX86MaskedShift(Builder, *CI, IID);
} else if (IsX86 && Name.startswith("avx512.mask.psra")) {
bool IsImmediate = Name[16] == 'i' ||
(Name.size() > 18 && Name[18] == 'i');
bool IsVariable = Name[16] == 'v';
char Size = Name[16] == '.' ? Name[17] :
Name[17] == '.' ? Name[18] :
[AVX-512] Replace masked 16-bit element variable shift intrinsics with new unmasked versions and selects. The same thing was done to 32-bit and 64-bit element sizes previously. This will allow us to support these shuffls in InstCombineCalls along with the other variable shift intrinsics. llvm-svn: 287312
2016-11-18 13:04:44 +08:00
Name[18] == '.' ? Name[19] :
Name[20];
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
Intrinsic::ID IID;
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
if (IsVariable && Name[17] != '.') {
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
if (Size == 's' && Name[17] == '4') // avx512.mask.psrav4.si
IID = Intrinsic::x86_avx2_psrav_d;
else if (Size == 's' && Name[17] == '8') // avx512.mask.psrav8.si
IID = Intrinsic::x86_avx2_psrav_d_256;
[AVX-512] Replace masked 16-bit element variable shift intrinsics with new unmasked versions and selects. The same thing was done to 32-bit and 64-bit element sizes previously. This will allow us to support these shuffls in InstCombineCalls along with the other variable shift intrinsics. llvm-svn: 287312
2016-11-18 13:04:44 +08:00
else if (Size == 'h' && Name[17] == '8') // avx512.mask.psrav8.hi
IID = Intrinsic::x86_avx512_psrav_w_128;
else if (Size == 'h' && Name[17] == '1') // avx512.mask.psrav16.hi
IID = Intrinsic::x86_avx512_psrav_w_256;
else if (Name[17] == '3' && Name[18] == '2') // avx512.mask.psrav32hi
IID = Intrinsic::x86_avx512_psrav_w_512;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else
llvm_unreachable("Unexpected size");
} else if (Name.endswith(".128")) {
if (Size == 'd') // avx512.mask.psra.d.128, avx512.mask.psra.di.128
IID = IsImmediate ? Intrinsic::x86_sse2_psrai_d
: Intrinsic::x86_sse2_psra_d;
else if (Size == 'q') // avx512.mask.psra.q.128, avx512.mask.psra.qi.128
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
IID = IsImmediate ? Intrinsic::x86_avx512_psrai_q_128 :
IsVariable ? Intrinsic::x86_avx512_psrav_q_128 :
Intrinsic::x86_avx512_psra_q_128;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else if (Size == 'w') // avx512.mask.psra.w.128, avx512.mask.psra.wi.128
IID = IsImmediate ? Intrinsic::x86_sse2_psrai_w
: Intrinsic::x86_sse2_psra_w;
else
llvm_unreachable("Unexpected size");
} else if (Name.endswith(".256")) {
if (Size == 'd') // avx512.mask.psra.d.256, avx512.mask.psra.di.256
IID = IsImmediate ? Intrinsic::x86_avx2_psrai_d
: Intrinsic::x86_avx2_psra_d;
else if (Size == 'q') // avx512.mask.psra.q.256, avx512.mask.psra.qi.256
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
IID = IsImmediate ? Intrinsic::x86_avx512_psrai_q_256 :
IsVariable ? Intrinsic::x86_avx512_psrav_q_256 :
Intrinsic::x86_avx512_psra_q_256;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else if (Size == 'w') // avx512.mask.psra.w.256, avx512.mask.psra.wi.256
IID = IsImmediate ? Intrinsic::x86_avx2_psrai_w
: Intrinsic::x86_avx2_psra_w;
else
llvm_unreachable("Unexpected size");
} else {
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
if (Size == 'd') // psra.di.512, psrai.d, psra.d, psrav.d.512
IID = IsImmediate ? Intrinsic::x86_avx512_psrai_d_512 :
IsVariable ? Intrinsic::x86_avx512_psrav_d_512 :
Intrinsic::x86_avx512_psra_d_512;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else if (Size == 'q') // psra.qi.512, psrai.q, psra.q
[AVX-512] Remove and autoupgrade masked dword/qword variable shift intrinsics to the new unmasked versions and selects. llvm-svn: 286786
2016-11-14 09:53:22 +08:00
IID = IsImmediate ? Intrinsic::x86_avx512_psrai_q_512 :
IsVariable ? Intrinsic::x86_avx512_psrav_q_512 :
Intrinsic::x86_avx512_psra_q_512;
[X86][IR] Reduce the number of full string comparisons in the code that autoupgrades masked shift intrinsics. llvm-svn: 286768
2016-11-14 03:09:56 +08:00
else if (Size == 'w') // psra.wi.512, psrai.w, psra.w
IID = IsImmediate ? Intrinsic::x86_avx512_psrai_w_512
: Intrinsic::x86_avx512_psra_w_512;
else
llvm_unreachable("Unexpected size");
}
Rep = UpgradeX86MaskedShift(Builder, *CI, IID);
[X86][AVX512] Removing llvm x86 intrinsics for _mm_mask_move_{ss|sd} intrinsics. Differential Revision: https://reviews.llvm.org/D26128 llvm-svn: 287087
2016-11-16 17:00:28 +08:00
} else if (IsX86 && Name.startswith("avx512.mask.move.s")) {
Rep = upgradeMaskedMove(Builder, *CI);
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
} else {
[X86] Greatly simplify the llvm.x86.avx.vpermil.* auto-upgrade code. We can fully derive everything using types of the intrinsic arguments rather than writing separate loops for each intrinsic. NFC llvm-svn: 272496
2016-06-12 11:10:47 +08:00
llvm_unreachable("Unknown function for CallInst upgrade.");
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
}
[SSP] Remove llvm.stackprotectorcheck. This is a cleanup patch for SSP support in LLVM. There is no functional change. llvm.stackprotectorcheck is not needed, because SelectionDAG isn't actually lowering it in SelectBasicBlock; rather, it adds check code in FinishBasicBlock, ignoring the position where the intrinsic is inserted (See FindSplitPointForStackProtector()). llvm-svn: 265851
2016-04-09 05:26:31 +08:00
if (Rep)
CI->replaceAllUsesWith(Rep);
Add auto upgrade support for x86 pcmpgt/pcmpeq intrinics removed in r149367. llvm-svn: 149678
2012-02-03 14:10:55 +08:00
CI->eraseFromParent();
return;
}
Trying to fix the Hexagon and debian-fast bots arm32_neon_vcnt_upgrade.ll test. llvm-svn: 233558
2015-03-31 00:10:39 +08:00
std::string Name = CI->getName();
Move the complex address expression out of DIVariable and into an extra argument of the llvm.dbg.declare/llvm.dbg.value intrinsics. Previously, DIVariable was a variable-length field that has an optional reference to a Metadata array consisting of a variable number of complex address expressions. In the case of OpPiece expressions this is wasting a lot of storage in IR, because when an aggregate type is, e.g., SROA'd into all of its n individual members, the IR will contain n copies of the DIVariable, all alike, only differing in the complex address reference at the end. By making the complex address into an extra argument of the dbg.value/dbg.declare intrinsics, all of the pieces can reference the same variable and the complex address expressions can be uniqued across the CU, too. Down the road, this will allow us to move other flags, such as "indirection" out of the DIVariable, too. The new intrinsics look like this: declare void @llvm.dbg.declare(metadata %storage, metadata %var, metadata %expr) declare void @llvm.dbg.value(metadata %storage, i64 %offset, metadata %var, metadata %expr) This patch adds a new LLVM-local tag to DIExpressions, so we can detect and pretty-print DIExpression metadata nodes. What this patch doesn't do: This patch does not touch the "Indirect" field in DIVariable; but moving that into the expression would be a natural next step. http://reviews.llvm.org/D4919 rdar://problem/17994491 Thanks to dblaikie and dexonsmith for reviewing this patch! Note: I accidentally committed a bogus older version of this patch previously. llvm-svn: 218787
2014-10-02 02:55:02 +08:00
if (!Name.empty())
CI->setName(Name + ".old");
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
switch (NewFn->getIntrinsicID()) {
default:
Eli managed to kill off llvm.membarrier in llvm 3.0 also, this means that mainline needs no autoupgrade logic for intrinsics yet, woohoo! llvm-svn: 145178
2011-11-27 16:42:07 +08:00
llvm_unreachable("Unknown function for CallInst upgrade.");
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
[ARM][NEON] Use address space in vld([1234]|[234]lane) and vst([1234]|[234]lane) instructions This commit changes the interface of the vld[1234], vld[234]lane, and vst[1234], vst[234]lane ARM neon intrinsics and associates an address space with the pointer that these intrinsics take. This changes, e.g., <2 x i32> @llvm.arm.neon.vld1.v2i32(i8*, i32) to <2 x i32> @llvm.arm.neon.vld1.v2i32.p0i8(i8*, i32) This change ensures that address spaces are fully taken into account in the ARM target during lowering of interleaved loads and stores. Differential Revision: http://reviews.llvm.org/D12985 llvm-svn: 248887
2015-09-30 18:56:37 +08:00
case Intrinsic::arm_neon_vld1:
case Intrinsic::arm_neon_vld2:
case Intrinsic::arm_neon_vld3:
case Intrinsic::arm_neon_vld4:
case Intrinsic::arm_neon_vld2lane:
case Intrinsic::arm_neon_vld3lane:
case Intrinsic::arm_neon_vld4lane:
case Intrinsic::arm_neon_vst1:
case Intrinsic::arm_neon_vst2:
case Intrinsic::arm_neon_vst3:
case Intrinsic::arm_neon_vst4:
case Intrinsic::arm_neon_vst2lane:
case Intrinsic::arm_neon_vst3lane:
case Intrinsic::arm_neon_vst4lane: {
SmallVector<Value *, 4> Args(CI->arg_operands().begin(),
CI->arg_operands().end());
CI->replaceAllUsesWith(Builder.CreateCall(NewFn, Args));
CI->eraseFromParent();
return;
}
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
case Intrinsic::ctlz:
revert my previous patches that introduced an additional parameter to the objectsize intrinsic. After a lot of discussion, we realized it's not the best option for run-time bounds checking llvm-svn: 157255
2012-05-22 23:25:31 +08:00
case Intrinsic::cttz:
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
assert(CI->getNumArgOperands() == 1 &&
"Mismatch between function args and call args");
Simplify IRBuilder::CreateCall* by using ArrayRef+initializer_list/braced init only llvm-svn: 237624
2015-05-19 06:13:54 +08:00
CI->replaceAllUsesWith(Builder.CreateCall(
NewFn, {CI->getArgOperand(0), Builder.getFalse()}, Name));
Switch llvm.cttz and llvm.ctlz to accept a second i1 parameter which indicates whether the intrinsic has a defined result for a first argument equal to zero. This will eventually allow these intrinsics to accurately model the semantics of GCC's __builtin_ctz and __builtin_clz and the X86 instructions (prior to AVX) which implement them. This patch merely sets the stage by extending the signature of these intrinsics and establishing auto-upgrade logic so that the old spelling still works both in IR and in bitcode. The upgrade logic preserves the existing (inefficient) semantics. This patch should not change any behavior. CodeGen isn't updated because it can use the existing semantics regardless of the flag's value. Note that this will be followed by API updates to Clang and DragonEgg. Reviewed by Nick Lewycky! llvm-svn: 146357
2011-12-12 12:26:04 +08:00
CI->eraseFromParent();
return;
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
Change objectsize intrinsic to accept different address spaces. Bitcasting everything to i8* won't work. Autoupgrade the old intrinsic declarations to use the new mangling. llvm-svn: 192117
2013-10-08 02:06:48 +08:00
case Intrinsic::objectsize:
Simplify IRBuilder::CreateCall* by using ArrayRef+initializer_list/braced init only llvm-svn: 237624
2015-05-19 06:13:54 +08:00
CI->replaceAllUsesWith(Builder.CreateCall(
NewFn, {CI->getArgOperand(0), CI->getArgOperand(1)}, Name));
Change objectsize intrinsic to accept different address spaces. Bitcasting everything to i8* won't work. Autoupgrade the old intrinsic declarations to use the new mangling. llvm-svn: 192117
2013-10-08 02:06:48 +08:00
CI->eraseFromParent();
return;
More replacing of target-dependent intrinsics with target-indepdent intrinsics. The second instruction(s) to be handled are the vector versions of count set bits (ctpop). The changes here are to clang so that it generates a target independent vector ctpop when it sees an ARM dependent vector bits set count. The changes in llvm are to match the target independent vector ctpop and in VMCore/AutoUpgrade.cpp to update any existing bc files containing ARM dependent vector pop counts with target-independent ctpops. There are also changes to an existing test case in llvm for ARM vector count instructions and to a test for the bitcode upgrade. <rdar://problem/11892519> There is deliberately no test for the change to clang, as so far as I know, no consensus has been reached regarding how to test neon instructions in clang; q.v. <rdar://problem/8762292> llvm-svn: 160410
2012-07-18 08:02:16 +08:00
case Intrinsic::ctpop: {
Simplify IRBuilder::CreateCall* by using ArrayRef+initializer_list/braced init only llvm-svn: 237624
2015-05-19 06:13:54 +08:00
CI->replaceAllUsesWith(Builder.CreateCall(NewFn, {CI->getArgOperand(0)}));
More replacing of target-dependent intrinsics with target-indepdent intrinsics. The second instruction(s) to be handled are the vector versions of count set bits (ctpop). The changes here are to clang so that it generates a target independent vector ctpop when it sees an ARM dependent vector bits set count. The changes in llvm are to match the target independent vector ctpop and in VMCore/AutoUpgrade.cpp to update any existing bc files containing ARM dependent vector pop counts with target-independent ctpops. There are also changes to an existing test case in llvm for ARM vector count instructions and to a test for the bitcode upgrade. <rdar://problem/11892519> There is deliberately no test for the change to clang, as so far as I know, no consensus has been reached regarding how to test neon instructions in clang; q.v. <rdar://problem/8762292> llvm-svn: 160410
2012-07-18 08:02:16 +08:00
CI->eraseFromParent();
return;
}
This is one of the first steps at moving to replace target-dependent intrinsics with target-indepdent intrinsics. The first instruction(s) to be handled are the vector versions of count leading zeros (ctlz). The changes here are to clang so that it generates a target independent vector ctlz when it sees an ARM dependent vector ctlz. The changes in llvm are to match the target independent vector ctlz and in VMCore/AutoUpgrade.cpp to update any existing bc files containing ARM dependent vector ctlzs with target-independent ctlzs. There are also changes to an existing test case in llvm for ARM vector count instructions and a new test for the bitcode upgrade. <rdar://problem/11831778> There is deliberately no test for the change to clang, as so far as I know, no consensus has been reached regarding how to test neon instructions in clang; q.v. <rdar://problem/8762292> llvm-svn: 160200
2012-07-14 07:25:25 +08:00
Fix intrinsics for XOP frczss/sd instructions. These instructions only take one source register and zero the upper bits of the destination rather than preserving them. llvm-svn: 158396
2012-06-13 15:18:53 +08:00
case Intrinsic::x86_xop_vfrcz_ss:
case Intrinsic::x86_xop_vfrcz_sd:
Simplify IRBuilder::CreateCall* by using ArrayRef+initializer_list/braced init only llvm-svn: 237624
2015-05-19 06:13:54 +08:00
CI->replaceAllUsesWith(
Builder.CreateCall(NewFn, {CI->getArgOperand(1)}, Name));
Fix intrinsics for XOP frczss/sd instructions. These instructions only take one source register and zero the upper bits of the destination rather than preserving them. llvm-svn: 158396
2012-06-13 15:18:53 +08:00
CI->eraseFromParent();
return;
[X86][XOP] Support for VPERMIL2PD/VPERMIL2PS 2-input shuffle instructions This patch begins adding support for lowering to the XOP VPERMIL2PD/VPERMIL2PS shuffle instructions - adding the X86ISD::VPERMIL2 opcode and cleaning up the usage. The internal llvm intrinsics were assuming the shuffle mask operand was the same type as the float/double input operands (I guess to simplify the intrinsic definitions in X86InstrXOP.td to a single value type). These needed changing to integer types (matching the clang builtin and the AMD intrinsics definitions), an auto upgrade path is added to convert old calls. Mask decoding/target shuffle support will be added in future patches. Differential Revision: http://reviews.llvm.org/D20049 llvm-svn: 271633
2016-06-03 16:06:03 +08:00
case Intrinsic::x86_xop_vpermil2pd:
case Intrinsic::x86_xop_vpermil2ps:
case Intrinsic::x86_xop_vpermil2pd_256:
case Intrinsic::x86_xop_vpermil2ps_256: {
SmallVector<Value *, 4> Args(CI->arg_operands().begin(),
CI->arg_operands().end());
VectorType *FltIdxTy = cast<VectorType>(Args[2]->getType());
VectorType *IntIdxTy = VectorType::getInteger(FltIdxTy);
Args[2] = Builder.CreateBitCast(Args[2], IntIdxTy);
CI->replaceAllUsesWith(Builder.CreateCall(NewFn, Args, Name));
CI->eraseFromParent();
return;
}
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
case Intrinsic::x86_sse41_ptestc:
case Intrinsic::x86_sse41_ptestz:
Fix intrinsics for XOP frczss/sd instructions. These instructions only take one source register and zero the upper bits of the destination rather than preserving them. llvm-svn: 158396
2012-06-13 15:18:53 +08:00
case Intrinsic::x86_sse41_ptestnzc: {
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
// The arguments for these intrinsics used to be v4f32, and changed
// to v2i64. This is purely a nop, since those are bitwise intrinsics.
// So, the only thing required is a bitcast for both arguments.
// First, check the arguments have the old type.
Value *Arg0 = CI->getArgOperand(0);
if (Arg0->getType() != VectorType::get(Type::getFloatTy(C), 4))
return;
// Old intrinsic, add bitcasts
Value *Arg1 = CI->getArgOperand(1);
[opaque pointer type] Verifier/AutoUpgrade: Remove a few uses of PointerType::getElementType llvm-svn: 235777
2015-04-25 05:16:07 +08:00
Type *NewVecTy = VectorType::get(Type::getInt64Ty(C), 2);
Value *BC0 = Builder.CreateBitCast(Arg0, NewVecTy, "cast");
Value *BC1 = Builder.CreateBitCast(Arg1, NewVecTy, "cast");
Simplify IRBuilder::CreateCall* by using ArrayRef+initializer_list/braced init only llvm-svn: 237624
2015-05-19 06:13:54 +08:00
CallInst *NewCall = Builder.CreateCall(NewFn, {BC0, BC1}, Name);
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
CI->replaceAllUsesWith(NewCall);
[x86] Fix a pretty horrible bug and inconsistency in the x86 asm parsing (and latent bug in the instruction definitions). This is effectively a revert of r136287 which tried to address a specific and narrow case of immediate operands failing to be accepted by x86 instructions with a pretty heavy hammer: it introduced a new kind of operand that behaved differently. All of that is removed with this commit, but the test cases are both preserved and enhanced. The core problem that r136287 and this commit are trying to handle is that gas accepts both of the following instructions: insertps $192, %xmm0, %xmm1 insertps $-64, %xmm0, %xmm1 These will encode to the same byte sequence, with the immediate occupying an 8-bit entry. The first form was fixed by r136287 but that broke the prior handling of the second form! =[ Ironically, we would still emit the second form in some cases and then be unable to re-assemble the output. The reason why the first instruction failed to be handled is because prior to r136287 the operands ere marked 'i32i8imm' which forces them to be sign-extenable. Clearly, that won't work for 192 in a single byte. However, making thim zero-extended or "unsigned" doesn't really address the core issue either because it breaks negative immediates. The correct fix is to make these operands 'i8imm' reflecting that they can be either signed or unsigned but must be 8-bit immediates. This patch backs out r136287 and then changes those places as well as some others to use 'i8imm' rather than one of the extended variants. Naturally, this broke something else. The custom DAG nodes had to be updated to have a much more accurate type constraint of an i8 node, and a bunch of Pat immediates needed to be specified as i8 values. The fallout didn't end there though. We also then ceased to be able to match the instruction-specific intrinsics to the instructions so modified. Digging, this is because they too used i32 rather than i8 in their signature. So I've also switched those intrinsics to i8 arguments in line with the instructions. In order to make the intrinsic adjustments of course, I also had to add auto upgrading for the intrinsics. I suspect that the intrinsic argument types may have led everything down this rabbit hole. Pretty happy with the result. llvm-svn: 217310
2014-09-06 18:00:01 +08:00
CI->eraseFromParent();
return;
}
case Intrinsic::x86_sse41_insertps:
case Intrinsic::x86_sse41_dppd:
case Intrinsic::x86_sse41_dpps:
case Intrinsic::x86_sse41_mpsadbw:
case Intrinsic::x86_avx_dp_ps_256:
case Intrinsic::x86_avx2_mpsadbw: {
// Need to truncate the last argument from i32 to i8 -- this argument models
// an inherently 8-bit immediate operand to these x86 instructions.
SmallVector<Value *, 4> Args(CI->arg_operands().begin(),
CI->arg_operands().end());
// Replace the last argument with a trunc.
Args.back() = Builder.CreateTrunc(Args.back(), Type::getInt8Ty(C), "trunc");
CallInst *NewCall = Builder.CreateCall(NewFn, Args);
CI->replaceAllUsesWith(NewCall);
Add AutoUpgrade support for the SSE4 ptest intrinsics. Patch by Michael Kuperstein. llvm-svn: 158295
2012-06-11 02:42:51 +08:00
CI->eraseFromParent();
return;
Bring back int_x86_sse2_movl_dq intrinsic for backward compatibility. Make sure it's auto-upgraded to a shufflevector instruction. llvm-svn: 45131
2007-12-18 06:33:23 +08:00
}
[AArch64] [ARM] Make a target-independent llvm.thread.pointer intrinsic. Both AArch64 and ARM support llvm.<arch>.thread.pointer intrinsics that just return the thread pointer. I have a pending patch that does the same for SystemZ (D19054), and there are many more targets that could benefit from one. This patch merges the ARM and AArch64 intrinsics into a single target independent one that will also be used by subsequent targets. Differential Revision: http://reviews.llvm.org/D19098 llvm-svn: 266818
2016-04-20 04:51:05 +08:00
case Intrinsic::thread_pointer: {
CI->replaceAllUsesWith(Builder.CreateCall(NewFn, {}));
CI->eraseFromParent();
return;
}
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
Revert "Revert "Invariant start/end intrinsics overloaded for address space"" This reverts commit 32fc6488e48eafc0ca1bac1bd9cbf0008224d530. llvm-svn: 278609
2016-08-14 07:31:24 +08:00
case Intrinsic::invariant_start:
case Intrinsic::invariant_end:
Support arbitrary addrspace pointers in masked load/store intrinsics This is a resubmittion of 263158 change after fixing the existing problem with intrinsics mangling (see LTO and intrinsics mangling llvm-dev thread for details). This patch fixes the problem which occurs when loop-vectorize tries to use @llvm.masked.load/store intrinsic for a non-default addrspace pointer. It fails with "Calling a function with a bad signature!" assertion in CallInst constructor because it tries to pass a non-default addrspace pointer to the pointer argument which has default addrspace. The fix is to add pointer type as another overloaded type to @llvm.masked.load/store intrinsics. Reviewed By: reames Differential Revision: http://reviews.llvm.org/D17270 llvm-svn: 274043
2016-06-29 02:27:25 +08:00
case Intrinsic::masked_load:
case Intrinsic::masked_store: {
SmallVector<Value *, 4> Args(CI->arg_operands().begin(),
CI->arg_operands().end());
CI->replaceAllUsesWith(Builder.CreateCall(NewFn, Args));
CI->eraseFromParent();
return;
}
Fix intrinsics for XOP frczss/sd instructions. These instructions only take one source register and zero the upper bits of the destination rather than preserving them. llvm-svn: 158396
2012-06-13 15:18:53 +08:00
}
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
}
try to make comments more meaningful; NFC Retry r266541 without the range-based-for-loop-change that was wrong. llvm-svn: 266658
2016-04-19 03:11:57 +08:00
void llvm::UpgradeCallsToIntrinsic(Function *F) {
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
assert(F && "Illegal attempt to upgrade a non-existent intrinsic.");
try to make comments more meaningful; NFC Retry r266541 without the range-based-for-loop-change that was wrong. llvm-svn: 266658
2016-04-19 03:11:57 +08:00
// Check if this function should be upgraded and get the replacement function
// if there is one.
rip out a ton of intrinsic modernization logic from AutoUpgrade.cpp, which is for pre-2.9 bitcode files. We keep x86 unaligned loads, movnt, crc32, and the target indep prefetch change. As usual, updating the testsuite is a PITA. llvm-svn: 133337
2011-06-18 14:05:24 +08:00
Function *NewFn;
Bring back int_x86_sse2_movl_dq intrinsic for backward compatibility. Make sure it's auto-upgraded to a shufflevector instruction. llvm-svn: 45131
2007-12-18 06:33:23 +08:00
if (UpgradeIntrinsicFunction(F, NewFn)) {
try to make comments more meaningful; NFC Retry r266541 without the range-based-for-loop-change that was wrong. llvm-svn: 266658
2016-04-19 03:11:57 +08:00
// Replace all users of the old function with the new function or new
// instructions. This is not a range loop because the call is deleted.
for (auto UI = F->user_begin(), UE = F->user_end(); UI != UE; )
Revert "use range loop, try to make comments more meaningful; NFCI" This reverts commit r266541 since it introduces a use-after-free: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/11471 llvm-svn: 266550
2016-04-17 11:59:37 +08:00
if (CallInst *CI = dyn_cast<CallInst>(*UI++))
Remove always-true comparison, NFC. Summary: Looking at r241279, I noticed that UpgradedIntrinsics only gets written to in the following code: if (UpgradeIntrinsicFunction(&F, NewFn)) UpgradedIntrinsics[&F] = NewFn; Looking through UpgradeIntrinsicFunction, we always return false OR NewFn will be set to a different function from our source. This patch pulls the F != NewFn into UpgradeIntrinsicFunction as an assert, and removes the check from callers of UpgradeIntrinsicFunction. Reviewers: rafael, chandlerc Subscribers: llvm-commits-list Differential Revision: http://reviews.llvm.org/D10915 llvm-svn: 241369
2015-07-04 04:12:01 +08:00
UpgradeIntrinsicCall(CI, NewFn);
try to make comments more meaningful; NFC Retry r266541 without the range-based-for-loop-change that was wrong. llvm-svn: 266658
2016-04-19 03:11:57 +08:00
Remove always-true comparison, NFC. Summary: Looking at r241279, I noticed that UpgradedIntrinsics only gets written to in the following code: if (UpgradeIntrinsicFunction(&F, NewFn)) UpgradedIntrinsics[&F] = NewFn; Looking through UpgradeIntrinsicFunction, we always return false OR NewFn will be set to a different function from our source. This patch pulls the F != NewFn into UpgradeIntrinsicFunction as an assert, and removes the check from callers of UpgradeIntrinsicFunction. Reviewers: rafael, chandlerc Subscribers: llvm-commits-list Differential Revision: http://reviews.llvm.org/D10915 llvm-svn: 241369
2015-07-04 04:12:01 +08:00
// Remove old function, no longer used, from the module.
F->eraseFromParent();
This is the patch to provide clean intrinsic function overloading support in LLVM. It cleans up the intrinsic definitions and generally smooths the process for more complicated intrinsic writing. It will be used by the upcoming atomic intrinsics as well as vector and float intrinsics in the future. This also changes the syntax for llvm.bswap, llvm.part.set, llvm.part.select, and llvm.ct* intrinsics. They are automatically upgraded by both the LLVM ASM reader and the bitcode reader. The test cases have been updated, with special tests added to ensure the automatic upgrading is supported. llvm-svn: 40807
2007-08-04 09:51:18 +08:00
}
}
Reapply 79977. Use MDNodes to encode debug info in llvm IR. llvm-svn: 80406
2009-08-29 07:24:31 +08:00
Fix auto-upgrade of TBAA tags in Bitcode Reader If TBAA is on an intrinsic and it gets upgraded, it'll delete the call instruction that we collected in a vector. Even if we were to use WeakVH, it'll drop the TBAA and we'll hit the assert on the upgrade path. r263673 gave a shot to make sure the TBAA upgrade happens before intrinsics upgrade, but failed to account for all cases. Instead of collecting instructions in a vector, this patch makes it just upgrade the TBAA on the fly, because metadata are always already loaded at this point. Differential Revision: https://reviews.llvm.org/D24533 llvm-svn: 281549
2016-09-15 06:29:59 +08:00
MDNode *llvm::UpgradeTBAANode(MDNode &MD) {
AutoUpgrade: upgrade from scalar TBAA format to struct-path aware TBAA format. We treat TBAA tags as struct-path aware TBAA format when the first operand is a MDNode and the tag has 3 or more operands. llvm-svn: 191593
2013-09-28 08:22:27 +08:00
// Check if the tag uses struct-path aware TBAA format.
Fix auto-upgrade of TBAA tags in Bitcode Reader If TBAA is on an intrinsic and it gets upgraded, it'll delete the call instruction that we collected in a vector. Even if we were to use WeakVH, it'll drop the TBAA and we'll hit the assert on the upgrade path. r263673 gave a shot to make sure the TBAA upgrade happens before intrinsics upgrade, but failed to account for all cases. Instead of collecting instructions in a vector, this patch makes it just upgrade the TBAA on the fly, because metadata are always already loaded at this point. Differential Revision: https://reviews.llvm.org/D24533 llvm-svn: 281549
2016-09-15 06:29:59 +08:00
if (isa<MDNode>(MD.getOperand(0)) && MD.getNumOperands() >= 3)
return &MD;
AutoUpgrade: upgrade from scalar TBAA format to struct-path aware TBAA format. We treat TBAA tags as struct-path aware TBAA format when the first operand is a MDNode and the tag has 3 or more operands. llvm-svn: 191593
2013-09-28 08:22:27 +08:00
Fix auto-upgrade of TBAA tags in Bitcode Reader If TBAA is on an intrinsic and it gets upgraded, it'll delete the call instruction that we collected in a vector. Even if we were to use WeakVH, it'll drop the TBAA and we'll hit the assert on the upgrade path. r263673 gave a shot to make sure the TBAA upgrade happens before intrinsics upgrade, but failed to account for all cases. Instead of collecting instructions in a vector, this patch makes it just upgrade the TBAA on the fly, because metadata are always already loaded at this point. Differential Revision: https://reviews.llvm.org/D24533 llvm-svn: 281549
2016-09-15 06:29:59 +08:00
auto &Context = MD.getContext();
if (MD.getNumOperands() == 3) {
Metadata *Elts[] = {MD.getOperand(0), MD.getOperand(1)};
MDNode *ScalarType = MDNode::get(Context, Elts);
AutoUpgrade: upgrade from scalar TBAA format to struct-path aware TBAA format. We treat TBAA tags as struct-path aware TBAA format when the first operand is a MDNode and the tag has 3 or more operands. llvm-svn: 191593
2013-09-28 08:22:27 +08:00
// Create a MDNode <ScalarType, ScalarType, offset 0, const>
IR: Split Metadata from Value Split `Metadata` away from the `Value` class hierarchy, as part of PR21532. Assembly and bitcode changes are in the wings, but this is the bulk of the change for the IR C++ API. I have a follow-up patch prepared for `clang`. If this breaks other sub-projects, I apologize in advance :(. Help me compile it on Darwin I'll try to fix it. FWIW, the errors should be easy to fix, so it may be simpler to just fix it yourself. This breaks the build for all metadata-related code that's out-of-tree. Rest assured the transition is mechanical and the compiler should catch almost all of the problems. Here's a quick guide for updating your code: - `Metadata` is the root of a class hierarchy with three main classes: `MDNode`, `MDString`, and `ValueAsMetadata`. It is distinct from the `Value` class hierarchy. It is typeless -- i.e., instances do *not* have a `Type`. - `MDNode`'s operands are all `Metadata *` (instead of `Value *`). - `TrackingVH<MDNode>` and `WeakVH` referring to metadata can be replaced with `TrackingMDNodeRef` and `TrackingMDRef`, respectively. If you're referring solely to resolved `MDNode`s -- post graph construction -- just use `MDNode*`. - `MDNode` (and the rest of `Metadata`) have only limited support for `replaceAllUsesWith()`. As long as an `MDNode` is pointing at a forward declaration -- the result of `MDNode::getTemporary()` -- it maintains a side map of its uses and can RAUW itself. Once the forward declarations are fully resolved RAUW support is dropped on the ground. This means that uniquing collisions on changing operands cause nodes to become "distinct". (This already happened fairly commonly, whenever an operand went to null.) If you're constructing complex (non self-reference) `MDNode` cycles, you need to call `MDNode::resolveCycles()` on each node (or on a top-level node that somehow references all of the nodes). Also, don't do that. Metadata cycles (and the RAUW machinery needed to construct them) are expensive. - An `MDNode` can only refer to a `Constant` through a bridge called `ConstantAsMetadata` (one of the subclasses of `ValueAsMetadata`). As a side effect, accessing an operand of an `MDNode` that is known to be, e.g., `ConstantInt`, takes three steps: first, cast from `Metadata` to `ConstantAsMetadata`; second, extract the `Constant`; third, cast down to `ConstantInt`. The eventual goal is to introduce `MDInt`/`MDFloat`/etc. and have metadata schema owners transition away from using `Constant`s when the type isn't important (and they don't care about referring to `GlobalValue`s). In the meantime, I've added transitional API to the `mdconst` namespace that matches semantics with the old code, in order to avoid adding the error-prone three-step equivalent to every call site. If your old code was: MDNode *N = foo(); bar(isa <ConstantInt>(N->getOperand(0))); baz(cast <ConstantInt>(N->getOperand(1))); bak(cast_or_null <ConstantInt>(N->getOperand(2))); bat(dyn_cast <ConstantInt>(N->getOperand(3))); bay(dyn_cast_or_null<ConstantInt>(N->getOperand(4))); you can trivially match its semantics with: MDNode *N = foo(); bar(mdconst::hasa <ConstantInt>(N->getOperand(0))); baz(mdconst::extract <ConstantInt>(N->getOperand(1))); bak(mdconst::extract_or_null <ConstantInt>(N->getOperand(2))); bat(mdconst::dyn_extract <ConstantInt>(N->getOperand(3))); bay(mdconst::dyn_extract_or_null<ConstantInt>(N->getOperand(4))); and when you transition your metadata schema to `MDInt`: MDNode *N = foo(); bar(isa <MDInt>(N->getOperand(0))); baz(cast <MDInt>(N->getOperand(1))); bak(cast_or_null <MDInt>(N->getOperand(2))); bat(dyn_cast <MDInt>(N->getOperand(3))); bay(dyn_cast_or_null<MDInt>(N->getOperand(4))); - A `CallInst` -- specifically, intrinsic instructions -- can refer to metadata through a bridge called `MetadataAsValue`. This is a subclass of `Value` where `getType()->isMetadataTy()`. `MetadataAsValue` is the *only* class that can legally refer to a `LocalAsMetadata`, which is a bridged form of non-`Constant` values like `Argument` and `Instruction`. It can also refer to any other `Metadata` subclass. (I'll break all your testcases in a follow-up commit, when I propagate this change to assembly.) llvm-svn: 223802
2014-12-10 02:38:53 +08:00
Metadata *Elts2[] = {ScalarType, ScalarType,
Fix auto-upgrade of TBAA tags in Bitcode Reader If TBAA is on an intrinsic and it gets upgraded, it'll delete the call instruction that we collected in a vector. Even if we were to use WeakVH, it'll drop the TBAA and we'll hit the assert on the upgrade path. r263673 gave a shot to make sure the TBAA upgrade happens before intrinsics upgrade, but failed to account for all cases. Instead of collecting instructions in a vector, this patch makes it just upgrade the TBAA on the fly, because metadata are always already loaded at this point. Differential Revision: https://reviews.llvm.org/D24533 llvm-svn: 281549
2016-09-15 06:29:59 +08:00
ConstantAsMetadata::get(
Constant::getNullValue(Type::getInt64Ty(Context))),
MD.getOperand(2)};
return MDNode::get(Context, Elts2);
AutoUpgrade: upgrade from scalar TBAA format to struct-path aware TBAA format. We treat TBAA tags as struct-path aware TBAA format when the first operand is a MDNode and the tag has 3 or more operands. llvm-svn: 191593
2013-09-28 08:22:27 +08:00
}
Fix auto-upgrade of TBAA tags in Bitcode Reader If TBAA is on an intrinsic and it gets upgraded, it'll delete the call instruction that we collected in a vector. Even if we were to use WeakVH, it'll drop the TBAA and we'll hit the assert on the upgrade path. r263673 gave a shot to make sure the TBAA upgrade happens before intrinsics upgrade, but failed to account for all cases. Instead of collecting instructions in a vector, this patch makes it just upgrade the TBAA on the fly, because metadata are always already loaded at this point. Differential Revision: https://reviews.llvm.org/D24533 llvm-svn: 281549
2016-09-15 06:29:59 +08:00
// Create a MDNode <MD, MD, offset 0>
Metadata *Elts[] = {&MD, &MD, ConstantAsMetadata::get(Constant::getNullValue(
Type::getInt64Ty(Context)))};
return MDNode::get(Context, Elts);
AutoUpgrade: upgrade from scalar TBAA format to struct-path aware TBAA format. We treat TBAA tags as struct-path aware TBAA format when the first operand is a MDNode and the tag has 3 or more operands. llvm-svn: 191593
2013-09-28 08:22:27 +08:00
}
Add addrspacecast instruction. Patch by Michele Scandale! llvm-svn: 194760
2013-11-15 09:34:59 +08:00
Instruction *llvm::UpgradeBitCastInst(unsigned Opc, Value *V, Type *DestTy,
Instruction *&Temp) {
if (Opc != Instruction::BitCast)
[C++11] More 'nullptr' conversion or in some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 205831
2014-04-09 14:08:46 +08:00
return nullptr;
Add addrspacecast instruction. Patch by Michele Scandale! llvm-svn: 194760
2013-11-15 09:34:59 +08:00
[C++11] More 'nullptr' conversion or in some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 205831
2014-04-09 14:08:46 +08:00
Temp = nullptr;
Add addrspacecast instruction. Patch by Michele Scandale! llvm-svn: 194760
2013-11-15 09:34:59 +08:00
Type *SrcTy = V->getType();
if (SrcTy->isPtrOrPtrVectorTy() && DestTy->isPtrOrPtrVectorTy() &&
SrcTy->getPointerAddressSpace() != DestTy->getPointerAddressSpace()) {
LLVMContext &Context = V->getContext();
// We have no information about target data layout, so we assume that
// the maximum pointer size is 64bit.
Type *MidTy = Type::getInt64Ty(Context);
Temp = CastInst::Create(Instruction::PtrToInt, V, MidTy);
return CastInst::Create(Instruction::IntToPtr, Temp, DestTy);
}
[C++11] More 'nullptr' conversion or in some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 205831
2014-04-09 14:08:46 +08:00
return nullptr;
Add addrspacecast instruction. Patch by Michele Scandale! llvm-svn: 194760
2013-11-15 09:34:59 +08:00
}
Value *llvm::UpgradeBitCastExpr(unsigned Opc, Constant *C, Type *DestTy) {
if (Opc != Instruction::BitCast)
[C++11] More 'nullptr' conversion or in some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 205831
2014-04-09 14:08:46 +08:00
return nullptr;
Add addrspacecast instruction. Patch by Michele Scandale! llvm-svn: 194760
2013-11-15 09:34:59 +08:00
Type *SrcTy = C->getType();
if (SrcTy->isPtrOrPtrVectorTy() && DestTy->isPtrOrPtrVectorTy() &&
SrcTy->getPointerAddressSpace() != DestTy->getPointerAddressSpace()) {
LLVMContext &Context = C->getContext();
// We have no information about target data layout, so we assume that
// the maximum pointer size is 64bit.
Type *MidTy = Type::getInt64Ty(Context);
return ConstantExpr::getIntToPtr(ConstantExpr::getPtrToInt(C, MidTy),
DestTy);
}
[C++11] More 'nullptr' conversion or in some cases just using a boolean check instead of comparing to nullptr. llvm-svn: 205831
2014-04-09 14:08:46 +08:00
return nullptr;
Add addrspacecast instruction. Patch by Michele Scandale! llvm-svn: 194760
2013-11-15 09:34:59 +08:00
}
Debug Info: drop debug info via upgrading path if version number does not match. Add a helper function getDebugInfoVersionFromModule to return the debug info version number for a module. "Verifier/module-flags-1.ll" checks for verification errors. It will seg fault when calling getDebugInfoVersionFromModule because of the incorrect format for module flags in the testing case. We make getModuleFlagsMetadata more robust by checking for error conditions. PR17982 llvm-svn: 196158
2013-12-03 05:29:56 +08:00
/// Check the debug info version number, if it is out-dated, drop the debug
/// info. Return true if module is modified.
bool llvm::UpgradeDebugInfo(Module &M) {
Report a warning when dropping outdated debug info metadata. Use DiagnosticInfo to emit the warning. llvm-svn: 199346
2014-01-16 09:51:12 +08:00
unsigned Version = getDebugMetadataVersionFromModule(M);
if (Version == DEBUG_METADATA_VERSION)
Debug Info: drop debug info via upgrading path if version number does not match. Add a helper function getDebugInfoVersionFromModule to return the debug info version number for a module. "Verifier/module-flags-1.ll" checks for verification errors. It will seg fault when calling getDebugInfoVersionFromModule because of the incorrect format for module flags in the testing case. We make getModuleFlagsMetadata more robust by checking for error conditions. PR17982 llvm-svn: 196158
2013-12-03 05:29:56 +08:00
return false;
Report a warning when dropping outdated debug info metadata. Use DiagnosticInfo to emit the warning. llvm-svn: 199346
2014-01-16 09:51:12 +08:00
bool RetCode = StripDebugInfo(M);
if (RetCode) {
DiagnosticInfoDebugMetadataVersion DiagVersion(M, Version);
M.getContext().diagnose(DiagVersion);
}
return RetCode;
Debug Info: drop debug info via upgrading path if version number does not match. Add a helper function getDebugInfoVersionFromModule to return the debug info version number for a module. "Verifier/module-flags-1.ll" checks for verification errors. It will seg fault when calling getDebugInfoVersionFromModule because of the incorrect format for module flags in the testing case. We make getModuleFlagsMetadata more robust by checking for error conditions. PR17982 llvm-svn: 196158
2013-12-03 05:29:56 +08:00
}
Rename loop unrolling and loop vectorizer metadata to have a common prefix. [LLVM part] These patches rename the loop unrolling and loop vectorizer metadata such that they have a common 'llvm.loop.' prefix. Metadata name changes: llvm.vectorizer.* => llvm.loop.vectorizer.* llvm.loopunroll.* => llvm.loop.unroll.* This was a suggestion from an earlier review (http://reviews.llvm.org/D4090) which added the loop unrolling metadata. Patch by Mark Heffernan. llvm-svn: 211710
2014-06-25 23:41:00 +08:00
Objective-C Class Properties: Autoupgrade "Class Properties" module flag. When we have "Image Info Version" module flag but don't have "Class Properties" module flag, set "Class Properties" module flag to 0, so we can correctly emit errors when one module has the flag set and another module does not. rdar://26469641 llvm-svn: 270791
2016-05-26 07:14:48 +08:00
bool llvm::UpgradeModuleFlags(Module &M) {
const NamedMDNode *ModFlags = M.getModuleFlagsMetadata();
if (!ModFlags)
return false;
bool HasObjCFlag = false, HasClassProperties = false;
for (unsigned I = 0, E = ModFlags->getNumOperands(); I != E; ++I) {
MDNode *Op = ModFlags->getOperand(I);
if (Op->getNumOperands() < 2)
continue;
MDString *ID = dyn_cast_or_null<MDString>(Op->getOperand(1));
if (!ID)
continue;
if (ID->getString() == "Objective-C Image Info Version")
HasObjCFlag = true;
if (ID->getString() == "Objective-C Class Properties")
HasClassProperties = true;
}
// "Objective-C Class Properties" is recently added for Objective-C. We
// upgrade ObjC bitcodes to contain a "Objective-C Class Properties" module
Fix autoupgrade logic for Objective-C class properties module flag Previous we were issuing an error when linking a module containing the new Objective-C metadata structure for class properties with an "old" one. Now instead we downgrade the module flag so that the Objective-C runtime does not expect the new metadata structure. This is consistent with what ld64 is doing on binary files. Differential Revision: https://reviews.llvm.org/D24620 llvm-svn: 281685
2016-09-16 08:38:18 +08:00
// flag of value 0, so we can correclty downgrade this flag when trying to
// link an ObjC bitcode without this module flag with an ObjC bitcode with
// this module flag.
Objective-C Class Properties: Autoupgrade "Class Properties" module flag. When we have "Image Info Version" module flag but don't have "Class Properties" module flag, set "Class Properties" module flag to 0, so we can correctly emit errors when one module has the flag set and another module does not. rdar://26469641 llvm-svn: 270791
2016-05-26 07:14:48 +08:00
if (HasObjCFlag && !HasClassProperties) {
Fix autoupgrade logic for Objective-C class properties module flag Previous we were issuing an error when linking a module containing the new Objective-C metadata structure for class properties with an "old" one. Now instead we downgrade the module flag so that the Objective-C runtime does not expect the new metadata structure. This is consistent with what ld64 is doing on binary files. Differential Revision: https://reviews.llvm.org/D24620 llvm-svn: 281685
2016-09-16 08:38:18 +08:00
M.addModuleFlag(llvm::Module::Override, "Objective-C Class Properties",
Objective-C Class Properties: Autoupgrade "Class Properties" module flag. When we have "Image Info Version" module flag but don't have "Class Properties" module flag, set "Class Properties" module flag to 0, so we can correctly emit errors when one module has the flag set and another module does not. rdar://26469641 llvm-svn: 270791
2016-05-26 07:14:48 +08:00
(uint32_t)0);
return true;
}
return false;
}
IR: Stop upgrading !llvm.loop attachments via MDString Remove logic to upgrade !llvm.loop by changing the MDString tag directly. This old logic would check (and change) arbitrary strings that had nothing to do with loop metadata. Instead, check !llvm.loop attachments directly, and change which strings get attached. Rather than updating the assembly-based upgrade, drop it entirely. It has been quite a while since we supported upgrading textual IR. llvm-svn: 264373
2016-03-25 08:56:13 +08:00
static bool isOldLoopArgument(Metadata *MD) {
auto *T = dyn_cast_or_null<MDTuple>(MD);
if (!T)
return false;
if (T->getNumOperands() < 1)
return false;
auto *S = dyn_cast_or_null<MDString>(T->getOperand(0));
if (!S)
return false;
return S->getString().startswith("llvm.vectorizer.");
}
static MDString *upgradeLoopTag(LLVMContext &C, StringRef OldTag) {
StringRef OldPrefix = "llvm.vectorizer.";
assert(OldTag.startswith(OldPrefix) && "Expected old prefix");
if (OldTag == "llvm.vectorizer.unroll")
return MDString::get(C, "llvm.loop.interleave.count");
return MDString::get(
C, (Twine("llvm.loop.vectorize.") + OldTag.drop_front(OldPrefix.size()))
.str());
}
static Metadata *upgradeLoopArgument(Metadata *MD) {
auto *T = dyn_cast_or_null<MDTuple>(MD);
if (!T)
return MD;
if (T->getNumOperands() < 1)
return MD;
auto *OldTag = dyn_cast_or_null<MDString>(T->getOperand(0));
if (!OldTag)
return MD;
if (!OldTag->getString().startswith("llvm.vectorizer."))
return MD;
// This has an old tag. Upgrade it.
SmallVector<Metadata *, 8> Ops;
Ops.reserve(T->getNumOperands());
Ops.push_back(upgradeLoopTag(T->getContext(), OldTag->getString()));
for (unsigned I = 1, E = T->getNumOperands(); I != E; ++I)
Ops.push_back(T->getOperand(I));
return MDTuple::get(T->getContext(), Ops);
}
MDNode *llvm::upgradeInstructionLoopAttachment(MDNode &N) {
auto *T = dyn_cast<MDTuple>(&N);
if (!T)
return &N;
Use range algorithms instead of unpacking begin/end No functionality change is intended. llvm-svn: 278417
2016-08-12 05:15:00 +08:00
if (none_of(T->operands(), isOldLoopArgument))
IR: Stop upgrading !llvm.loop attachments via MDString Remove logic to upgrade !llvm.loop by changing the MDString tag directly. This old logic would check (and change) arbitrary strings that had nothing to do with loop metadata. Instead, check !llvm.loop attachments directly, and change which strings get attached. Rather than updating the assembly-based upgrade, drop it entirely. It has been quite a while since we supported upgrading textual IR. llvm-svn: 264373
2016-03-25 08:56:13 +08:00
return &N;
SmallVector<Metadata *, 8> Ops;
Ops.reserve(T->getNumOperands());
for (Metadata *MD : T->operands())
Ops.push_back(upgradeLoopArgument(MD));
return MDTuple::get(T->getContext(), Ops);
Rename loop unrolling and loop vectorizer metadata to have a common prefix. [LLVM part] These patches rename the loop unrolling and loop vectorizer metadata such that they have a common 'llvm.loop.' prefix. Metadata name changes: llvm.vectorizer.* => llvm.loop.vectorizer.* llvm.loopunroll.* => llvm.loop.unroll.* This was a suggestion from an earlier review (http://reviews.llvm.org/D4090) which added the loop unrolling metadata. Patch by Mark Heffernan. llvm-svn: 211710
2014-06-25 23:41:00 +08:00
}