maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/Target/AMDGPU/AMDGPULibFunc.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

1055 lines
38 KiB
C++
Raw Normal View History

[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
//===-- AMDGPULibFunc.cpp -------------------------------------------------===//
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 16:50:56 +08:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
//
//===----------------------------------------------------------------------===//
//
// This file contains utility functions to work with Itanium mangled names
//
//===----------------------------------------------------------------------===//
#include "AMDGPULibFunc.h"
AMDGPULibFunc - fix include order. NFC. Ensure AMDGPULibFunc.h module header is first, and fix exposed missing forward declaration.
2020-05-24 20:25:59 +08:00
#include "AMDGPU.h"
#include "llvm/ADT/SmallString.h"
#include "llvm/ADT/SmallVector.h"
[TypeSize] Allow returning scalable size in implicit conversion to uint64_t This patch removes compiler runtime assertions that ensure the implicit conversion are only guaranteed to work for fixed-width vectors. With the assert it would be impossible to get _anything_ to build until the entire codebase has been upgraded, even when the indiscriminate uses of the size as uint64_t would work fine for both scalable and fixed-width types. This issue will need to be addressed differently, with build-time errors rather than assertion failures, but that effort falls beyond the scope of this patch. Returning the scalable size and avoiding the assert in getFixedSize() is a temporary stop-gap in order to use LLVM for compiling and using the SVE ACLE intrinsics. Reviewers: efriedma, huntergr, rovka, ctetreau, rengolin Reviewed By: efriedma Tags: #llvm Differential Revision: https://reviews.llvm.org/D75297
2020-03-13 17:13:34 +08:00
#include "llvm/ADT/StringExtras.h"
AMDGPULibFunc - fix include order. NFC. Ensure AMDGPULibFunc.h module header is first, and fix exposed missing forward declaration.
2020-05-24 20:25:59 +08:00
#include "llvm/ADT/StringSwitch.h"
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
#include "llvm/IR/Attributes.h"
#include "llvm/IR/DerivedTypes.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/Module.h"
#include "llvm/IR/ValueSymbolTable.h"
AMDGPULibFunc - fix include order. NFC. Ensure AMDGPULibFunc.h module header is first, and fix exposed missing forward declaration.
2020-05-24 20:25:59 +08:00
#include "llvm/Support/raw_ostream.h"
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
#include <string>
using namespace llvm;
namespace {
enum EManglingParam {
E_NONE,
EX_EVENT,
EX_FLOAT4,
EX_INTV4,
EX_RESERVEDID,
EX_SAMPLER,
EX_SIZET,
EX_UINT,
EX_UINTV4,
E_ANY,
E_CONSTPTR_ANY,
E_CONSTPTR_SWAPGL,
E_COPY,
E_IMAGECOORDS,
E_POINTEE,
E_SETBASE_I32,
E_SETBASE_U32,
E_MAKEBASE_UNS,
E_V16_OF_POINTEE,
E_V2_OF_POINTEE,
E_V3_OF_POINTEE,
E_V4_OF_POINTEE,
E_V8_OF_POINTEE,
E_VLTLPTR_ANY,
};
struct ManglingRule {
[AMDGPU] Downgrade from StringLiteral to const char* in an attempt to make GCC 5 happy llvm-svn: 369867
2019-08-25 20:47:31 +08:00
const char *Name;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
unsigned char Lead[2];
unsigned char Param[5];
int maxLeadIndex() const { return (std::max)(Lead[0], Lead[1]); }
int getNumLeads() const { return (Lead[0] ? 1 : 0) + (Lead[1] ? 1 : 0); }
unsigned getNumArgs() const;
Delay initialization of three static global maps, NFC This avoids allocating a few KB of heap memory on startup, and instead allocates these maps lazily. I noticed this while profiling LLD. llvm-svn: 357192
2019-03-29 01:33:41 +08:00
static StringMap<int> buildManglingRulesMap();
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
};
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
// Information about library functions with unmangled names.
class UnmangledFuncInfo {
[AMDGPU] Downgrade from StringLiteral to const char* in an attempt to make GCC 5 happy llvm-svn: 369867
2019-08-25 20:47:31 +08:00
const char *Name;
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
unsigned NumArgs;
// Table for all lib functions with unmangled names.
static const UnmangledFuncInfo Table[];
// Number of entries in Table.
static const unsigned TableSize;
Delay initialization of three static global maps, NFC This avoids allocating a few KB of heap memory on startup, and instead allocates these maps lazily. I noticed this while profiling LLD. llvm-svn: 357192
2019-03-29 01:33:41 +08:00
static StringMap<unsigned> buildNameMap();
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
public:
using ID = AMDGPULibFunc::EFuncId;
[AMDGPU] Downgrade from StringLiteral to const char* in an attempt to make GCC 5 happy llvm-svn: 369867
2019-08-25 20:47:31 +08:00
constexpr UnmangledFuncInfo(const char *_Name, unsigned _NumArgs)
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
: Name(_Name), NumArgs(_NumArgs) {}
// Get index to Table by function name.
static bool lookup(StringRef Name, ID &Id);
static unsigned toIndex(ID Id) {
assert(static_cast<unsigned>(Id) >
static_cast<unsigned>(AMDGPULibFunc::EI_LAST_MANGLED) &&
"Invalid unmangled library function");
return static_cast<unsigned>(Id) - 1 -
static_cast<unsigned>(AMDGPULibFunc::EI_LAST_MANGLED);
}
static ID toFuncId(unsigned Index) {
Delay initialization of three static global maps, NFC This avoids allocating a few KB of heap memory on startup, and instead allocates these maps lazily. I noticed this while profiling LLD. llvm-svn: 357192
2019-03-29 01:33:41 +08:00
assert(Index < TableSize &&
"Invalid unmangled library function");
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
return static_cast<ID>(
Index + 1 + static_cast<unsigned>(AMDGPULibFunc::EI_LAST_MANGLED));
}
static unsigned getNumArgs(ID Id) { return Table[toIndex(Id)].NumArgs; }
static StringRef getName(ID Id) { return Table[toIndex(Id)].Name; }
};
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
unsigned ManglingRule::getNumArgs() const {
unsigned I=0;
while (I < (sizeof Param/sizeof Param[0]) && Param[I]) ++I;
return I;
}
// This table describes function formal argument type rules. The order of rules
// corresponds to the EFuncId enum at AMDGPULibFunc.h
//
// "<func name>", { <leads> }, { <param rules> }
// where:
// <leads> - list of integers that are one-based indexes of formal argument
// used to mangle a function name. Other argument types are derived from types
// of these 'leads'. The order of integers in this list correspond to the
// order in which these arguments are mangled in the EDG mangling scheme. The
// same order should be preserved for arguments in the AMDGPULibFunc structure
// when it is used for mangling. For example:
// { "vstorea_half", {3,1}, {E_ANY,EX_SIZET,E_ANY}},
// will be mangled in EDG scheme as vstorea_half_<3dparam>_<1stparam>
// When mangling from code use:
// AMDGPULibFunc insc;
// insc.param[0] = ... // describe 3rd parameter
// insc.param[1] = ... // describe 1rd parameter
//
// <param rules> - list of rules used to derive all of the function formal
// argument types. EX_ prefixed are simple types, other derived from the
// latest 'lead' argument type in the order of encoding from first to last.
// E_ANY - use prev lead type, E_CONSTPTR_ANY - make const pointer out of
// prev lead type, etc. see ParamIterator::getNextParam() for details.
Fix some accidental global initializers by using StringLiteral instead of StringRef llvm-svn: 369850
2019-08-24 23:24:25 +08:00
static constexpr ManglingRule manglingRules[] = {
{ "", {0}, {0} },
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
{ "abs" , {1}, {E_ANY}},
{ "abs_diff" , {1}, {E_ANY,E_COPY}},
{ "acos" , {1}, {E_ANY}},
{ "acosh" , {1}, {E_ANY}},
{ "acospi" , {1}, {E_ANY}},
{ "add_sat" , {1}, {E_ANY,E_COPY}},
{ "all" , {1}, {E_ANY}},
{ "any" , {1}, {E_ANY}},
{ "asin" , {1}, {E_ANY}},
{ "asinh" , {1}, {E_ANY}},
{ "asinpi" , {1}, {E_ANY}},
{ "async_work_group_copy" , {1}, {E_ANY,E_CONSTPTR_SWAPGL,EX_SIZET,EX_EVENT}},
{ "async_work_group_strided_copy" , {1}, {E_ANY,E_CONSTPTR_SWAPGL,EX_SIZET,EX_SIZET,EX_EVENT}},
{ "atan" , {1}, {E_ANY}},
{ "atan2" , {1}, {E_ANY,E_COPY}},
{ "atan2pi" , {1}, {E_ANY,E_COPY}},
{ "atanh" , {1}, {E_ANY}},
{ "atanpi" , {1}, {E_ANY}},
{ "atomic_add" , {1}, {E_VLTLPTR_ANY,E_POINTEE}},
{ "atomic_and" , {1}, {E_VLTLPTR_ANY,E_POINTEE}},
{ "atomic_cmpxchg" , {1}, {E_VLTLPTR_ANY,E_POINTEE,E_POINTEE}},
{ "atomic_dec" , {1}, {E_VLTLPTR_ANY}},
{ "atomic_inc" , {1}, {E_VLTLPTR_ANY}},
{ "atomic_max" , {1}, {E_VLTLPTR_ANY,E_POINTEE}},
{ "atomic_min" , {1}, {E_VLTLPTR_ANY,E_POINTEE}},
{ "atomic_or" , {1}, {E_VLTLPTR_ANY,E_POINTEE}},
{ "atomic_sub" , {1}, {E_VLTLPTR_ANY,E_POINTEE}},
{ "atomic_xchg" , {1}, {E_VLTLPTR_ANY,E_POINTEE}},
{ "atomic_xor" , {1}, {E_VLTLPTR_ANY,E_POINTEE}},
{ "bitselect" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "cbrt" , {1}, {E_ANY}},
{ "ceil" , {1}, {E_ANY}},
{ "clamp" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "clz" , {1}, {E_ANY}},
{ "commit_read_pipe" , {1}, {E_ANY,EX_RESERVEDID}},
{ "commit_write_pipe" , {1}, {E_ANY,EX_RESERVEDID}},
{ "copysign" , {1}, {E_ANY,E_COPY}},
{ "cos" , {1}, {E_ANY}},
{ "cosh" , {1}, {E_ANY}},
{ "cospi" , {1}, {E_ANY}},
{ "cross" , {1}, {E_ANY,E_COPY}},
{ "ctz" , {1}, {E_ANY}},
{ "degrees" , {1}, {E_ANY}},
{ "distance" , {1}, {E_ANY,E_COPY}},
{ "divide" , {1}, {E_ANY,E_COPY}},
{ "dot" , {1}, {E_ANY,E_COPY}},
{ "erf" , {1}, {E_ANY}},
{ "erfc" , {1}, {E_ANY}},
{ "exp" , {1}, {E_ANY}},
{ "exp10" , {1}, {E_ANY}},
{ "exp2" , {1}, {E_ANY}},
{ "expm1" , {1}, {E_ANY}},
{ "fabs" , {1}, {E_ANY}},
{ "fast_distance" , {1}, {E_ANY,E_COPY}},
{ "fast_length" , {1}, {E_ANY}},
{ "fast_normalize" , {1}, {E_ANY}},
{ "fdim" , {1}, {E_ANY,E_COPY}},
{ "floor" , {1}, {E_ANY}},
{ "fma" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "fmax" , {1}, {E_ANY,E_COPY}},
{ "fmin" , {1}, {E_ANY,E_COPY}},
{ "fmod" , {1}, {E_ANY,E_COPY}},
{ "fract" , {2}, {E_POINTEE,E_ANY}},
{ "frexp" , {1,2}, {E_ANY,E_ANY}},
{ "get_image_array_size" , {1}, {E_ANY}},
{ "get_image_channel_data_type" , {1}, {E_ANY}},
{ "get_image_channel_order" , {1}, {E_ANY}},
{ "get_image_dim" , {1}, {E_ANY}},
{ "get_image_height" , {1}, {E_ANY}},
{ "get_image_width" , {1}, {E_ANY}},
{ "get_pipe_max_packets" , {1}, {E_ANY}},
{ "get_pipe_num_packets" , {1}, {E_ANY}},
{ "hadd" , {1}, {E_ANY,E_COPY}},
{ "hypot" , {1}, {E_ANY,E_COPY}},
{ "ilogb" , {1}, {E_ANY}},
{ "isequal" , {1}, {E_ANY,E_COPY}},
{ "isfinite" , {1}, {E_ANY}},
{ "isgreater" , {1}, {E_ANY,E_COPY}},
{ "isgreaterequal" , {1}, {E_ANY,E_COPY}},
{ "isinf" , {1}, {E_ANY}},
{ "isless" , {1}, {E_ANY,E_COPY}},
{ "islessequal" , {1}, {E_ANY,E_COPY}},
{ "islessgreater" , {1}, {E_ANY,E_COPY}},
{ "isnan" , {1}, {E_ANY}},
{ "isnormal" , {1}, {E_ANY}},
{ "isnotequal" , {1}, {E_ANY,E_COPY}},
{ "isordered" , {1}, {E_ANY,E_COPY}},
{ "isunordered" , {1}, {E_ANY,E_COPY}},
{ "ldexp" , {1}, {E_ANY,E_SETBASE_I32}},
{ "length" , {1}, {E_ANY}},
{ "lgamma" , {1}, {E_ANY}},
{ "lgamma_r" , {1,2}, {E_ANY,E_ANY}},
{ "log" , {1}, {E_ANY}},
{ "log10" , {1}, {E_ANY}},
{ "log1p" , {1}, {E_ANY}},
{ "log2" , {1}, {E_ANY}},
{ "logb" , {1}, {E_ANY}},
{ "mad" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "mad24" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "mad_hi" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "mad_sat" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "max" , {1}, {E_ANY,E_COPY}},
{ "maxmag" , {1}, {E_ANY,E_COPY}},
{ "min" , {1}, {E_ANY,E_COPY}},
{ "minmag" , {1}, {E_ANY,E_COPY}},
{ "mix" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "modf" , {2}, {E_POINTEE,E_ANY}},
{ "mul24" , {1}, {E_ANY,E_COPY}},
{ "mul_hi" , {1}, {E_ANY,E_COPY}},
{ "nan" , {1}, {E_ANY}},
{ "nextafter" , {1}, {E_ANY,E_COPY}},
{ "normalize" , {1}, {E_ANY}},
{ "popcount" , {1}, {E_ANY}},
{ "pow" , {1}, {E_ANY,E_COPY}},
{ "pown" , {1}, {E_ANY,E_SETBASE_I32}},
{ "powr" , {1}, {E_ANY,E_COPY}},
{ "prefetch" , {1}, {E_CONSTPTR_ANY,EX_SIZET}},
{ "radians" , {1}, {E_ANY}},
{ "recip" , {1}, {E_ANY}},
{ "remainder" , {1}, {E_ANY,E_COPY}},
{ "remquo" , {1,3}, {E_ANY,E_COPY,E_ANY}},
{ "reserve_read_pipe" , {1}, {E_ANY,EX_UINT}},
{ "reserve_write_pipe" , {1}, {E_ANY,EX_UINT}},
{ "rhadd" , {1}, {E_ANY,E_COPY}},
{ "rint" , {1}, {E_ANY}},
{ "rootn" , {1}, {E_ANY,E_SETBASE_I32}},
{ "rotate" , {1}, {E_ANY,E_COPY}},
{ "round" , {1}, {E_ANY}},
{ "rsqrt" , {1}, {E_ANY}},
{ "select" , {1,3}, {E_ANY,E_COPY,E_ANY}},
{ "shuffle" , {1,2}, {E_ANY,E_ANY}},
{ "shuffle2" , {1,3}, {E_ANY,E_COPY,E_ANY}},
{ "sign" , {1}, {E_ANY}},
{ "signbit" , {1}, {E_ANY}},
{ "sin" , {1}, {E_ANY}},
{ "sincos" , {2}, {E_POINTEE,E_ANY}},
{ "sinh" , {1}, {E_ANY}},
{ "sinpi" , {1}, {E_ANY}},
{ "smoothstep" , {1}, {E_ANY,E_COPY,E_COPY}},
{ "sqrt" , {1}, {E_ANY}},
{ "step" , {1}, {E_ANY,E_COPY}},
{ "sub_group_broadcast" , {1}, {E_ANY,EX_UINT}},
{ "sub_group_commit_read_pipe" , {1}, {E_ANY,EX_RESERVEDID}},
{ "sub_group_commit_write_pipe" , {1}, {E_ANY,EX_RESERVEDID}},
{ "sub_group_reduce_add" , {1}, {E_ANY}},
{ "sub_group_reduce_max" , {1}, {E_ANY}},
{ "sub_group_reduce_min" , {1}, {E_ANY}},
{ "sub_group_reserve_read_pipe" , {1}, {E_ANY,EX_UINT}},
{ "sub_group_reserve_write_pipe" , {1}, {E_ANY,EX_UINT}},
{ "sub_group_scan_exclusive_add" , {1}, {E_ANY}},
{ "sub_group_scan_exclusive_max" , {1}, {E_ANY}},
{ "sub_group_scan_exclusive_min" , {1}, {E_ANY}},
{ "sub_group_scan_inclusive_add" , {1}, {E_ANY}},
{ "sub_group_scan_inclusive_max" , {1}, {E_ANY}},
{ "sub_group_scan_inclusive_min" , {1}, {E_ANY}},
{ "sub_sat" , {1}, {E_ANY,E_COPY}},
{ "tan" , {1}, {E_ANY}},
{ "tanh" , {1}, {E_ANY}},
{ "tanpi" , {1}, {E_ANY}},
{ "tgamma" , {1}, {E_ANY}},
{ "trunc" , {1}, {E_ANY}},
{ "upsample" , {1}, {E_ANY,E_MAKEBASE_UNS}},
{ "vec_step" , {1}, {E_ANY}},
{ "vstore" , {3}, {E_POINTEE,EX_SIZET,E_ANY}},
{ "vstore16" , {3}, {E_V16_OF_POINTEE,EX_SIZET,E_ANY}},
{ "vstore2" , {3}, {E_V2_OF_POINTEE,EX_SIZET,E_ANY}},
{ "vstore3" , {3}, {E_V3_OF_POINTEE,EX_SIZET,E_ANY}},
{ "vstore4" , {3}, {E_V4_OF_POINTEE,EX_SIZET,E_ANY}},
{ "vstore8" , {3}, {E_V8_OF_POINTEE,EX_SIZET,E_ANY}},
{ "work_group_commit_read_pipe" , {1}, {E_ANY,EX_RESERVEDID}},
{ "work_group_commit_write_pipe" , {1}, {E_ANY,EX_RESERVEDID}},
{ "work_group_reduce_add" , {1}, {E_ANY}},
{ "work_group_reduce_max" , {1}, {E_ANY}},
{ "work_group_reduce_min" , {1}, {E_ANY}},
{ "work_group_reserve_read_pipe" , {1}, {E_ANY,EX_UINT}},
{ "work_group_reserve_write_pipe" , {1}, {E_ANY,EX_UINT}},
{ "work_group_scan_exclusive_add" , {1}, {E_ANY}},
{ "work_group_scan_exclusive_max" , {1}, {E_ANY}},
{ "work_group_scan_exclusive_min" , {1}, {E_ANY}},
{ "work_group_scan_inclusive_add" , {1}, {E_ANY}},
{ "work_group_scan_inclusive_max" , {1}, {E_ANY}},
{ "work_group_scan_inclusive_min" , {1}, {E_ANY}},
{ "write_imagef" , {1}, {E_ANY,E_IMAGECOORDS,EX_FLOAT4}},
{ "write_imagei" , {1}, {E_ANY,E_IMAGECOORDS,EX_INTV4}},
{ "write_imageui" , {1}, {E_ANY,E_IMAGECOORDS,EX_UINTV4}},
{ "ncos" , {1}, {E_ANY} },
{ "nexp2" , {1}, {E_ANY} },
{ "nfma" , {1}, {E_ANY, E_COPY, E_COPY} },
{ "nlog2" , {1}, {E_ANY} },
{ "nrcp" , {1}, {E_ANY} },
{ "nrsqrt" , {1}, {E_ANY} },
{ "nsin" , {1}, {E_ANY} },
{ "nsqrt" , {1}, {E_ANY} },
{ "ftz" , {1}, {E_ANY} },
{ "fldexp" , {1}, {E_ANY, EX_UINT} },
{ "class" , {1}, {E_ANY, EX_UINT} },
{ "rcbrt" , {1}, {E_ANY} },
};
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
// Library functions with unmangled name.
const UnmangledFuncInfo UnmangledFuncInfo::Table[] = {
{"__read_pipe_2", 4},
{"__read_pipe_4", 6},
{"__write_pipe_2", 4},
{"__write_pipe_4", 6},
};
const unsigned UnmangledFuncInfo::TableSize =
Delay initialization of three static global maps, NFC This avoids allocating a few KB of heap memory on startup, and instead allocates these maps lazily. I noticed this while profiling LLD. llvm-svn: 357192
2019-03-29 01:33:41 +08:00
array_lengthof(UnmangledFuncInfo::Table);
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
static AMDGPULibFunc::Param getRetType(AMDGPULibFunc::EFuncId id,
const AMDGPULibFunc::Param (&Leads)[2]) {
AMDGPULibFunc::Param Res = Leads[0];
// TBD - This switch may require to be extended for other intriniscs
switch (id) {
case AMDGPULibFunc::EI_SINCOS:
Res.PtrKind = AMDGPULibFunc::BYVALUE;
break;
default:
break;
}
return Res;
}
class ParamIterator {
const AMDGPULibFunc::Param (&Leads)[2];
const ManglingRule& Rule;
int Index;
public:
ParamIterator(const AMDGPULibFunc::Param (&leads)[2],
const ManglingRule& rule)
: Leads(leads), Rule(rule), Index(0) {}
AMDGPULibFunc::Param getNextParam();
};
AMDGPULibFunc::Param ParamIterator::getNextParam() {
AMDGPULibFunc::Param P;
if (Index >= int(sizeof Rule.Param/sizeof Rule.Param[0])) return P;
const char R = Rule.Param[Index];
switch (R) {
case E_NONE: break;
case EX_UINT:
P.ArgType = AMDGPULibFunc::U32; break;
case EX_INTV4:
P.ArgType = AMDGPULibFunc::I32; P.VectorSize = 4; break;
case EX_UINTV4:
P.ArgType = AMDGPULibFunc::U32; P.VectorSize = 4; break;
case EX_FLOAT4:
P.ArgType = AMDGPULibFunc::F32; P.VectorSize = 4; break;
case EX_SIZET:
P.ArgType = AMDGPULibFunc::U64; break;
case EX_EVENT:
P.ArgType = AMDGPULibFunc::EVENT; break;
case EX_SAMPLER:
P.ArgType = AMDGPULibFunc::SAMPLER; break;
case EX_RESERVEDID: break; // TBD
default:
if (Index == (Rule.Lead[1] - 1)) P = Leads[1];
else P = Leads[0];
switch (R) {
case E_ANY:
case E_COPY: break;
case E_POINTEE:
P.PtrKind = AMDGPULibFunc::BYVALUE; break;
case E_V2_OF_POINTEE:
P.VectorSize = 2; P.PtrKind = AMDGPULibFunc::BYVALUE; break;
case E_V3_OF_POINTEE:
P.VectorSize = 3; P.PtrKind = AMDGPULibFunc::BYVALUE; break;
case E_V4_OF_POINTEE:
P.VectorSize = 4; P.PtrKind = AMDGPULibFunc::BYVALUE; break;
case E_V8_OF_POINTEE:
P.VectorSize = 8; P.PtrKind = AMDGPULibFunc::BYVALUE; break;
case E_V16_OF_POINTEE:
P.VectorSize = 16; P.PtrKind = AMDGPULibFunc::BYVALUE; break;
case E_CONSTPTR_ANY:
P.PtrKind |= AMDGPULibFunc::CONST; break;
case E_VLTLPTR_ANY:
P.PtrKind |= AMDGPULibFunc::VOLATILE; break;
case E_SETBASE_I32:
P.ArgType = AMDGPULibFunc::I32; break;
case E_SETBASE_U32:
P.ArgType = AMDGPULibFunc::U32; break;
case E_MAKEBASE_UNS:
P.ArgType &= ~AMDGPULibFunc::BASE_TYPE_MASK;
P.ArgType |= AMDGPULibFunc::UINT;
break;
case E_IMAGECOORDS:
switch (P.ArgType) {
case AMDGPULibFunc::IMG1DA: P.VectorSize = 2; break;
case AMDGPULibFunc::IMG1DB: P.VectorSize = 1; break;
case AMDGPULibFunc::IMG2DA: P.VectorSize = 4; break;
case AMDGPULibFunc::IMG1D: P.VectorSize = 1; break;
case AMDGPULibFunc::IMG2D: P.VectorSize = 2; break;
case AMDGPULibFunc::IMG3D: P.VectorSize = 4; break;
}
P.PtrKind = AMDGPULibFunc::BYVALUE;
P.ArgType = AMDGPULibFunc::I32;
break;
[AMDGPU] Remove hardcoded address space value from AMDGPULibFunc AMDGPULibFunc hardcodes address space values of the old address space mapping, which causes invalid addrspacecast instructions and undefined functions in APPSDK sample MonteCarloAsianDP. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39616 llvm-svn: 317409
2017-11-05 01:37:43 +08:00
case E_CONSTPTR_SWAPGL: {
unsigned AS = AMDGPULibFunc::getAddrSpaceFromEPtrKind(P.PtrKind);
switch (AS) {
case AMDGPUAS::GLOBAL_ADDRESS: AS = AMDGPUAS::LOCAL_ADDRESS; break;
case AMDGPUAS::LOCAL_ADDRESS: AS = AMDGPUAS::GLOBAL_ADDRESS; break;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
}
[AMDGPU] Remove hardcoded address space value from AMDGPULibFunc AMDGPULibFunc hardcodes address space values of the old address space mapping, which causes invalid addrspacecast instructions and undefined functions in APPSDK sample MonteCarloAsianDP. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39616 llvm-svn: 317409
2017-11-05 01:37:43 +08:00
P.PtrKind = AMDGPULibFunc::getEPtrKindFromAddrSpace(AS);
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
P.PtrKind |= AMDGPULibFunc::CONST;
break;
[AMDGPU] Remove hardcoded address space value from AMDGPULibFunc AMDGPULibFunc hardcodes address space values of the old address space mapping, which causes invalid addrspacecast instructions and undefined functions in APPSDK sample MonteCarloAsianDP. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39616 llvm-svn: 317409
2017-11-05 01:37:43 +08:00
}
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
default: llvm_unreachable("Unhandeled param rule");
}
}
++Index;
return P;
}
inline static void drop_front(StringRef& str, size_t n = 1) {
str = str.drop_front(n);
}
static bool eatTerm(StringRef& mangledName, const char c) {
if (mangledName.front() == c) {
drop_front(mangledName);
return true;
}
return false;
}
template <size_t N>
static bool eatTerm(StringRef& mangledName, const char (&str)[N]) {
if (mangledName.startswith(StringRef(str, N-1))) {
drop_front(mangledName, N-1);
return true;
}
return false;
}
static int eatNumber(StringRef& s) {
size_t const savedSize = s.size();
int n = 0;
while (!s.empty() && isDigit(s.front())) {
n = n*10 + s.front() - '0';
drop_front(s);
}
return s.size() < savedSize ? n : -1;
}
static StringRef eatLengthPrefixedName(StringRef& mangledName) {
int const Len = eatNumber(mangledName);
if (Len <= 0 || static_cast<size_t>(Len) > mangledName.size())
return StringRef();
StringRef Res = mangledName.substr(0, Len);
drop_front(mangledName, Len);
return Res;
}
} // end anonymous namespace
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
AMDGPUMangledLibFunc::AMDGPUMangledLibFunc() {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
FuncId = EI_NONE;
FKind = NOPFX;
Leads[0].reset();
Leads[1].reset();
Name.clear();
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
AMDGPUUnmangledLibFunc::AMDGPUUnmangledLibFunc() {
FuncId = EI_NONE;
FuncTy = nullptr;
}
AMDGPUMangledLibFunc::AMDGPUMangledLibFunc(
EFuncId id, const AMDGPUMangledLibFunc &copyFrom) {
FuncId = id;
FKind = copyFrom.FKind;
Leads[0] = copyFrom.Leads[0];
Leads[1] = copyFrom.Leads[1];
}
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
///////////////////////////////////////////////////////////////////////////////
// Demangling
static int parseVecSize(StringRef& mangledName) {
size_t const Len = eatNumber(mangledName);
switch (Len) {
case 2: case 3: case 4: case 8: case 16:
return Len;
default:
break;
}
return 1;
}
static AMDGPULibFunc::ENamePrefix parseNamePrefix(StringRef& mangledName) {
std::pair<StringRef, StringRef> const P = mangledName.split('_');
AMDGPULibFunc::ENamePrefix Pfx =
StringSwitch<AMDGPULibFunc::ENamePrefix>(P.first)
.Case("native", AMDGPULibFunc::NATIVE)
.Case("half" , AMDGPULibFunc::HALF)
.Default(AMDGPULibFunc::NOPFX);
if (Pfx != AMDGPULibFunc::NOPFX)
mangledName = P.second;
return Pfx;
}
Delay initialization of three static global maps, NFC This avoids allocating a few KB of heap memory on startup, and instead allocates these maps lazily. I noticed this while profiling LLD. llvm-svn: 357192
2019-03-29 01:33:41 +08:00
StringMap<int> ManglingRule::buildManglingRulesMap() {
StringMap<int> Map(array_lengthof(manglingRules));
int Id = 0;
for (auto Rule : manglingRules)
Map.insert({Rule.Name, Id++});
return Map;
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
bool AMDGPUMangledLibFunc::parseUnmangledName(StringRef FullName) {
Delay initialization of three static global maps, NFC This avoids allocating a few KB of heap memory on startup, and instead allocates these maps lazily. I noticed this while profiling LLD. llvm-svn: 357192
2019-03-29 01:33:41 +08:00
static const StringMap<int> manglingRulesMap =
ManglingRule::buildManglingRulesMap();
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
FuncId = static_cast<EFuncId>(manglingRulesMap.lookup(FullName));
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
return FuncId != EI_NONE;
}
///////////////////////////////////////////////////////////////////////////////
// Itanium Demangling
Move helper classes into anonymous namespaces. No functionality change intended. llvm-svn: 311288
2017-08-20 21:03:48 +08:00
namespace {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
struct ItaniumParamParser {
AMDGPULibFunc::Param Prev;
bool parseItaniumParam(StringRef& param, AMDGPULibFunc::Param &res);
};
Move helper classes into anonymous namespaces. No functionality change intended. llvm-svn: 311288
2017-08-20 21:03:48 +08:00
} // namespace
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
bool ItaniumParamParser::parseItaniumParam(StringRef& param,
AMDGPULibFunc::Param &res) {
res.reset();
if (param.empty()) return false;
// parse pointer prefix
if (eatTerm(param, 'P')) {
if (eatTerm(param, 'K')) res.PtrKind |= AMDGPULibFunc::CONST;
if (eatTerm(param, 'V')) res.PtrKind |= AMDGPULibFunc::VOLATILE;
[AMDGPU] Remove hardcoded address space value from AMDGPULibFunc AMDGPULibFunc hardcodes address space values of the old address space mapping, which causes invalid addrspacecast instructions and undefined functions in APPSDK sample MonteCarloAsianDP. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39616 llvm-svn: 317409
2017-11-05 01:37:43 +08:00
unsigned AS;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
if (!eatTerm(param, "U3AS")) {
[AMDGPU] Remove hardcoded address space value from AMDGPULibFunc AMDGPULibFunc hardcodes address space values of the old address space mapping, which causes invalid addrspacecast instructions and undefined functions in APPSDK sample MonteCarloAsianDP. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39616 llvm-svn: 317409
2017-11-05 01:37:43 +08:00
AS = 0;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
} else {
[AMDGPU] Remove hardcoded address space value from AMDGPULibFunc AMDGPULibFunc hardcodes address space values of the old address space mapping, which causes invalid addrspacecast instructions and undefined functions in APPSDK sample MonteCarloAsianDP. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39616 llvm-svn: 317409
2017-11-05 01:37:43 +08:00
AS = param.front() - '0';
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
drop_front(param, 1);
}
[AMDGPU] Remove hardcoded address space value from AMDGPULibFunc AMDGPULibFunc hardcodes address space values of the old address space mapping, which causes invalid addrspacecast instructions and undefined functions in APPSDK sample MonteCarloAsianDP. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39616 llvm-svn: 317409
2017-11-05 01:37:43 +08:00
res.PtrKind |= AMDGPULibFuncBase::getEPtrKindFromAddrSpace(AS);
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
} else {
res.PtrKind = AMDGPULibFunc::BYVALUE;
}
// parse vector size
if (eatTerm(param,"Dv")) {
res.VectorSize = parseVecSize(param);
if (res.VectorSize==1 || !eatTerm(param, '_')) return false;
}
// parse type
char const TC = param.front();
[TypeSize] Allow returning scalable size in implicit conversion to uint64_t This patch removes compiler runtime assertions that ensure the implicit conversion are only guaranteed to work for fixed-width vectors. With the assert it would be impossible to get _anything_ to build until the entire codebase has been upgraded, even when the indiscriminate uses of the size as uint64_t would work fine for both scalable and fixed-width types. This issue will need to be addressed differently, with build-time errors rather than assertion failures, but that effort falls beyond the scope of this patch. Returning the scalable size and avoiding the assert in getFixedSize() is a temporary stop-gap in order to use LLVM for compiling and using the SVE ACLE intrinsics. Reviewers: efriedma, huntergr, rovka, ctetreau, rengolin Reviewed By: efriedma Tags: #llvm Differential Revision: https://reviews.llvm.org/D75297
2020-03-13 17:13:34 +08:00
if (isDigit(TC)) {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
res.ArgType = StringSwitch<AMDGPULibFunc::EType>
(eatLengthPrefixedName(param))
.Case("ocl_image1darray" , AMDGPULibFunc::IMG1DA)
.Case("ocl_image1dbuffer", AMDGPULibFunc::IMG1DB)
.Case("ocl_image2darray" , AMDGPULibFunc::IMG2DA)
.Case("ocl_image1d" , AMDGPULibFunc::IMG1D)
.Case("ocl_image2d" , AMDGPULibFunc::IMG2D)
.Case("ocl_image3d" , AMDGPULibFunc::IMG3D)
.Case("ocl_event" , AMDGPULibFunc::DUMMY)
.Case("ocl_sampler" , AMDGPULibFunc::DUMMY)
.Default(AMDGPULibFunc::DUMMY);
} else {
drop_front(param);
switch (TC) {
case 'h': res.ArgType = AMDGPULibFunc::U8; break;
case 't': res.ArgType = AMDGPULibFunc::U16; break;
case 'j': res.ArgType = AMDGPULibFunc::U32; break;
case 'm': res.ArgType = AMDGPULibFunc::U64; break;
case 'c': res.ArgType = AMDGPULibFunc::I8; break;
case 's': res.ArgType = AMDGPULibFunc::I16; break;
case 'i': res.ArgType = AMDGPULibFunc::I32; break;
case 'l': res.ArgType = AMDGPULibFunc::I64; break;
case 'f': res.ArgType = AMDGPULibFunc::F32; break;
case 'd': res.ArgType = AMDGPULibFunc::F64; break;
case 'D': if (!eatTerm(param, 'h')) return false;
res.ArgType = AMDGPULibFunc::F16; break;
case 'S':
if (!eatTerm(param, '_')) {
eatNumber(param);
if (!eatTerm(param, '_')) return false;
}
res.VectorSize = Prev.VectorSize;
res.ArgType = Prev.ArgType;
break;
default:;
}
}
if (res.ArgType == 0) return false;
Prev.VectorSize = res.VectorSize;
Prev.ArgType = res.ArgType;
return true;
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
bool AMDGPUMangledLibFunc::parseFuncName(StringRef &mangledName) {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
StringRef Name = eatLengthPrefixedName(mangledName);
FKind = parseNamePrefix(Name);
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
if (!parseUnmangledName(Name))
return false;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
const ManglingRule& Rule = manglingRules[FuncId];
ItaniumParamParser Parser;
for (int I=0; I < Rule.maxLeadIndex(); ++I) {
Param P;
if (!Parser.parseItaniumParam(mangledName, P))
return false;
if ((I + 1) == Rule.Lead[0]) Leads[0] = P;
if ((I + 1) == Rule.Lead[1]) Leads[1] = P;
}
return true;
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
bool AMDGPUUnmangledLibFunc::parseFuncName(StringRef &Name) {
if (!UnmangledFuncInfo::lookup(Name, FuncId))
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
return false;
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
setName(Name);
return true;
}
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
bool AMDGPULibFunc::parse(StringRef FuncName, AMDGPULibFunc &F) {
if (FuncName.empty()) {
F.Impl = std::unique_ptr<AMDGPULibFuncImpl>();
return false;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
if (eatTerm(FuncName, "_Z"))
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
F.Impl = std::make_unique<AMDGPUMangledLibFunc>();
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
else
[llvm] Migrate llvm::make_unique to std::make_unique Now that we've moved to C++14, we no longer need the llvm::make_unique implementation from STLExtras.h. This patch is a mechanical replacement of (hopefully) all the llvm::make_unique instances across the monorepo. llvm-svn: 369013
2019-08-15 23:54:37 +08:00
F.Impl = std::make_unique<AMDGPUUnmangledLibFunc>();
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
if (F.Impl->parseFuncName(FuncName))
return true;
F.Impl = std::unique_ptr<AMDGPULibFuncImpl>();
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
return false;
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
StringRef AMDGPUMangledLibFunc::getUnmangledName(StringRef mangledName) {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
StringRef S = mangledName;
if (eatTerm(S, "_Z"))
return eatLengthPrefixedName(S);
return StringRef();
}
///////////////////////////////////////////////////////////////////////////////
// Mangling
template <typename Stream>
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
void AMDGPUMangledLibFunc::writeName(Stream &OS) const {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
const char *Pfx = "";
switch (FKind) {
case NATIVE: Pfx = "native_"; break;
case HALF: Pfx = "half_"; break;
default: break;
}
if (!Name.empty()) {
OS << Pfx << Name;
} else if (FuncId != EI_NONE) {
OS << Pfx;
const StringRef& S = manglingRules[FuncId].Name;
OS.write(S.data(), S.size());
}
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
std::string AMDGPUMangledLibFunc::mangle() const { return mangleNameItanium(); }
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
///////////////////////////////////////////////////////////////////////////////
// Itanium Mangling
static const char *getItaniumTypeName(AMDGPULibFunc::EType T) {
switch (T) {
case AMDGPULibFunc::U8: return "h";
case AMDGPULibFunc::U16: return "t";
case AMDGPULibFunc::U32: return "j";
case AMDGPULibFunc::U64: return "m";
case AMDGPULibFunc::I8: return "c";
case AMDGPULibFunc::I16: return "s";
case AMDGPULibFunc::I32: return "i";
case AMDGPULibFunc::I64: return "l";
case AMDGPULibFunc::F16: return "Dh";
case AMDGPULibFunc::F32: return "f";
case AMDGPULibFunc::F64: return "d";
case AMDGPULibFunc::IMG1DA: return "16ocl_image1darray";
case AMDGPULibFunc::IMG1DB: return "17ocl_image1dbuffer";
case AMDGPULibFunc::IMG2DA: return "16ocl_image2darray";
case AMDGPULibFunc::IMG1D: return "11ocl_image1d";
case AMDGPULibFunc::IMG2D: return "11ocl_image2d";
case AMDGPULibFunc::IMG3D: return "11ocl_image3d";
case AMDGPULibFunc::SAMPLER: return "11ocl_sampler";
case AMDGPULibFunc::EVENT: return "9ocl_event";
default: llvm_unreachable("Unhandeled param type");
}
return nullptr;
}
Move helper classes into anonymous namespaces. No functionality change intended. llvm-svn: 311288
2017-08-20 21:03:48 +08:00
namespace {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
// Itanium mangling ABI says:
// "5.1.8. Compression
// ... Each non-terminal in the grammar for which <substitution> appears on the
// right-hand side is both a source of future substitutions and a candidate
// for being substituted. There are two exceptions that appear to be
// substitution candidates from the grammar, but are explicitly excluded:
// 1. <builtin-type> other than vendor extended types ..."
// For the purpose of functions the following productions make sence for the
// substitution:
// <type> ::= <builtin-type>
// ::= <class-enum-type>
// ::= <array-type>
// ::=<CV-qualifiers> <type>
// ::= P <type> # pointer-to
// ::= <substitution>
//
// Note that while types like images, samplers and events are by the ABI encoded
// using <class-enum-type> production rule they're not used for substitution
// because clang consider them as builtin types.
//
// DvNN_ type is GCC extension for vectors and is a subject for the substitution.
class ItaniumMangler {
SmallVector<AMDGPULibFunc::Param, 10> Str; // list of accumulated substituions
bool UseAddrSpace;
int findSubst(const AMDGPULibFunc::Param& P) const {
for(unsigned I = 0; I < Str.size(); ++I) {
const AMDGPULibFunc::Param& T = Str[I];
if (P.PtrKind == T.PtrKind &&
P.VectorSize == T.VectorSize &&
P.ArgType == T.ArgType) {
return I;
}
}
return -1;
}
template <typename Stream>
bool trySubst(Stream& os, const AMDGPULibFunc::Param& p) {
int const subst = findSubst(p);
if (subst < 0) return false;
// Substitutions are mangled as S(XX)?_ where XX is a hexadecimal number
// 0 1 2
// S_ S0_ S1_
if (subst == 0) os << "S_";
else os << 'S' << (subst-1) << '_';
return true;
}
public:
ItaniumMangler(bool useAddrSpace)
: UseAddrSpace(useAddrSpace) {}
template <typename Stream>
void operator()(Stream& os, AMDGPULibFunc::Param p) {
// Itanium mangling ABI 5.1.8. Compression:
// Logically, the substitutable components of a mangled name are considered
// left-to-right, components before the composite structure of which they
// are a part. If a component has been encountered before, it is substituted
// as described below. This decision is independent of whether its components
// have been substituted, so an implementation may optimize by considering
// large structures for substitution before their components. If a component
// has not been encountered before, its mangling is identified, and it is
// added to a dictionary of substitution candidates. No entity is added to
// the dictionary twice.
AMDGPULibFunc::Param Ptr;
if (p.PtrKind) {
if (trySubst(os, p)) return;
os << 'P';
if (p.PtrKind & AMDGPULibFunc::CONST) os << 'K';
if (p.PtrKind & AMDGPULibFunc::VOLATILE) os << 'V';
[AMDGPU] Remove hardcoded address space value from AMDGPULibFunc AMDGPULibFunc hardcodes address space values of the old address space mapping, which causes invalid addrspacecast instructions and undefined functions in APPSDK sample MonteCarloAsianDP. This patch fixes that. Differential Revision: https://reviews.llvm.org/D39616 llvm-svn: 317409
2017-11-05 01:37:43 +08:00
unsigned AS = UseAddrSpace
? AMDGPULibFuncBase::getAddrSpaceFromEPtrKind(p.PtrKind)
: 0;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
if (AS != 0) os << "U3AS" << AS;
Ptr = p;
p.PtrKind = 0;
}
if (p.VectorSize > 1) {
if (trySubst(os, p)) goto exit;
Str.push_back(p);
os << "Dv" << static_cast<unsigned>(p.VectorSize) << '_';
}
os << getItaniumTypeName((AMDGPULibFunc::EType)p.ArgType);
exit:
if (Ptr.ArgType) Str.push_back(Ptr);
}
};
Move helper classes into anonymous namespaces. No functionality change intended. llvm-svn: 311288
2017-08-20 21:03:48 +08:00
} // namespace
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
std::string AMDGPUMangledLibFunc::mangleNameItanium() const {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
SmallString<128> Buf;
raw_svector_ostream S(Buf);
SmallString<128> NameBuf;
raw_svector_ostream Name(NameBuf);
writeName(Name);
const StringRef& NameStr = Name.str();
S << "_Z" << static_cast<int>(NameStr.size()) << NameStr;
ItaniumMangler Mangler(true);
ParamIterator I(Leads, manglingRules[FuncId]);
Param P;
while ((P = I.getNextParam()).ArgType != 0)
Mangler(S, P);
Make llvm::StringRef to std::string conversions explicit. This is how it should've been and brings it more in line with std::string_view. There should be no functional change here. This is mostly mechanical from a custom clang-tidy check, with a lot of manual fixups. It uncovers a lot of minor inefficiencies. This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-29 03:23:46 +08:00
return std::string(S.str());
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
}
///////////////////////////////////////////////////////////////////////////////
// Misc
static Type* getIntrinsicParamType(
LLVMContext& C,
const AMDGPULibFunc::Param& P,
bool useAddrSpace) {
Type* T = nullptr;
switch (P.ArgType) {
case AMDGPULibFunc::U8:
case AMDGPULibFunc::I8: T = Type::getInt8Ty(C); break;
case AMDGPULibFunc::U16:
case AMDGPULibFunc::I16: T = Type::getInt16Ty(C); break;
case AMDGPULibFunc::U32:
case AMDGPULibFunc::I32: T = Type::getInt32Ty(C); break;
case AMDGPULibFunc::U64:
case AMDGPULibFunc::I64: T = Type::getInt64Ty(C); break;
case AMDGPULibFunc::F16: T = Type::getHalfTy(C); break;
case AMDGPULibFunc::F32: T = Type::getFloatTy(C); break;
case AMDGPULibFunc::F64: T = Type::getDoubleTy(C); break;
case AMDGPULibFunc::IMG1DA:
case AMDGPULibFunc::IMG1DB:
case AMDGPULibFunc::IMG2DA:
case AMDGPULibFunc::IMG1D:
case AMDGPULibFunc::IMG2D:
case AMDGPULibFunc::IMG3D:
T = StructType::create(C,"ocl_image")->getPointerTo(); break;
case AMDGPULibFunc::SAMPLER:
T = StructType::create(C,"ocl_sampler")->getPointerTo(); break;
case AMDGPULibFunc::EVENT:
T = StructType::create(C,"ocl_event")->getPointerTo(); break;
default:
llvm_unreachable("Unhandeled param type");
return nullptr;
}
if (P.VectorSize > 1)
[SVE] Eliminate calls to default-false VectorType::get() from AMDGPU Reviewers: efriedma, david-arm, fpetrogalli, arsenm Reviewed By: david-arm Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, tschuett, hiraditya, rkruppe, psnobl, kerbowa, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D80328
2020-05-30 08:44:51 +08:00
T = FixedVectorType::get(T, P.VectorSize);
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
if (P.PtrKind != AMDGPULibFunc::BYVALUE)
T = useAddrSpace ? T->getPointerTo((P.PtrKind & AMDGPULibFunc::ADDR_SPACE)
- 1)
: T->getPointerTo();
return T;
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
FunctionType *AMDGPUMangledLibFunc::getFunctionType(Module &M) const {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
LLVMContext& C = M.getContext();
std::vector<Type*> Args;
ParamIterator I(Leads, manglingRules[FuncId]);
Param P;
while ((P=I.getNextParam()).ArgType != 0)
Args.push_back(getIntrinsicParamType(C, P, true));
return FunctionType::get(
getIntrinsicParamType(C, getRetType(FuncId, Leads), true),
Args, false);
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
unsigned AMDGPUMangledLibFunc::getNumArgs() const {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
return manglingRules[FuncId].getNumArgs();
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
unsigned AMDGPUUnmangledLibFunc::getNumArgs() const {
return UnmangledFuncInfo::getNumArgs(FuncId);
}
std::string AMDGPUMangledLibFunc::getName() const {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
SmallString<128> Buf;
raw_svector_ostream OS(Buf);
writeName(OS);
Make llvm::StringRef to std::string conversions explicit. This is how it should've been and brings it more in line with std::string_view. There should be no functional change here. This is mostly mechanical from a custom clang-tidy check, with a lot of manual fixups. It uncovers a lot of minor inefficiencies. This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-29 03:23:46 +08:00
return std::string(OS.str());
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
Function *AMDGPULibFunc::getFunction(Module *M, const AMDGPULibFunc &fInfo) {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
std::string FuncName = fInfo.mangle();
Function *F = dyn_cast_or_null<Function>(
M->getValueSymbolTable().lookup(FuncName));
// check formal with actual types conformance
if (F && !F->isDeclaration()
&& !F->isVarArg()
&& F->arg_size() == fInfo.getNumArgs()) {
return F;
}
return nullptr;
}
[opaque pointer types] Add a FunctionCallee wrapper type, and use it. Recommit r352791 after tweaking DerivedTypes.h slightly, so that gcc doesn't choke on it, hopefully. Original Message: The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352827
2019-02-01 10:28:03 +08:00
FunctionCallee AMDGPULibFunc::getOrInsertFunction(Module *M,
const AMDGPULibFunc &fInfo) {
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
std::string const FuncName = fInfo.mangle();
Function *F = dyn_cast_or_null<Function>(
M->getValueSymbolTable().lookup(FuncName));
// check formal with actual types conformance
if (F && !F->isDeclaration()
&& !F->isVarArg()
&& F->arg_size() == fInfo.getNumArgs()) {
return F;
}
FunctionType *FuncTy = fInfo.getFunctionType(*M);
bool hasPtr = false;
for (FunctionType::param_iterator
PI = FuncTy->param_begin(),
PE = FuncTy->param_end();
PI != PE; ++PI) {
const Type* argTy = static_cast<const Type*>(*PI);
if (argTy->isPointerTy()) {
hasPtr = true;
break;
}
}
[opaque pointer types] Add a FunctionCallee wrapper type, and use it. Recommit r352791 after tweaking DerivedTypes.h slightly, so that gcc doesn't choke on it, hopefully. Original Message: The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352827
2019-02-01 10:28:03 +08:00
FunctionCallee C;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
if (hasPtr) {
// Do not set extra attributes for functions with pointer arguments.
C = M->getOrInsertFunction(FuncName, FuncTy);
} else {
AttributeList Attr;
LLVMContext &Ctx = M->getContext();
[AMDGPU] Fix discarded result of addAttribute Summary: `llvm::AttributeList` and `llvm::AttributeSet` are immutable, and so methods defined on these classes, such as `addAttribute`, return a new immutable object with the attribute added. In https://reviews.llvm.org/D55217 I attempted to annotate methods such as `addAttribute` with `LLVM_NODISCARD`, since calling these methods has no side-effects, and so ignoring the result that is returned is almost certainly a programmer error. However, committing the change resulted in new warnings in the AMDGPU target. The AMDGPU simplify libcalls pass added in https://reviews.llvm.org/D36436 attempts to add the readonly and nounwind attributes to simplified library functions, but instead calls the `addAttribute` methods and ignores the result. Modify the simplify libcalls pass to actually add the nounwind and readonly attributes. Also update the simplify libcalls test to assert that these attributes are actually being set. Reviewers: rampitec, vpykhtin, rnk Reviewed By: rampitec Subscribers: arsenm, kzhuravl, jvesely, wdng, nhaehnle, yaxunl, dstuttard, tpr, t-tye, llvm-commits Differential Revision: https://reviews.llvm.org/D55435 llvm-svn: 348732
2018-12-10 05:56:50 +08:00
Attr = Attr.addAttribute(Ctx, AttributeList::FunctionIndex,
Attribute::ReadOnly);
Attr = Attr.addAttribute(Ctx, AttributeList::FunctionIndex,
Attribute::NoUnwind);
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
C = M->getOrInsertFunction(FuncName, FuncTy, Attr);
}
[opaque pointer types] Add a FunctionCallee wrapper type, and use it. Recommit r352791 after tweaking DerivedTypes.h slightly, so that gcc doesn't choke on it, hopefully. Original Message: The FunctionCallee type is effectively a {FunctionType*,Value*} pair, and is a useful convenience to enable code to continue passing the result of getOrInsertFunction() through to EmitCall, even once pointer types lose their pointee-type. Then: - update the CallInst/InvokeInst instruction creation functions to take a Callee, - modify getOrInsertFunction to return FunctionCallee, and - update all callers appropriately. One area of particular note is the change to the sanitizer code. Previously, they had been casting the result of `getOrInsertFunction` to a `Function*` via `checkSanitizerInterfaceFunction`, and storing that. That would report an error if someone had already inserted a function declaraction with a mismatching signature. However, in general, LLVM allows for such mismatches, as `getOrInsertFunction` will automatically insert a bitcast if needed. As part of this cleanup, cause the sanitizer code to do the same. (It will call its functions using the expected signature, however they may have been declared.) Finally, in a small number of locations, callers of `getOrInsertFunction` actually were expecting/requiring that a brand new function was being created. In such cases, I've switched them to Function::Create instead. Differential Revision: https://reviews.llvm.org/D57315 llvm-svn: 352827
2019-02-01 10:28:03 +08:00
return C;
[AMDGPU] Ported and adopted AMDLibCalls pass The pass does simplifications of well known AMD library calls. If given -amdgpu-prelink option it works in a pre-link mode which allows to reference new library functions which will be linked in later. In addition it also used to process traditional AMD option -fuse-native which allows to replace some of the functions with their fast native implementations from the library. The necessary glue to pass the prelink option and translate -fuse-native is to be added to the driver. Differential Revision: https://reviews.llvm.org/D36436 llvm-svn: 310731
2017-08-12 00:42:09 +08:00
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
Delay initialization of three static global maps, NFC This avoids allocating a few KB of heap memory on startup, and instead allocates these maps lazily. I noticed this while profiling LLD. llvm-svn: 357192
2019-03-29 01:33:41 +08:00
StringMap<unsigned> UnmangledFuncInfo::buildNameMap() {
StringMap<unsigned> Map;
for (unsigned I = 0; I != TableSize; ++I)
Map[Table[I].Name] = I;
return Map;
}
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
bool UnmangledFuncInfo::lookup(StringRef Name, ID &Id) {
Delay initialization of three static global maps, NFC This avoids allocating a few KB of heap memory on startup, and instead allocates these maps lazily. I noticed this while profiling LLD. llvm-svn: 357192
2019-03-29 01:33:41 +08:00
static const StringMap<unsigned> Map = buildNameMap();
[AMDGPU] Transform __read_pipe_* and __write_pipe_* When packet size equals packet align and is power of 2, transform __read_pipe* and __write_pipe* to specialized library function. Differential Revision: https://reviews.llvm.org/D36831 llvm-svn: 312598
2017-09-06 08:30:27 +08:00
auto Loc = Map.find(Name);
if (Loc != Map.end()) {
Id = toFuncId(Loc->second);
return true;
}
Id = AMDGPULibFunc::EI_NONE;
return false;
}
AMDGPULibFunc::AMDGPULibFunc(const AMDGPULibFunc &F) {
if (auto *MF = dyn_cast<AMDGPUMangledLibFunc>(F.Impl.get()))
Impl.reset(new AMDGPUMangledLibFunc(*MF));
else if (auto *UMF = dyn_cast<AMDGPUUnmangledLibFunc>(F.Impl.get()))
Impl.reset(new AMDGPUUnmangledLibFunc(*UMF));
else
Impl = std::unique_ptr<AMDGPULibFuncImpl>();
}
AMDGPULibFunc &AMDGPULibFunc::operator=(const AMDGPULibFunc &F) {
if (this == &F)
return *this;
new (this) AMDGPULibFunc(F);
return *this;
}
AMDGPULibFunc::AMDGPULibFunc(EFuncId Id, const AMDGPULibFunc &CopyFrom) {
assert(AMDGPULibFuncBase::isMangled(Id) && CopyFrom.isMangled() &&
"not supported");
Impl.reset(new AMDGPUMangledLibFunc(
Id, *cast<AMDGPUMangledLibFunc>(CopyFrom.Impl.get())));
}
AMDGPULibFunc::AMDGPULibFunc(StringRef Name, FunctionType *FT) {
Impl.reset(new AMDGPUUnmangledLibFunc(Name, FT));
}
void AMDGPULibFunc::initMangled() { Impl.reset(new AMDGPUMangledLibFunc()); }
AMDGPULibFunc::Param *AMDGPULibFunc::getLeads() {
if (!Impl)
initMangled();
return cast<AMDGPUMangledLibFunc>(Impl.get())->Leads;
}
const AMDGPULibFunc::Param *AMDGPULibFunc::getLeads() const {
return cast<const AMDGPUMangledLibFunc>(Impl.get())->Leads;
}