maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/Target/AMDGPU/SIISelLowering.h

272 lines
12 KiB
C
Raw Normal View History

Add R600 backend A new backend supporting AMD GPUs: Radeon HD2XXX - HD7XXX llvm-svn: 169915
2012-12-12 05:25:42 +08:00
//===-- SIISelLowering.h - SI DAG Lowering Interface ------------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
//
/// \file
/// \brief SI DAG Lowering interface definition
//
//===----------------------------------------------------------------------===//
AMDGPU: R600 code splitting cleanup Move a few functions only used by R600 to R600 specific code, fix header macros to stop using R600, mark classes as final. llvm-svn: 263204
2016-03-11 16:00:27 +08:00
#ifndef LLVM_LIB_TARGET_AMDGPU_SIISELLOWERING_H
#define LLVM_LIB_TARGET_AMDGPU_SIISELLOWERING_H
Add R600 backend A new backend supporting AMD GPUs: Radeon HD2XXX - HD7XXX llvm-svn: 169915
2012-12-12 05:25:42 +08:00
#include "AMDGPUISelLowering.h"
AMDGPU: Pass special input registers to functions llvm-svn: 309998
2017-08-04 07:00:29 +08:00
#include "AMDGPUArgumentUsageInfo.h"
Add R600 backend A new backend supporting AMD GPUs: Radeon HD2XXX - HD7XXX llvm-svn: 169915
2012-12-12 05:25:42 +08:00
#include "SIInstrInfo.h"
namespace llvm {
AMDGPU: R600 code splitting cleanup Move a few functions only used by R600 to R600 specific code, fix header macros to stop using R600, mark classes as final. llvm-svn: 263204
2016-03-11 16:00:27 +08:00
class SITargetLowering final : public AMDGPUTargetLowering {
AMDGPU: Refactor argument lowering Split into smaller functions and prepare for handling non-entry functions. llvm-svn: 299998
2017-04-12 06:29:24 +08:00
SDValue lowerKernArgParameterPtr(SelectionDAG &DAG, const SDLoc &SL,
SDValue Chain, uint64_t Offset) const;
AMDGPU: Annotate implicitarg.ptr usage We need to pass something to functions for this to work. It isn't derivable just from the kernarg segment pointer because the implicit arguments are placed after the kernel arguments. Also fixes missing test for the intrinsic. llvm-svn: 309398
2017-07-28 23:52:08 +08:00
SDValue getImplicitArgPtr(SelectionDAG &DAG, const SDLoc &SL) const;
AMDGPU: Refactor argument lowering Split into smaller functions and prepare for handling non-entry functions. llvm-svn: 299998
2017-04-12 06:29:24 +08:00
SDValue lowerKernargMemParameter(SelectionDAG &DAG, EVT VT, EVT MemVT,
const SDLoc &SL, SDValue Chain,
uint64_t Offset, bool Signed,
const ISD::InputArg *Arg = nullptr) const;
AMDGPU: Start defining a calling convention Partially implement callee-side for arguments and return values. byval doesn't work properly, and most likely sret or other on-stack return values most as well. llvm-svn: 303308
2017-05-18 05:56:25 +08:00
SDValue lowerStackParameter(SelectionDAG &DAG, CCValAssign &VA,
const SDLoc &SL, SDValue Chain,
const ISD::InputArg &Arg) const;
AMDGPU: Pass special input registers to functions llvm-svn: 309998
2017-08-04 07:00:29 +08:00
SDValue getPreloadedValue(SelectionDAG &DAG,
const SIMachineFunctionInfo &MFI,
EVT VT,
AMDGPUFunctionArgInfo::PreloadedValue) const;
AMDGPU: Start defining a calling convention Partially implement callee-side for arguments and return values. byval doesn't work properly, and most likely sret or other on-stack return values most as well. llvm-svn: 303308
2017-05-18 05:56:25 +08:00
AMDGPU/SI: Refactor fixup handling for constant addrspace variables Summary: We now use a standard fixup type applying the pc-relative address of constant address space variables, and we have the GlobalAddress lowering code add the required 4 byte offset to the global address rather than doing it as part of the fixup. This refactoring will make it easier to use the same code for global address space variables and also simplifies the code. Re-commit this after fixing a bug where we were trying to use a reference to a Triple object that had already been destroyed. Reviewers: arsenm, kzhuravl Subscribers: arsenm, kzhuravl, llvm-commits Differential Revision: http://reviews.llvm.org/D21154 llvm-svn: 272705
2016-06-15 04:29:59 +08:00
SDValue LowerGlobalAddress(AMDGPUMachineFunction *MFI, SDValue Op,
SelectionDAG &DAG) const override;
AMDGPU: Use assert zext for workgroup sizes llvm-svn: 254328
2015-12-01 05:15:45 +08:00
SDValue lowerImplicitZextParam(SelectionDAG &DAG, SDValue Op,
MVT VT, unsigned Offset) const;
R600: Move intrinsic lowering to separate functions llvm-svn: 214023
2014-07-26 14:23:37 +08:00
SDValue LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG) const;
AMDGPU: Add atomic_inc + atomic_dec intrinsics These are different than atomicrmw add 1 because they have an additional input value to clamp the result. llvm-svn: 266074
2016-04-12 22:05:04 +08:00
SDValue LowerINTRINSIC_W_CHAIN(SDValue Op, SelectionDAG &DAG) const;
R600: Move intrinsic lowering to separate functions llvm-svn: 214023
2014-07-26 14:23:37 +08:00
SDValue LowerINTRINSIC_VOID(SDValue Op, SelectionDAG &DAG) const;
R600/SI: Add support for private address space load/store Private address space is emulated using the register file with MOVRELS and MOVRELD instructions. llvm-svn: 194626
2013-11-14 07:36:50 +08:00
SDValue LowerLOAD(SDValue Op, SelectionDAG &DAG) const;
R600/SI: Custom lower i64 ISD::SELECT llvm-svn: 200774
2014-02-05 01:18:40 +08:00
SDValue LowerSELECT(SDValue Op, SelectionDAG &DAG) const;
AMDGPU: Change fdiv lowering based on !fpmath metadata If 2.5 ulp is acceptable, denormals are not required, and isn't a reciprocal which will already be handled, replace with a faster fdiv. Simplify the lowering tests by using per function subtarget features. llvm-svn: 276051
2016-07-20 07:16:53 +08:00
SDValue lowerFastUnsafeFDIV(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerFDIV_FAST(SDValue Op, SelectionDAG &DAG) const;
AMDGPU: Custom lower f16 fdiv llvm-svn: 290301
2016-12-22 11:05:41 +08:00
SDValue LowerFDIV16(SDValue Op, SelectionDAG &DAG) const;
R600/SI: Implement less wrong f32 fdiv Assuming single precision denormals and accurate sqrt/div are not reported, this passes the OpenCL conformance test. llvm-svn: 213089
2014-07-16 04:18:31 +08:00
SDValue LowerFDIV32(SDValue Op, SelectionDAG &DAG) const;
SDValue LowerFDIV64(SDValue Op, SelectionDAG &DAG) const;
SDValue LowerFDIV(SDValue Op, SelectionDAG &DAG) const;
R600: Custom lower [s|u]int_to_fp for i64 -> f64 llvm-svn: 219037
2014-10-04 07:54:41 +08:00
SDValue LowerINT_TO_FP(SDValue Op, SelectionDAG &DAG, bool Signed) const;
R600/SI: Add support for private address space load/store Private address space is emulated using the register file with MOVRELS and MOVRELD instructions. llvm-svn: 194626
2013-11-14 07:36:50 +08:00
SDValue LowerSTORE(SDValue Op, SelectionDAG &DAG) const;
R600/SI: implement range reduction for sin/cos These instructions can only take a limited input range, and return the constant value 1 out of range. We should do range reduction to be able to process arbitrary values. Use a FRACT instruction after normalization to achieve this. Also add a test for constant folding with the lowered code with unsafe-fp-math enabled. v2: use DAG lowering instead of intrinsic, adapt test v3: calculate constant, fold pattern into instruction definition v4: misc style fixes, add sin-fold testcase, cosmetics Patch by Grigori Goronzy llvm-svn: 213458
2014-07-20 02:44:39 +08:00
SDValue LowerTrig(SDValue Op, SelectionDAG &DAG) const;
AMDGPU: Implement {BUFFER,FLAT}_ATOMIC_CMPSWAP{,_X2} Summary: Implement BUFFER_ATOMIC_CMPSWAP{,_X2} instructions on all GCN targets, and FLAT_ATOMIC_CMPSWAP{,_X2} on CI+. 32-bit instruction variants tested manually on Kabini and Bonaire. Tests and parts of code provided by Jan Veselý. Patch by: Vedran Miletić Reviewers: arsenm, tstellarAMD, nhaehnle Subscribers: jvesely, scchan, kanarayan, arsenm Differential Revision: http://reviews.llvm.org/D17280 llvm-svn: 265170
2016-04-02 02:27:37 +08:00
SDValue LowerATOMIC_CMP_SWAP(SDValue Op, SelectionDAG &DAG) const;
R600: New control flow for SI v2 This patch replaces the control flow handling with a new pass which structurize the graph before transforming it to machine instruction. This has a couple of different advantages and currently fixes 20 piglit tests without a single regression. It is now a general purpose transformation that could be not only be used for SI/R6xx, but also for other hardware implementations that use a form of structurized control flow. v2: further cleanup, fixes and documentation Patch by: Christian König Signed-off-by: Christian König <deathsimple@vodafone.de> Reviewed-by: Tom Stellard <thomas.stellard@amd.com> Tested-by: Michel Dänzer <michel.daenzer@amd.com> llvm-svn: 170591
2012-12-20 06:10:31 +08:00
SDValue LowerBRCOND(SDValue Op, SelectionDAG &DAG) const;
Add R600 backend A new backend supporting AMD GPUs: Radeon HD2XXX - HD7XXX llvm-svn: 169915
2012-12-12 05:25:42 +08:00
[AMDGPU] Add f16 support (VI+) Differential Revision: https://reviews.llvm.org/D25975 llvm-svn: 286753
2016-11-13 15:01:11 +08:00
/// \brief Converts \p Op, which must be of floating point type, to the
/// floating point type \p VT, by either extending or truncating it.
SDValue getFPExtOrFPTrunc(SelectionDAG &DAG,
SDValue Op,
const SDLoc &DL,
EVT VT) const;
AMDGPU: Refactor argument lowering Split into smaller functions and prepare for handling non-entry functions. llvm-svn: 299998
2017-04-12 06:29:24 +08:00
SDValue convertArgType(
SelectionDAG &DAG, EVT VT, EVT MemVT, const SDLoc &SL, SDValue Val,
bool Signed, const ISD::InputArg *Arg = nullptr) const;
[AMDGPU] Custom lower f16 = fp_round f64 llvm-svn: 287203
2016-11-17 12:28:37 +08:00
/// \brief Custom lowering for ISD::FP_ROUND for MVT::f16.
SDValue lowerFP_ROUND(SDValue Op, SelectionDAG &DAG) const;
AMDGPU/GFX9: Fix shared and private aperture queries Differential Revision: https://reviews.llvm.org/D31786 llvm-svn: 299727
2017-04-07 07:02:33 +08:00
SDValue getSegmentAperture(unsigned AS, const SDLoc &DL,
SelectionDAG &DAG) const;
AMDGPU: Implement addrspacecast llvm-svn: 267452
2016-04-26 03:27:24 +08:00
SDValue lowerADDRSPACECAST(SDValue Op, SelectionDAG &DAG) const;
AMDGPU: Custom lower more vector operations This avoids stack usage. llvm-svn: 292846
2017-01-24 07:09:58 +08:00
SDValue lowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerEXTRACT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
AMDGPU: Temporarily select trap to s_endpgm This should select to s_trap, but that requires additonal work to setup and enable the trap handler. For now emit s_endpgm so bugpoint stops getting stuck on the unsupported call to abort. Emit a warning that this will only terminate the wave and not really trap. llvm-svn: 273062
2016-06-18 06:27:03 +08:00
SDValue lowerTRAP(SDValue Op, SelectionDAG &DAG) const;
AMDGPU: Implement addrspacecast llvm-svn: 267452
2016-04-26 03:27:24 +08:00
R600/SI: adjust writemask to only the used components Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Michel Dänzer <michel.daenzer@amd.com> llvm-svn: 179165
2013-04-10 16:39:08 +08:00
void adjustWritemask(MachineSDNode *&N, SelectionDAG &DAG) const;
R600/SI: Fix bad code with unaligned byte vector loads Don't do the v4i8 -> v4f32 combine if the load will need to be expanded due to alignment. This stops adding instructions to repack into a single register that the v_cvt_ubyteN_f32 instructions read. llvm-svn: 225926
2015-01-14 09:35:22 +08:00
SDValue performUCharToFloatCombine(SDNode *N,
DAGCombinerInfo &DCI) const;
R600/SI: Fix offset folding in some cases with shifted pointers. Ordinarily (shl (add x, c1), c2) -> (add (shl x, c2), c1 << c2) is only done if the add has one use. If the resulting constant add can be folded into an addressing mode, force this to happen for the pointer operand. This ends up happening a lot because of how LDS objects are allocated. Since the globals are allocated next to each other, acessing the first element of the second object is directly indexed by a shifted pointer. llvm-svn: 215739
2014-08-16 01:49:05 +08:00
SDValue performSHLPtrCombine(SDNode *N,
unsigned AS,
DAGCombinerInfo &DCI) const;
AMDGPU: Improve splitting 64-bit bit ops by constants This addresses a TODO to handle operations besides and. This also starts eliminating no-op operations with a constant that can emerge later. llvm-svn: 281488
2016-09-14 23:19:03 +08:00
AMDGPU: Move combines into separate functions llvm-svn: 290309
2016-12-22 11:44:42 +08:00
SDValue performMemSDNodeCombine(MemSDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Improve splitting 64-bit bit ops by constants This addresses a TODO to handle operations besides and. This also starts eliminating no-op operations with a constant that can emerge later. llvm-svn: 281488
2016-09-14 23:19:03 +08:00
SDValue splitBinaryBitConstantOp(DAGCombinerInfo &DCI, const SDLoc &SL,
unsigned Opc, SDValue LHS,
const ConstantSDNode *CRHS) const;
R600/SI: Add combine for isinfinite pattern llvm-svn: 225310
2015-01-07 07:00:46 +08:00
SDValue performAndCombine(SDNode *N, DAGCombinerInfo &DCI) const;
R600/SI: Add basic DAG combines for fp_class llvm-svn: 225306
2015-01-07 07:00:39 +08:00
SDValue performOrCombine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Improve splitting 64-bit bit ops by constants This addresses a TODO to handle operations besides and. This also starts eliminating no-op operations with a constant that can emerge later. llvm-svn: 281488
2016-09-14 23:19:03 +08:00
SDValue performXorCombine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Remove unnecessary ands when f16 is legal Add a new node to act as a fancy bitcast from f16 operations to i32 that implicitly zero the high 16-bits of the result. Alternatively could try making v2f16 legal and canonicalizing on build_vectors. llvm-svn: 299246
2017-04-01 03:53:03 +08:00
SDValue performZeroExtendCombine(SDNode *N, DAGCombinerInfo &DCI) const;
R600/SI: Add basic DAG combines for fp_class llvm-svn: 225306
2015-01-07 07:00:39 +08:00
SDValue performClassCombine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Implement canonicalize Also add generic DAG node for it. llvm-svn: 266272
2016-04-14 09:42:16 +08:00
SDValue performFCanonicalizeCombine(SDNode *N, DAGCombinerInfo &DCI) const;
R600/SI: Use v_cvt_f32_ubyte* instructions This eliminates extra extract instructions when loading an i8 vector to a float vector. llvm-svn: 210666
2014-06-12 01:50:44 +08:00
AMDGPU: Redefine clamp node as clamp 0.0-1.0 Change implementation to use max instead of add. min/max/med3 do not flush denormals regardless of the mode, so it is OK to use it whether or not they are enabled. Also allow using clamp with f16, and use knowledge of dx10_clamp. llvm-svn: 295788
2017-02-22 07:35:48 +08:00
SDValue performFPMed3ImmCombine(SelectionDAG &DAG, const SDLoc &SL,
SDValue Op0, SDValue Op1) const;
AMDGPU: Use v_med3_{f16|i16|u16} llvm-svn: 296401
2017-02-28 06:40:39 +08:00
SDValue performIntMed3ImmCombine(SelectionDAG &DAG, const SDLoc &SL,
SDValue Op0, SDValue Op1, bool Signed) const;
AMDGPU: Match some med3 patterns llvm-svn: 259089
2016-01-29 04:53:42 +08:00
SDValue performMinMaxCombine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Redefine clamp node as clamp 0.0-1.0 Change implementation to use max instead of add. min/max/med3 do not flush denormals regardless of the mode, so it is OK to use it whether or not they are enabled. Also allow using clamp with f16, and use knowledge of dx10_clamp. llvm-svn: 295788
2017-02-22 07:35:48 +08:00
SDValue performFMed3Combine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Add cvt.pkrtz intrinsic Convert llvm.SI.packf16 test uses llvm-svn: 295797
2017-02-22 08:27:34 +08:00
SDValue performCvtPkRTZCombine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Pull fneg out of extract_vector_elt This allows folding source modifiers in more f16 cases. Makes it easier to select per-component packed neg modifiers. llvm-svn: 302813
2017-05-12 01:26:25 +08:00
SDValue performExtractVectorEltCombine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Match some med3 patterns llvm-svn: 259089
2016-01-29 04:53:42 +08:00
AMDGPU: Check fast math flags in fadd/fsub combines llvm-svn: 290312
2016-12-22 12:03:35 +08:00
unsigned getFusedOpcode(const SelectionDAG &DAG,
const SDNode *N0, const SDNode *N1) const;
[AMDGPU] simplify add x, *ext (setcc) => addc|subb x, 0, setcc This simplification allows to avoid generating v_cndmask_b32 to serialize condition code between compare and use. Differential Revision: https://reviews.llvm.org/D34300 llvm-svn: 305962
2017-06-22 06:05:06 +08:00
SDValue performAddCombine(SDNode *N, DAGCombinerInfo &DCI) const;
[AMDGPU] Combine add and adde, sub and sube If one of the arguments of adde/sube is zero we can fold another add/sub into it. Differential Revision: https://reviews.llvm.org/D34374 llvm-svn: 305964
2017-06-22 06:30:01 +08:00
SDValue performAddCarrySubCarryCombine(SDNode *N, DAGCombinerInfo &DCI) const;
SDValue performSubCombine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Move combines into separate functions llvm-svn: 290309
2016-12-22 11:44:42 +08:00
SDValue performFAddCombine(SDNode *N, DAGCombinerInfo &DCI) const;
SDValue performFSubCombine(SDNode *N, DAGCombinerInfo &DCI) const;
R600/SI: Pattern match isinf to v_cmp_class instructions llvm-svn: 225307
2015-01-07 07:00:41 +08:00
SDValue performSetCCCombine(SDNode *N, DAGCombinerInfo &DCI) const;
AMDGPU: Move combines into separate functions llvm-svn: 290309
2016-12-22 11:44:42 +08:00
SDValue performCvtF32UByteNCombine(SDNode *N, DAGCombinerInfo &DCI) const;
R600/SI: Combine min3/max3 instructions llvm-svn: 222032
2014-11-15 04:08:52 +08:00
AMDGPU/SI: Add VI patterns to select FLAT instructions for global memory ops Summary: The MUBUF addr64 bit has been removed on VI, so we must use FLAT instructions when the pointer is stored in VGPRs. Reviewers: arsenm Subscribers: llvm-commits Differential Revision: http://reviews.llvm.org/D11067 llvm-svn: 242673
2015-07-20 22:28:41 +08:00
bool isLegalFlatAddressingMode(const AddrMode &AM) const;
AMDGPU: Teach isLegalAddressingMode about global_* instructions Also refine the flat check to respect flat-for-global feature, and constant fallback should check global handling, not specifically MUBUF. llvm-svn: 309471
2017-07-29 09:12:31 +08:00
bool isLegalGlobalAddressingMode(const AddrMode &AM) const;
AMDGPU: Assume SMRD access for constant address space Since r243294 these are selected to SMRD and moved later if required. llvm-svn: 244354
2015-08-08 04:18:34 +08:00
bool isLegalMUBUFAddressingMode(const AddrMode &AM) const;
AMDGPU/SI: Detect uniform branches and emit s_cbranch instructions Reviewers: arsenm Subscribers: mareko, MatzeB, qcolombet, arsenm, llvm-commits Differential Revision: http://reviews.llvm.org/D16603 llvm-svn: 260765
2016-02-13 07:45:29 +08:00
AMDGPU: Cleanup control flow intrinsics Move backend internal intrinsics along with the rest of the normal intrinsics, and use the Intrinsic::getDeclaration API instead of manually constructing the type list. It's surprising this was working before. fdiv.fast had the wrong number of parameters. The control flow intrinsic declaration attributes were not being applied, and their types were inconsistent. The actual IR use types did not match the declaration, and were closer to the types used for the patterns. The brcond lowering was changing the types, so introduce new nodes for those. llvm-svn: 298119
2017-03-18 04:41:45 +08:00
unsigned isCFIntrinsic(const SDNode *Intr) const;
[AMDGPU] Emit debugger prologue and emit the rest of the debugger fields in the kernel code header Debugger prologue is emitted if -mattr=+amdgpu-debugger-emit-prologue. Debugger prologue writes work group IDs and work item IDs to scratch memory at fixed location in the following format: - offset 0: work group ID x - offset 4: work group ID y - offset 8: work group ID z - offset 16: work item ID x - offset 20: work item ID y - offset 24: work item ID z Set - amd_kernel_code_t::debug_wavefront_private_segment_offset_sgpr to scratch wave offset reg - amd_kernel_code_t::debug_private_segment_buffer_sgpr to scratch rsrc reg - amd_kernel_code_t::is_debug_supported to true if all debugger features are enabled Differential Revision: http://reviews.llvm.org/D20335 llvm-svn: 273769
2016-06-25 11:11:28 +08:00
void createDebuggerPrologueStackObjects(MachineFunction &MF) const;
[AMDGPU] Emit constant address space data in .rodata section and use relocations instead of fixups (amdhsa only) Differential Revision: https://reviews.llvm.org/D25693 llvm-svn: 284759
2016-10-21 02:12:38 +08:00
/// \returns True if fixup needs to be emitted for given global value \p GV,
/// false otherwise.
bool shouldEmitFixup(const GlobalValue *GV) const;
/// \returns True if GOT relocation needs to be emitted for given global value
/// \p GV, false otherwise.
bool shouldEmitGOTReloc(const GlobalValue *GV) const;
/// \returns True if PC-relative relocation needs to be emitted for given
/// global value \p GV, false otherwise.
bool shouldEmitPCReloc(const GlobalValue *GV) const;
Add R600 backend A new backend supporting AMD GPUs: Radeon HD2XXX - HD7XXX llvm-svn: 169915
2012-12-12 05:25:42 +08:00
public:
AMDGPU: Cleanup subtarget handling. Split AMDGPUSubtarget into amdgcn/r600 specific subclasses. This removes most of the static_casting of the basic codegen classes everywhere, and tries to restrict the features visible on the wrong target. llvm-svn: 273652
2016-06-24 14:30:11 +08:00
SITargetLowering(const TargetMachine &tm, const SISubtarget &STI);
const SISubtarget *getSubtarget() const;
R600/SI: Implement isLegalAddressingMode The default assumes that a 16-bit signed offset is used. LDS instruction use a 16-bit unsigned offset, so it wasn't being used in some cases where it was assumed a negative offset could be used. More should be done here, but first isLegalAddressingMode needs to gain an addressing mode argument. For now, copy most of the rest of the default implementation with the immediate offset change. llvm-svn: 215732
2014-08-16 01:17:07 +08:00
TargetLowering: Change isShuffleMaskLegal's mask argument type to ArrayRef<int>. NFCI. Changing mask argument type from const SmallVectorImpl<int>& to ArrayRef<int>. This came up in D35700 where a mask is received as an ArrayRef<int> and we want to pass it to TargetLowering::isShuffleMaskLegal(). Also saves a few lines of code. llvm-svn: 309085
2017-07-26 16:06:58 +08:00
bool isShuffleMaskLegal(ArrayRef<int> /*Mask*/, EVT /*VT*/) const override;
AMDGPU: Allow sinking of addressing modes for atomic_inc/dec llvm-svn: 297913
2017-03-16 07:15:12 +08:00
AMDGPU: Add atomic_inc + atomic_dec intrinsics These are different than atomicrmw add 1 because they have an additional input value to clamp the result. llvm-svn: 266074
2016-04-12 22:05:04 +08:00
bool getTgtMemIntrinsic(IntrinsicInfo &, const CallInst &,
unsigned IntrinsicID) const override;
AMDGPU: Allow sinking of addressing modes for atomic_inc/dec llvm-svn: 297913
2017-03-16 07:15:12 +08:00
bool getAddrModeArguments(IntrinsicInst * /*I*/,
SmallVectorImpl<Value*> &/*Ops*/,
Type *&/*AccessTy*/) const override;
R600/SI: Add pattern for bswap llvm-svn: 220304
2014-10-22 00:25:08 +08:00
Make isLegalAddressingMode() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, llvm-commits, rafael, yaron.keren Differential Revision: http://reviews.llvm.org/D11040 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241778
2015-07-09 10:09:40 +08:00
bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM, Type *Ty,
[SystemZ, LoopStrengthReduce] This patch makes LSR generate better code for SystemZ in the cases of memory intrinsics, Load->Store pairs or comparison of immediate with memory. In order to achieve this, the following common code changes were made: * New TTI hook: LSRWithInstrQueries(), which defaults to false. Controls if LSR should do instruction-based addressing evaluations by calling isLegalAddressingMode() with the Instruction pointers. * In LoopStrengthReduce: handle address operands of memset, memmove and memcpy as address uses, and call isFoldableMemAccessOffset() for any LSRUse::Address, not just loads or stores. SystemZ changes: * isLSRCostLess() implemented with Insns first, and without ImmCost. * New function supportedAddressingMode() that is a helper for TTI methods looking at Instructions passed via pointers. Review: Ulrich Weigand, Quentin Colombet https://reviews.llvm.org/D35262 https://reviews.llvm.org/D35049 llvm-svn: 308729
2017-07-21 19:59:37 +08:00
unsigned AS,
Instruction *I = nullptr) const override;
R600/SI: Implement isLegalAddressingMode The default assumes that a 16-bit signed offset is used. LDS instruction use a 16-bit unsigned offset, so it wasn't being used in some cases where it was assumed a negative offset could be used. More should be done here, but first isLegalAddressingMode needs to gain an addressing mode argument. For now, copy most of the rest of the default implementation with the immediate offset change. llvm-svn: 215732
2014-08-16 01:17:07 +08:00
Add DAG argument to canMergeStoresTo NFC. llvm-svn: 307583
2017-07-11 04:25:54 +08:00
bool canMergeStoresTo(unsigned AS, EVT MemVT,
const SelectionDAG &DAG) const override;
[AMDGPU] Prevent too large store merges in AMDGPU Subtargets. NFCI. Various address spaces on the SI and R600 subtargets have stricter limits on memory access size that other address spaces. Use canMergeStoresTo predicate to prevent the DAGCombiner from creating these stores as they will be split up during legalization. llvm-svn: 303767
2017-05-24 23:59:09 +08:00
Add alignment value to allowsUnalignedMemoryAccess Rename to allowsMisalignedMemoryAccess. On R600, 8 and 16 byte accesses are mostly OK with 4-byte alignment, and don't need to be split into multiple accesses. Vector loads with an alignment of the element type are not uncommon in OpenCL code. llvm-svn: 214055
2014-07-28 01:46:40 +08:00
bool allowsMisalignedMemoryAccesses(EVT VT, unsigned AS,
unsigned Align,
bool *IsFast) const override;
[codegen,aarch64] Add a target hook to the code generator to control vector type legalization strategies in a more fine grained manner, and change the legalization of several v1iN types and v1f32 to be widening rather than scalarization on AArch64. This fixes an assertion failure caused by scalarizing nodes like "v1i32 trunc v1i64". As v1i64 is legal it will fail to scalarize v1i32. This also provides a foundation for other targets to have more granular control over how vector types are legalized. Patch by Hao Liu, reviewed by Tim Northover. I'm committing it to allow some work to start taking place on top of this patch as it adds some really important hooks to the backend that I'd like to immediately start using. =] http://reviews.llvm.org/D4322 llvm-svn: 212242
2014-07-03 08:23:43 +08:00
R600/SI: Implement getOptimalMemOpType The default guess uses i32. This needs an address space argument to really do the right thing in all cases. llvm-svn: 214104
2014-07-29 01:49:26 +08:00
EVT getOptimalMemOpType(uint64_t Size, unsigned DstAlign,
unsigned SrcAlign, bool IsMemset,
bool ZeroMemset,
bool MemcpyStrSrc,
MachineFunction &MF) const override;
AMDGPU/SI: Select constant loads with non-uniform addresses to MUBUF instructions Summary: We were previously selecting all constant loads to SMRD instructions and legalizing the SMRDs with non-uniform addresses during the SIFixSGPRCopesPass. This new solution is more simple and also generates much better code, because the instruction selector is able to take advantage of all the MUBUF addressing modes that are legalization pass wasn't able to. We also no longer need to generate v_add_* instructions when we have a uniform pointer and a non-uniform offset, as this is now folded into the MUBUF instruction during instruction selection. Reviewers: arsenm Subscribers: arsenm, llvm-commits Differential Revision: http://reviews.llvm.org/D15425 llvm-svn: 255672
2015-12-16 04:55:55 +08:00
bool isMemOpUniform(const SDNode *N) const;
[AMDGPU] Scalarization of global uniform loads. Summary: LC can currently select scalar load for uniform memory access basing on readonly memory address space only. This restriction originated from the fact that in HW prior to VI vector and scalar caches are not coherent. With MemoryDependenceAnalysis we can check that the memory location corresponding to the memory operand of the LOAD is not clobbered along the all paths from the function entry. Reviewers: rampitec, tstellarAMD, arsenm Subscribers: wdng, arsenm, nhaehnle Differential Revision: https://reviews.llvm.org/D26917 llvm-svn: 289076
2016-12-09 01:28:47 +08:00
bool isMemOpHasNoClobberedMemOperand(const SDNode *N) const;
AMDGPU: Implement isNoopAddrSpaceCast llvm-svn: 254468
2015-12-02 07:04:00 +08:00
bool isNoopAddrSpaceCast(unsigned SrcAS, unsigned DestAS) const override;
AMDGPU: Implement isCheapAddrSpaceCast llvm-svn: 288523
2016-12-03 02:12:53 +08:00
bool isCheapAddrSpaceCast(unsigned SrcAS, unsigned DestAS) const override;
AMDGPU: Implement isNoopAddrSpaceCast llvm-svn: 254468
2015-12-02 07:04:00 +08:00
[codegen,aarch64] Add a target hook to the code generator to control vector type legalization strategies in a more fine grained manner, and change the legalization of several v1iN types and v1f32 to be widening rather than scalarization on AArch64. This fixes an assertion failure caused by scalarizing nodes like "v1i32 trunc v1i64". As v1i64 is legal it will fail to scalarize v1i32. This also provides a foundation for other targets to have more granular control over how vector types are legalized. Patch by Hao Liu, reviewed by Tim Northover. I'm committing it to allow some work to start taking place on top of this patch as it adds some really important hooks to the backend that I'd like to immediately start using. =] http://reviews.llvm.org/D4322 llvm-svn: 212242
2014-07-03 08:23:43 +08:00
TargetLoweringBase::LegalizeTypeAction
getPreferredVectorAction(EVT VT) const override;
R600/SI: add proper formal parameter handling for SI Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Tom Stellard <thomas.stellard@amd.com> llvm-svn: 176623
2013-03-07 17:03:52 +08:00
[C++11] Add 'override' keywords and remove 'virtual'. Additionally add 'final' and leave 'virtual' on some methods that are marked virtual without overriding anything and have no obvious overrides themselves. R600 edition llvm-svn: 207503
2014-04-29 15:57:24 +08:00
bool shouldConvertConstantLoadToIntImm(const APInt &Imm,
Type *Ty) const override;
R600/SI: Implement shouldConvertConstantLoadToIntImm llvm-svn: 205244
2014-04-01 03:54:27 +08:00
AMDGPU/SI: Prevent the DAGCombiner from creating setcc with i1 inputs Reviewers: arsenm Subscribers: arsenm, llvm-commits Differential Revision: http://reviews.llvm.org/D15035 llvm-svn: 258256
2016-01-20 08:13:22 +08:00
bool isTypeDesirableForOp(unsigned Op, EVT VT) const override;
AMDGPU/SI: Make sure not to fold offsets into local address space globals Summary: Offset folding only works if you are emitting relocations, and we don't emit relocations for local address space globals. Reviewers: arsenm, nhaustov Subscribers: arsenm, llvm-commits, kzhuravl Differential Revision: http://reviews.llvm.org/D21647 llvm-svn: 273765
2016-06-25 09:59:16 +08:00
bool isOffsetFoldingLegal(const GlobalAddressSDNode *GA) const override;
AMDGPU: Initial implementation of calls Includes a hack to fix the type selected for the GlobalAddress of the function, which will be fixed by changing the default datalayout to use generic pointers for 0. llvm-svn: 309732
2017-08-02 03:54:18 +08:00
bool supportSplitCSR(MachineFunction *MF) const override;
void initializeSplitCSR(MachineBasicBlock *Entry) const override;
void insertCopiesSplitCSR(
MachineBasicBlock *Entry,
const SmallVectorImpl<MachineBasicBlock *> &Exits) const override;
R600/SI: add proper formal parameter handling for SI Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Tom Stellard <thomas.stellard@amd.com> llvm-svn: 176623
2013-03-07 17:03:52 +08:00
SDValue LowerFormalArguments(SDValue Chain, CallingConv::ID CallConv,
bool isVarArg,
const SmallVectorImpl<ISD::InputArg> &Ins,
Pass DebugLoc and SDLoc by const ref. This used to be free, copying and moving DebugLocs became expensive after the metadata rewrite. Passing by reference eliminates a ton of track/untrack operations. No functionality change intended. llvm-svn: 272512
2016-06-12 23:39:02 +08:00
const SDLoc &DL, SelectionDAG &DAG,
[C++11] Add 'override' keywords and remove 'virtual'. Additionally add 'final' and leave 'virtual' on some methods that are marked virtual without overriding anything and have no obvious overrides themselves. R600 edition llvm-svn: 207503
2014-04-29 15:57:24 +08:00
SmallVectorImpl<SDValue> &InVals) const override;
R600/SI: add proper formal parameter handling for SI Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Tom Stellard <thomas.stellard@amd.com> llvm-svn: 176623
2013-03-07 17:03:52 +08:00
AMDGPU: Start defining a calling convention Partially implement callee-side for arguments and return values. byval doesn't work properly, and most likely sret or other on-stack return values most as well. llvm-svn: 303308
2017-05-18 05:56:25 +08:00
bool CanLowerReturn(CallingConv::ID CallConv,
MachineFunction &MF, bool isVarArg,
const SmallVectorImpl<ISD::OutputArg> &Outs,
LLVMContext &Context) const override;
SDValue LowerReturn(SDValue Chain, CallingConv::ID CallConv, bool IsVarArg,
AMDGPU/SI: Add support for non-void functions Summary: Return values can be stored in SGPRs (i32) and VGPRs (f32). This will be used by functions which expect some bytecode or other binary to be appended at the end. It allows defining in which registers the return values will be stored. v2: don't do this for compute shaders Reviewers: tstellarAMD, arsenm Subscribers: arsenm Differential Revision: http://reviews.llvm.org/D16033 llvm-svn: 257621
2016-01-14 01:23:04 +08:00
const SmallVectorImpl<ISD::OutputArg> &Outs,
Pass DebugLoc and SDLoc by const ref. This used to be free, copying and moving DebugLocs became expensive after the metadata rewrite. Passing by reference eliminates a ton of track/untrack operations. No functionality change intended. llvm-svn: 272512
2016-06-12 23:39:02 +08:00
const SmallVectorImpl<SDValue> &OutVals, const SDLoc &DL,
SelectionDAG &DAG) const override;
AMDGPU/SI: Add support for non-void functions Summary: Return values can be stored in SGPRs (i32) and VGPRs (f32). This will be used by functions which expect some bytecode or other binary to be appended at the end. It allows defining in which registers the return values will be stored. v2: don't do this for compute shaders Reviewers: tstellarAMD, arsenm Subscribers: arsenm Differential Revision: http://reviews.llvm.org/D16033 llvm-svn: 257621
2016-01-14 01:23:04 +08:00
AMDGPU: Pass special input registers to functions llvm-svn: 309998
2017-08-04 07:00:29 +08:00
void passSpecialInputs(
CallLoweringInfo &CLI,
const SIMachineFunctionInfo &Info,
SmallVectorImpl<std::pair<unsigned, SDValue>> &RegsToPass,
SmallVectorImpl<SDValue> &MemOpChains,
SDValue Chain,
SDValue StackPtr) const;
AMDGPU: Initial implementation of calls Includes a hack to fix the type selected for the GlobalAddress of the function, which will be fixed by changing the default datalayout to use generic pointers for 0. llvm-svn: 309732
2017-08-02 03:54:18 +08:00
SDValue LowerCallResult(SDValue Chain, SDValue InFlag,
CallingConv::ID CallConv, bool isVarArg,
const SmallVectorImpl<ISD::InputArg> &Ins,
const SDLoc &DL, SelectionDAG &DAG,
SmallVectorImpl<SDValue> &InVals, bool isThisReturn,
SDValue ThisVal) const;
SDValue LowerCall(CallLoweringInfo &CLI,
SmallVectorImpl<SDValue> &InVals) const override;
AMDGPU: Implement read_register and write_register intrinsics Some of the special intrinsics now that now correspond to a instruction also have special setting of some registers, e.g. llvm.SI.sendmsg sets m0 as well as use s_sendmsg. Using these explicit register intrinsics may be a better option. Reading the exec mask and others may be useful for debugging. For this I'm not sure this is entirely correct because we would want this to be convergent, although it's possible this is already treated sufficently conservatively. llvm-svn: 258785
2016-01-26 12:29:24 +08:00
unsigned getRegisterByName(const char* RegName, EVT VT,
SelectionDAG &DAG) const override;
AMDGPU: Follow up to r275203 I meant to squash this into it. llvm-svn: 275220
2016-07-13 05:41:32 +08:00
MachineBasicBlock *splitKillBlock(MachineInstr &MI,
MachineBasicBlock *BB) const;
CodeGen: Use MachineInstr& in TargetLowering, NFC This is a mechanical change to make TargetLowering API take MachineInstr& (instead of MachineInstr*), since the argument is expected to be a valid MachineInstr. In one case, changed a parameter from MachineInstr* to MachineBasicBlock::iterator, since it was used as an insertion point. As a side effect, this removes a bunch of MachineInstr* to MachineBasicBlock::iterator implicit conversions, a necessary step toward fixing PR26753. llvm-svn: 274287
2016-07-01 06:52:52 +08:00
MachineBasicBlock *
EmitInstrWithCustomInserter(MachineInstr &MI,
MachineBasicBlock *BB) const override;
R600/SI: Implement enableAggressiveFMAFusion Add tests for the various combines. This should always be at least cycle neutral on all subtargets for f64, and faster on some. For f32 we should prefer selecting v_mad_f32 over v_fma_f32. llvm-svn: 227484
2015-01-30 03:34:32 +08:00
bool enableAggressiveFMAFusion(EVT VT) const override;
Make TargetLowering::getPointerTy() taking DataLayout as an argument Summary: This change is part of a series of commits dedicated to have a single DataLayout during compilation by using always the one owned by the module. Reviewers: echristo Subscribers: jholewinski, ted, yaron.keren, rafael, llvm-commits Differential Revision: http://reviews.llvm.org/D11028 From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241775
2015-07-09 10:09:04 +08:00
EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context,
EVT VT) const override;
Re-instate the EVT parameter to getScalarShiftAmountTy() for OOT user A documentation for this function would be nice by the way. From: Mehdi Amini <mehdi.amini@apple.com> llvm-svn: 241807
2015-07-09 23:12:23 +08:00
MVT getScalarShiftAmountTy(const DataLayout &, EVT) const override;
[C++11] Add 'override' keywords and remove 'virtual'. Additionally add 'final' and leave 'virtual' on some methods that are marked virtual without overriding anything and have no obvious overrides themselves. R600 edition llvm-svn: 207503
2014-04-29 15:57:24 +08:00
bool isFMAFasterThanFMulAndFAdd(EVT VT) const override;
SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override;
AMDGPU: Custom lower more vector operations This avoids stack usage. llvm-svn: 292846
2017-01-24 07:09:58 +08:00
void ReplaceNodeResults(SDNode *N, SmallVectorImpl<SDValue> &Results,
SelectionDAG &DAG) const override;
[C++11] Add 'override' keywords and remove 'virtual'. Additionally add 'final' and leave 'virtual' on some methods that are marked virtual without overriding anything and have no obvious overrides themselves. R600 edition llvm-svn: 207503
2014-04-29 15:57:24 +08:00
SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
SDNode *PostISelFolding(MachineSDNode *N, SelectionDAG &DAG) const override;
CodeGen: Use MachineInstr& in TargetLowering, NFC This is a mechanical change to make TargetLowering API take MachineInstr& (instead of MachineInstr*), since the argument is expected to be a valid MachineInstr. In one case, changed a parameter from MachineInstr* to MachineBasicBlock::iterator, since it was used as an insertion point. As a side effect, this removes a bunch of MachineInstr* to MachineBasicBlock::iterator implicit conversions, a necessary step toward fixing PR26753. llvm-svn: 274287
2016-07-01 06:52:52 +08:00
void AdjustInstrPostInstrSelection(MachineInstr &MI,
[C++11] Add 'override' keywords and remove 'virtual'. Additionally add 'final' and leave 'virtual' on some methods that are marked virtual without overriding anything and have no obvious overrides themselves. R600 edition llvm-svn: 207503
2014-04-29 15:57:24 +08:00
SDNode *Node) const override;
R600/SI: add post ISel folding for SI v2 Include immediate folding and SGPR limit handling for VOP3 instructions. v2: remove leftover hasExtraSrcRegAllocReq Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Tom Stellard <thomas.stellard@amd.com> llvm-svn: 176101
2013-02-27 01:52:23 +08:00
AMDGPU: Fix invalid copies when copying i1 to phys reg Insert a VReg_1 virtual register so the i1 workaround pass can handle it. llvm-svn: 300113
2017-04-13 05:58:23 +08:00
SDNode *legalizeTargetIndependentNode(SDNode *Node, SelectionDAG &DAG) const;
R600/SI: Remove SI_ADDR64_RSRC llvm-svn: 221382
2014-11-06 03:01:17 +08:00
Pass DebugLoc and SDLoc by const ref. This used to be free, copying and moving DebugLocs became expensive after the metadata rewrite. Passing by reference eliminates a ton of track/untrack operations. No functionality change intended. llvm-svn: 272512
2016-06-12 23:39:02 +08:00
MachineSDNode *wrapAddr64Rsrc(SelectionDAG &DAG, const SDLoc &DL,
SDValue Ptr) const;
MachineSDNode *buildRSRC(SelectionDAG &DAG, const SDLoc &DL, SDValue Ptr,
uint32_t RsrcDword1, uint64_t RsrcDword2And3) const;
[TargetLowering] StringRefize asm constraint getters. There is some functional change here because it changes target code from atoi(3) to StringRef::getAsInteger which has error checking. For valid constraints there should be no difference. llvm-svn: 241411
2015-07-06 03:29:18 +08:00
std::pair<unsigned, const TargetRegisterClass *>
getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
StringRef Constraint, MVT VT) const override;
AMDGPU/SI: Add support for sgpr and vgpr inline assembly constraints Summary: The 's' constraint represents sgprs and the 'v' constraint represents vgprs. Reviewers: arsenm, echristo Subscribers: arsenm, llvm-commits Differential Revision: http://reviews.llvm.org/D15342 llvm-svn: 255203
2015-12-10 10:12:53 +08:00
ConstraintType getConstraintType(StringRef Constraint) const override;
Pass DebugLoc and SDLoc by const ref. This used to be free, copying and moving DebugLocs became expensive after the metadata rewrite. Passing by reference eliminates a ton of track/untrack operations. No functionality change intended. llvm-svn: 272512
2016-06-12 23:39:02 +08:00
SDValue copyToM0(SelectionDAG &DAG, SDValue Chain, const SDLoc &DL,
SDValue V) const;
AMDGPU: Figure out private memory regs after lowering Introduce pseudo-registers for registers needed for stack access, which are replaced during finalizeLowering. Note these pseudo-registers are currently only used for the used register location, and not for determining their input argument register. This is better because it avoids the need to try to predict whether a call will be emitted from the IR, and also detects stack objects introduced by legalization. Test changes are from the HasStackObjects check being more accurate since stack objects introduced during legalization are now known. llvm-svn: 308325
2017-07-19 00:44:56 +08:00
void finalizeLowering(MachineFunction &MF) const override;
Add R600 backend A new backend supporting AMD GPUs: Radeon HD2XXX - HD7XXX llvm-svn: 169915
2012-12-12 05:25:42 +08:00
};
} // End namespace llvm
Canonicalize header guards into a common format. Add header guards to files that were missing guards. Remove #endif comments as they don't seem common in LLVM (we can easily add them back if we decide they're useful) Changes made by clang-tidy with minor tweaks. llvm-svn: 215558
2014-08-14 00:26:38 +08:00
#endif