maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/Target/RISCV/RISCVISelLowering.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

619 lines
25 KiB
C
Raw Normal View History

[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
//===-- RISCVISelLowering.h - RISCV DAG Lowering Interface ------*- C++ -*-===//
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 16:50:56 +08:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
//
//===----------------------------------------------------------------------===//
//
// This file defines the interfaces that RISCV uses to lower LLVM code into a
// selection DAG.
//
//===----------------------------------------------------------------------===//
#ifndef LLVM_LIB_TARGET_RISCV_RISCVISELLOWERING_H
#define LLVM_LIB_TARGET_RISCV_RISCVISELLOWERING_H
#include "RISCV.h"
[RISCV] Support vector types in combination with fastcc This patch extends the RISC-V lowering of the 'fastcc' calling convention to vector types, both fixed-length and scalable. Without this patch, any function passing or returning vector types by value would throw a compiler error. Vectors are handled in 'fastcc' much as they are in the default calling convention, the noticeable difference being the extended set of scalar GPR registers that can be used to pass vectors indirectly. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D102505
2021-05-14 00:34:41 +08:00
#include "llvm/CodeGen/CallingConvLower.h"
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
#include "llvm/CodeGen/SelectionDAG.h"
Fix a bunch more layering of CodeGen headers that are in Target All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
2017-11-17 09:07:10 +08:00
#include "llvm/CodeGen/TargetLowering.h"
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
namespace llvm {
class RISCVSubtarget;
[RISCV] Support extraction of misaligned subvectors This patch extends the support for RVV EXTRACT_SUBVECTOR to cover those which don't align to a vector register boundary. It accomplishes this by extracting the nearest register-sized subvector (a subregister operation), then sliding the vector down with VSLIDEDOWN and extracting the subvector from the first position (a COPY operation). Since this procedure involves the use of VSCALE and multiplication, the handling of such operations is done during lowering to simplify the implementation and make use of DAG combining. This necessitated moving some helper functions from RISCVISelDAGToDAG to RISCVTargetLowering. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96959
2021-02-18 20:48:14 +08:00
struct RISCVRegisterInfo;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
namespace RISCVISD {
enum NodeType : unsigned {
FIRST_NUMBER = ISD::BUILTIN_OP_END,
[RISCV] Initial support for function calls Note that this is just enough for simple function call examples to generate working code. Support for varargs etc follows in future patches. Differential Revision: https://reviews.llvm.org/D29936 llvm-svn: 317691
2017-11-08 21:41:21 +08:00
RET_FLAG,
[RISCV] Add support for _interrupt attribute - Save/restore only registers that are used. This includes Callee saved registers and Caller saved registers (arguments and temporaries) for integer and FP registers. - If there is a call in the interrupt handler, save/restore all Caller saved registers (arguments and temporaries) and all FP registers. - Emit special return instructions depending on "interrupt" attribute type. Based on initial patch by Zhaoshi Zheng. Reviewers: asb Reviewed By: asb Subscribers: rkruppe, the_o, MartinMosbeck, brucehoult, rbar, johnrusso, simoncook, sabuasal, niosHD, kito-cheng, shiva0217, zzheng, edward-jones, mgrang, rogfer01, llvm-commits Differential Revision: https://reviews.llvm.org/D48411 llvm-svn: 338047
2018-07-27 01:49:43 +08:00
URET_FLAG,
SRET_FLAG,
MRET_FLAG,
[RISCV] Implement lowering of ISD::SELECT Although ISD::SELECT_CC is a more natural match for RISCVISD::SELECT_CC (and ultimately the integer RISC-V conditional branch instructions), we choose to expand ISD::SELECT_CC and lower ISD::SELECT. The appropriate compare+branch will be created in the case where an ISD::SELECT condition value is created by an ISD::SETCC node, which operates on XLen types. Other datatypes such as floating point don't have conditional branch instructions, and lowering ISD::SELECT allows more flexibility for handling these cases. Differential Revision: https://reviews.llvm.org/D29937 llvm-svn: 318735
2017-11-21 15:51:32 +08:00
CALL,
[RISCV] Optimize select_cc after fp compare expansion Some FP compares expand to a sequence ending with (xor X, 1) to invert the result. If the consumer is a select_cc we can likely get rid of this xor by fixing up the select_cc condition. This patch combines (select_cc (xor X, 1), 0, setne, trueV, falseV) - (select_cc X, 0, seteq, trueV, falseV) if we can prove X is 0/1. Reviewed By: lenary Differential Revision: https://reviews.llvm.org/D94546
2021-01-15 05:41:35 +08:00
/// Select with condition operator - This selects between a true value and
/// a false value (ops #3 and #4) based on the boolean result of comparing
/// the lhs and rhs (ops #0 and #1) of a conditional expression with the
/// condition code in op #2, a XLenVT constant from the ISD::CondCode enum.
/// The lhs and rhs are XLenVT integers. The true and false values can be
/// integer or floating point.
[RISCV] Codegen support for RV32D floating point load/store, fadd.d, calling conv fadd.d is required in order to force floating point registers to be used in test code, as parameters are passed in integer registers in the soft float ABI. Much of this patch is concerned with support for passing f64 on RV32D with a soft-float ABI. Similar to Mips, introduce pseudoinstructions to build an f64 out of a pair of i32 and to split an f64 to a pair of i32. BUILD_PAIR and EXTRACT_ELEMENT can't be used, as a BITCAST to i64 would be necessary, but i64 is not a legal type. llvm-svn: 329871
2018-04-12 13:34:25 +08:00
SELECT_CC,
[RISCV] Add RISCVISD::BR_CC similar to RISCVISD::SELECT_CC. This allows me to introduce similar combines for branches as we have recently added for SELECT_CC. Some of them are less useful for standalone setccs and only help branch instructions. By having a BR_CC node its easier to only affect branches. I'm using CondCodeSDNode to make isel patterns easier to write so we can refer to the codes by name. SELECT_CC uses a constant instead. I've translated the condition code just like SELECT_CC so we need less patterns for the swapped conditions. This includes special cases for X < 1 and X > -1 that get translated to blez and bgez by using a 0 constant. computeKnownBitsForTargetNode support for SELECT_CC is added to allow MaskedValueIsZero to work for cases where the true and false values of the SELECT_CC are setccs and the result of the SELECT_CC is used by a BR_CC. This was needed to avoid regressions in some of the overflow tests. Reviewed By: luismarques Differential Revision: https://reviews.llvm.org/D98159
2021-03-16 02:44:59 +08:00
BR_CC,
[RISCV] Codegen support for RV32D floating point load/store, fadd.d, calling conv fadd.d is required in order to force floating point registers to be used in test code, as parameters are passed in integer registers in the soft float ABI. Much of this patch is concerned with support for passing f64 on RV32D with a soft-float ABI. Similar to Mips, introduce pseudoinstructions to build an f64 out of a pair of i32 and to split an f64 to a pair of i32. BUILD_PAIR and EXTRACT_ELEMENT can't be used, as a BITCAST to i64 would be necessary, but i64 is not a legal type. llvm-svn: 329871
2018-04-12 13:34:25 +08:00
BuildPairF64,
[RISCV] Lower the tail pseudoinstruction This patch lowers the tail pseudoinstruction. This has been modeled after ARM's tail call opt. llvm-svn: 333137
2018-05-24 06:44:08 +08:00
SplitF64,
[RISCV] Custom-legalise 32-bit variable shifts on RV64 The previous DAG combiner-based approach had an issue with infinite loops between the target-dependent and target-independent combiner logic (see PR40333). Although this was worked around in rL351806, the combiner-based approach is still potentially brittle and can fail to select the 32-bit shift variant when profitable to do so, as demonstrated in the pr40333.ll test case. This patch instead introduces target-specific SelectionDAG nodes for SHLW/SRLW/SRAW and custom-lowers variable i32 shifts to them. pr40333.ll is a good example of how this approach can improve codegen. This adds DAG combine that does SimplifyDemandedBits on the operands (only lower 32-bits of first operand and lower 5 bits of second operand are read). This seems better than implementing SimplifyDemandedBitsForTargetNode as there is no guarantee that would be called (and it's not for e.g. the anyext return test cases). Also implements ComputeNumSignBitsForTargetNode. There are codegen changes in atomic-rmw.ll and atomic-cmpxchg.ll but the new instruction sequences are semantically equivalent. Differential Revision: https://reviews.llvm.org/D57085 llvm-svn: 352169
2019-01-25 13:04:00 +08:00
TAIL,
[RISCV] Add custom type legalization to form MULHSU when possible. There's no target independent ISD opcode for MULHSU, so custom legalize 2*XLen multiplies ourselves. We have to be a little careful to prefer MULHU or MULHSU. I thought about doing this in isel by pattern matching the (add (mul X, (srai Y, XLen-1)), (mulhu X, Y)) pattern. I decided against this because the add might become part of a chain of adds. I don't trust DAG combine not to reassociate with other adds making it difficult to find both pieces again. Reviewed By: asb Differential Revision: https://reviews.llvm.org/D99479
2021-04-02 00:41:36 +08:00
// Multiply high for signedxunsigned.
MULHSU,
[RISCV] Custom-legalise 32-bit variable shifts on RV64 The previous DAG combiner-based approach had an issue with infinite loops between the target-dependent and target-independent combiner logic (see PR40333). Although this was worked around in rL351806, the combiner-based approach is still potentially brittle and can fail to select the 32-bit shift variant when profitable to do so, as demonstrated in the pr40333.ll test case. This patch instead introduces target-specific SelectionDAG nodes for SHLW/SRLW/SRAW and custom-lowers variable i32 shifts to them. pr40333.ll is a good example of how this approach can improve codegen. This adds DAG combine that does SimplifyDemandedBits on the operands (only lower 32-bits of first operand and lower 5 bits of second operand are read). This seems better than implementing SimplifyDemandedBitsForTargetNode as there is no guarantee that would be called (and it's not for e.g. the anyext return test cases). Also implements ComputeNumSignBitsForTargetNode. There are codegen changes in atomic-rmw.ll and atomic-cmpxchg.ll but the new instruction sequences are semantically equivalent. Differential Revision: https://reviews.llvm.org/D57085 llvm-svn: 352169
2019-01-25 13:04:00 +08:00
// RV64I shifts, directly matching the semantics of the named RISC-V
// instructions.
SLLW,
SRAW,
[RISCV] Custom-legalise i32 SDIV/UDIV/UREM on RV64M Follow the same custom legalisation strategy as used in D57085 for variable-length shifts (see that patch summary for more discussion). Although we may lose out on some late-stage DAG combines, I think this custom legalisation strategy is ultimately easier to reason about. There are some codegen changes in rv64m-exhaustive-w-insts.ll but they are all neutral in terms of the number of instructions. Differential Revision: https://reviews.llvm.org/D57096 llvm-svn: 352171
2019-01-25 13:11:34 +08:00
SRLW,
// 32-bit operations from RV64M that can't be simply matched with a pattern
[RISCV] Custom type legalize i8/i16 UDIV/UREM/SDIV on RV64 so we can use divuw/remuw/divw. This makes our i8/i16 codegen more similar to the i32 codegen. I've also added computeKnownBits support for DIVUW/REMUW so that we can remove zero extending ANDs from the output. Without this we end up turning DIVUW/REMUW back into DIVU/REMU via some isel patterns. Reviewed By: frasercrmck, luismarques Differential Revision: https://reviews.llvm.org/D95322
2021-01-26 01:35:14 +08:00
// at instruction selection time. These have undefined behavior for division
// by 0 or overflow (divw) like their target independent counterparts.
[RISCV] Custom-legalise i32 SDIV/UDIV/UREM on RV64M Follow the same custom legalisation strategy as used in D57085 for variable-length shifts (see that patch summary for more discussion). Although we may lose out on some late-stage DAG combines, I think this custom legalisation strategy is ultimately easier to reason about. There are some codegen changes in rv64m-exhaustive-w-insts.ll but they are all neutral in terms of the number of instructions. Differential Revision: https://reviews.llvm.org/D57096 llvm-svn: 352171
2019-01-25 13:11:34 +08:00
DIVW,
DIVUW,
[RISCV] Add RV64F codegen support This requires a little extra work due tothe fact i32 is not a legal type. When call lowering happens post-legalisation (e.g. when an intrinsic was inserted during legalisation). A bitcast from f32 to i32 can't be introduced. This is similar to the challenges with RV32D. To handle this, we introduce target-specific DAG nodes that perform bitcast+anyext for f32->i64 and trunc+bitcast for i64->f32. Differential Revision: https://reviews.llvm.org/D53235 llvm-svn: 352807
2019-02-01 06:48:38 +08:00
REMUW,
[RISCV] Add RISCVISD::ROLW/RORW use those for custom legalizing i32 rotl/rotr on RV64IZbb. This should result in better utilization of RORIW since we don't need to look for a SIGN_EXTEND_INREG that may not exist. Also remove rotl/rotr isel matching to GREVI and just prefer RORI. This is to keep consistency so we don't have to match ROLW/RORW to GREVIW as well. I imagine RORI/RORIW performance will be the same or better than GREVI. Differential Revision: https://reviews.llvm.org/D91449
2020-11-21 02:11:34 +08:00
// RV64IB rotates, directly matching the semantics of the named RISC-V
// instructions.
ROLW,
RORW,
[RISCV] Add RISCVISD opcodes for CLZW and CTZW. Our CLZW isel pattern is quite easily broken by surrounding code preventing it from matching sometimes. This usually results in failing to remove the and X, 0xffffffff inserted by type legalization. The add with -32 that type legalization also inserts will often gets combined into other add/sub nodes. That doesn't usually result in extra code when we don't use clzw. CTTZ seems to be less fragile, but I wanted to keep it consistent with CTLZ. Reviewed By: asb, HsiangKai Differential Revision: https://reviews.llvm.org/D99317
2021-04-01 00:27:22 +08:00
// RV64IZbb bit counting instructions directly matching the semantics of the
// named RISC-V instructions.
CLZW,
CTZW,
[RISCV] Custom lower fshl/fshr with Zbt extension. We need to add a mask to the shift amount for these operations to use the FSR/FSL instructions. We were previously doing this in isel patterns, but custom lowering will make the mask visible to optimizations earlier.
2021-02-01 09:37:44 +08:00
// RV64IB/RV32IB funnel shifts, with the semantics of the named RISC-V
// instructions, but the same operand order as fshl/fshr intrinsics.
FSR,
FSL,
[RISCV] Custom type legalize i32 fshl/fshr on RV64 with Zbt. This adds custom opcodes for FSLW/FSRW so we can type legalize fshl/fshr without needing to match a sign_extend_inreg. I've used the operand order from fshl/fshr to make the isel pattern similar to the non-W form. It was also hard to decide another order since the register instruction has the shift amount as the second operand, but the immediate instruction has it as the third operand. Differential Revision: https://reviews.llvm.org/D91479
2020-11-26 01:43:16 +08:00
// RV64IB funnel shifts, with the semantics of the named RISC-V instructions,
// but the same operand order as fshl/fshr intrinsics.
FSRW,
FSLW,
[RISCV] Support Zfh half-precision floating-point extension. Support "Zfh" extension according to https://github.com/riscv/riscv-isa-manual/blob/zfh/src/zfh.tex Differential Revision: https://reviews.llvm.org/D90738
2020-07-03 22:57:59 +08:00
// FPR<->GPR transfer operations when the FPR is smaller than XLEN, needed as
// XLEN is the only legal integer width.
//
[RISCV] Merge FMV_H_X_RV32/FMV_H_X_RV64 into a single opcode. Same with FMV_X_ANYEXTH_RV32/RV64 Rather than having a different opcode for RV32 and RV64. Let's just say the integer type is XLenVT and use a single opcode for both modes. Differential Revision: https://reviews.llvm.org/D92538
2020-12-04 02:29:10 +08:00
// FMV_H_X matches the semantics of the FMV.H.X.
// FMV_X_ANYEXTH is similar to FMV.X.H but has an any-extended result.
[RISCV] Support Zfh half-precision floating-point extension. Support "Zfh" extension according to https://github.com/riscv/riscv-isa-manual/blob/zfh/src/zfh.tex Differential Revision: https://reviews.llvm.org/D90738
2020-07-03 22:57:59 +08:00
// FMV_W_X_RV64 matches the semantics of the FMV.W.X.
[RISCV] Add RV64F codegen support This requires a little extra work due tothe fact i32 is not a legal type. When call lowering happens post-legalisation (e.g. when an intrinsic was inserted during legalisation). A bitcast from f32 to i32 can't be introduced. This is similar to the challenges with RV32D. To handle this, we introduce target-specific DAG nodes that perform bitcast+anyext for f32->i64 and trunc+bitcast for i64->f32. Differential Revision: https://reviews.llvm.org/D53235 llvm-svn: 352807
2019-02-01 06:48:38 +08:00
// FMV_X_ANYEXTW_RV64 is similar to FMV.X.W but has an any-extended result.
[RISCV] Support Zfh half-precision floating-point extension. Support "Zfh" extension according to https://github.com/riscv/riscv-isa-manual/blob/zfh/src/zfh.tex Differential Revision: https://reviews.llvm.org/D90738
2020-07-03 22:57:59 +08:00
//
[RISCV] Add RV64F codegen support This requires a little extra work due tothe fact i32 is not a legal type. When call lowering happens post-legalisation (e.g. when an intrinsic was inserted during legalisation). A bitcast from f32 to i32 can't be introduced. This is similar to the challenges with RV32D. To handle this, we introduce target-specific DAG nodes that perform bitcast+anyext for f32->i64 and trunc+bitcast for i64->f32. Differential Revision: https://reviews.llvm.org/D53235 llvm-svn: 352807
2019-02-01 06:48:38 +08:00
// This is a more convenient semantic for producing dagcombines that remove
// unnecessary GPR->FPR->GPR moves.
[RISCV] Merge FMV_H_X_RV32/FMV_H_X_RV64 into a single opcode. Same with FMV_X_ANYEXTH_RV32/RV64 Rather than having a different opcode for RV32 and RV64. Let's just say the integer type is XLenVT and use a single opcode for both modes. Differential Revision: https://reviews.llvm.org/D92538
2020-12-04 02:29:10 +08:00
FMV_H_X,
FMV_X_ANYEXTH,
[RISCV] Add RV64F codegen support This requires a little extra work due tothe fact i32 is not a legal type. When call lowering happens post-legalisation (e.g. when an intrinsic was inserted during legalisation). A bitcast from f32 to i32 can't be introduced. This is similar to the challenges with RV32D. To handle this, we introduce target-specific DAG nodes that perform bitcast+anyext for f32->i64 and trunc+bitcast for i64->f32. Differential Revision: https://reviews.llvm.org/D53235 llvm-svn: 352807
2019-02-01 06:48:38 +08:00
FMV_W_X_RV64,
[RISCV] Support @llvm.readcyclecounter() Intrinsic On RISC-V, the `cycle` CSR holds a 64-bit count of the number of clock cycles executed by the core, from an arbitrary point in the past. This matches the intended semantics of `@llvm.readcyclecounter()`, which we currently leave to the default lowering (to the constant 0). With this patch, we will now correctly lower this intrinsic to the intended semantics, using the user-space instruction `rdcycle`. On 64-bit targets, we can directly lower to this instruction. On 32-bit targets, we need to do more, as `rdcycle` only returns the low 32-bits of the `cycle` CSR. In this case, we perform a custom lowering, based on the PowerPC lowering, using `rdcycleh` to obtain the high 32-bits of the `cycle` CSR. This custom lowering inserts a new basic block which detects overflow in the high 32-bits of the `cycle` CSR during reading (because multiple instructions are required to read). The emitted assembly matches the suggested assembly in the RISC-V specification. Differential Revision: https://reviews.llvm.org/D64125 llvm-svn: 365201
2019-07-05 20:35:21 +08:00
FMV_X_ANYEXTW_RV64,
// READ_CYCLE_WIDE - A read of the 64-bit cycle CSR on a 32-bit target
// (returns (Lo, Hi)). It takes a chain operand.
[RISCV] Lower GREVI and GORCI as custom nodes This moves the recognition of GREVI and GORCI from TableGen patterns into a DAGCombine. This is done primarily to match "deeper" patterns in the future, like (grevi (grevi x, 1) 2) -> (grevi x, 3). TableGen is not best suited to matching patterns such as these as the compile time of the DAG matchers quickly gets out of hand due to the expansion of commutative permutations. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D91259
2020-11-11 21:41:55 +08:00
READ_CYCLE_WIDE,
// Generalized Reverse and Generalized Or-Combine - directly matching the
// semantics of the named RISC-V instructions. Lowered as custom nodes as
// TableGen chokes when faced with commutative permutations in deeply-nested
[RISCV] Rename RISCVISD::GREVI(W)/GORCI(W) to RISCVISD::GREV(W)/GORC(W). Don't require second operand to be a constant. Prep work for adding intrinsics for these instructions in the future.
2021-04-13 22:54:24 +08:00
// DAGs. Each node takes an input operand and a control operand and outputs a
// bit-manipulated version of input. All operands are i32 or XLenVT.
GREV,
GREVW,
GORC,
GORCW,
[RISCV] Rename RISCVISD::SHFLI to RISCVISD::SHFL and don't require the second operand to be an immediate. Prep work for adding intrinsics in the future. Left an assert that the input is constant in ReplaceNodeResults, as the intrinsic shouldn't go through that path.
2021-04-13 14:27:44 +08:00
SHFL,
[RISCV] [1/2] Add IR intrinsic for Zbp extension RV32/64: grev grevi gorc gorci shfl shfli unshfl unshfli RV64 ONLY: grevw greviw gorcw gorciw shflw shfli (For non-existing shfliw) unshfli (For non-existing unshfliw) Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D100830
2021-04-23 07:19:15 +08:00
SHFLW,
UNSHFL,
UNSHFLW,
[RISCV] [1/2] Add IR intrinsic for Zbe extension RV32/64: bcompress bdecompress RV64 ONLY: bcompressw bdecompressw Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D101143
2021-04-26 09:49:40 +08:00
// Bit Compress/Decompress implement the generic bit extract and bit deposit
// functions. This operation is also referred to as bit gather/scatter, bit
// pack/unpack, parallel extract/deposit, compress/expand, or right
// compress/right expand.
BCOMPRESS,
BCOMPRESSW,
BDECOMPRESS,
BDECOMPRESSW,
[RISCV] Add intrinsics for vmv.x.s and vmv.s.x This adds intrinsics for vmv.x.s and vmv.s.x. I've used stricter type constraints on these intrinsics than what we've been doing on the arithmetic intrinsics so far. This will allow us to not need to pass the scalar type to the Intrinsic::getDeclaration call when creating these intrinsics. A custom ISD is used for vmv.x.s in order to implement the change in computeNumSignBitsForTargetNode which can remove sign extends on the result. I also modified the MC layer description of these instructions to show the tied source/dest operand. This is different than what we do for masked instructions where we drop the tied source operand when converting to MC. But it is a more accurate description of the instruction. We can't do this for masked instructions since we use the same MC instruction for masked and unmasked. Tools like llvm-mca operate in the MC layer and rely on ins/outs and Uses/Defs for analysis so I don't know if we'll be able to maintain the current behavior for masked instructions. So I went with the accurate description here since it was easy. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D93365
2020-12-19 01:50:23 +08:00
// Vector Extension
[RISCV] Add support for splat fixed length build_vectors using RVV. Building on the fixed vector support from D95705 I've added ISD nodes for vmv.v.x and vfmv.v.f and switched to lowering the intrinsics to it. This allows us to share the same isel patterns for both. This doesn't handle splats of i64 on RV32 yet. The build_vector gets converted to a vXi32 build_vector+bitcast during type legalization. Not sure the best way to handle this at the moment. Differential Revision: https://reviews.llvm.org/D96108
2021-02-09 02:59:38 +08:00
// VMV_V_X_VL matches the semantics of vmv.v.x but includes an extra operand
// for the VL value to be used for the operation.
VMV_V_X_VL,
// VFMV_V_F_VL matches the semantics of vfmv.v.f but includes an extra operand
// for the VL value to be used for the operation.
VFMV_V_F_VL,
[RISCV] Optimize sign-extended EXTRACT_VECTOR_ELT nodes This patch custom-legalizes all integer EXTRACT_VECTOR_ELT nodes where SEW < XLEN to VMV_S_X nodes to help the compiler infer sign bits from the result. This allows us to eliminate redundant sign extensions. For parity, all integer EXTRACT_VECTOR_ELT nodes are legalized this way so that we don't need TableGen patterns for some and not others. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95741
2021-01-31 00:53:32 +08:00
// VMV_X_S matches the semantics of vmv.x.s. The result is always XLenVT sign
// extended from the vector element size.
[RISCV] Add intrinsics for vmv.x.s and vmv.s.x This adds intrinsics for vmv.x.s and vmv.s.x. I've used stricter type constraints on these intrinsics than what we've been doing on the arithmetic intrinsics so far. This will allow us to not need to pass the scalar type to the Intrinsic::getDeclaration call when creating these intrinsics. A custom ISD is used for vmv.x.s in order to implement the change in computeNumSignBitsForTargetNode which can remove sign extends on the result. I also modified the MC layer description of these instructions to show the tied source/dest operand. This is different than what we do for masked instructions where we drop the tied source operand when converting to MC. But it is a more accurate description of the instruction. We can't do this for masked instructions since we use the same MC instruction for masked and unmasked. Tools like llvm-mca operate in the MC layer and rely on ins/outs and Uses/Defs for analysis so I don't know if we'll be able to maintain the current behavior for masked instructions. So I went with the accurate description here since it was easy. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D93365
2020-12-19 01:50:23 +08:00
VMV_X_S,
[RISCV] Split RISCVISD::VMV_S_XF_VL into separate integer and FP. It's a bit silly, but it allows us to write stricter type constraints for isel. There's still some extra type checks in the generated table due to some type interference limitations around HWMode.
2021-04-06 03:54:49 +08:00
// VMV_S_X_VL matches the semantics of vmv.s.x. It carries a VL operand.
VMV_S_X_VL,
// VFMV_S_F_VL matches the semantics of vfmv.s.f. It carries a VL operand.
VFMV_S_F_VL,
[RISCV] Add ISel support for RVV vector/scalar forms This patch extends the SDNode ISel support for RVV from only the vector/vector instructions to include the vector/scalar and vector/immediate forms. It uses splat_vector to carry the scalar in each case, except when XLEN<SEW (RV32 SEW=64) when a custom node `SPLAT_VECTOR_I64` is used for type-legalization and to encode the fact that the value is sign-extended to SEW. When the scalar is a full 64-bit value we use a sequence to materialize the constant into the vector register. The non-intrinsic ISel patterns have also been split into their own file. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D93312
2020-12-15 21:05:32 +08:00
// Splats an i64 scalar to a vector type (with element type i64) where the
// scalar is a sign-extended i32.
SPLAT_VECTOR_I64,
[RISCV] Optimize SEW=64 shifts by splat on RV32. SEW=64 shifts only uses the log2(64) bits of shift amount. If we're splatting a 64 bit value in 2 parts, we can avoid splatting the upper bits and just let the low bits be sign extended. They won't be read anyway. For the purposes of SelectionDAG semantics of the generic ISD opcodes, if hi was non-zero or bit 31 of the low is 1, the shift was already undefined so it should be ok to replace high with sign extend of low. In order do be able to find the split i64 value before it becomes a stack operation, I added a new ISD opcode that will be expanded to the stack spill in PreprocessISelDAG. This new node is conceptually similar to BuildPairF64, but I expanded earlier so that we could go through regular isel to get the right VLSE opcode for the LMUL. BuildPairF64 is expanded in a CustomInserter. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D102521
2021-05-27 01:23:30 +08:00
// Splats an 64-bit value that has been split into two i32 parts. This is
// expanded late to two scalar stores and a stride 0 vector load.
SPLAT_VECTOR_SPLIT_I64_VL,
[RISCV] Custom lower ISD::VSCALE. This patch custom lowers ISD::VSCALE into a csrr vlenb followed by a shift right by 3 followed by a multiply by the scale amount. I've added computeKnownBits support to indicate that the csrr vlenb always produces 3 trailng bits of 0s so the shift right is "exact". This allows the shift and multiply sequence to be nicely optimized into a single shift or removed completely when the scale amount is a power of 2. The non power of 2 case multiplying by 24 is still producing suboptimal code. We could remove the right shift and use a multiply by 3. Hopefully we can improve DAG combine to fix that since it's not unique to this sequence. This replaces D94144. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D94249
2021-01-14 09:14:45 +08:00
// Read VLENB CSR
READ_VLENB,
[RISCV] Support fixed-length vector truncates This patch extends support for our custom-lowering of scalable-vector truncates to include those of fixed-length vectors. It does this by co-opting the custom RISCVISD::TRUNCATE_VECTOR node and adding mask and VL operands. This avoids unnecessary duplication of patterns and inflation of the ISel table. Some truncates go through CONCAT_VECTORS which currently isn't efficiently handled, as it goes through the stack. This can be improved upon in the future. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97202
2021-02-23 00:51:24 +08:00
// Truncates a RVV integer vector by one power-of-two. Carries both an extra
// mask and VL operand.
TRUNCATE_VECTOR_VL,
[RISCV] Add RVV insertelt/extractelt scalable-vector patterns Original patch by @rogfer01. This patch adds support for insertelt and extractelt operations on scalable vectors. Special care must be taken on RV32 when dealing with i64 vectors as there are no straightforward ways to insert a 64-bit element without a register of that size. To that end, both are custom-lowered to different sequences. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D94615
2021-01-12 01:42:05 +08:00
// Matches the semantics of vslideup/vslidedown. The first operand is the
[RISCV] Convert VSLIDE(UP|DOWN) nodes to "VL" versions (NFC) This patch prepares the RISCV VSLIDEUP and VSLIDEDOWN custom nodes to ones carrying additional mask and vector-length operands. This is primarily so they can be used by both systems. This also takes the opportunity to create some helper functions to deal with the common task of getting the default (unmasked) VL operands. Reviewed By: craig.topper, arcbbb Differential Revision: https://reviews.llvm.org/D96505
2021-02-11 23:01:40 +08:00
// pass-thru operand, the second is the source vector, the third is the
// XLenVT index (either constant or non-constant), the fourth is the mask
// and the fifth the VL.
VSLIDEUP_VL,
VSLIDEDOWN_VL,
[RISCV] Support vslide1up/down intrinsics for SEW=64 on RV32. This can't use our normal strategy of splatting the scalar and using a .vv operation instead of .vx. Instead this patch bitcasts the vector to the equivalent SEW=32 vector and inserts the scalar parts using two vslide1up/down. We do that unmasked and apply the mask separately at the end with a vmerge. For vslide1up there maybe some other options here like getting i64 into element 0 and using vslideup.vi with this vector as vd and the original source as vs1. Masking would still need to be done afterwards. That idea doesn't work for vslide1down. We need to slidedown and then insert a single scalar at vl-1 which we could do with a vslideup, but that assumes vl > 0 which I don't think we can assume. The i32 double slide1down implemented here is the best I could come up with and I just made vslide1up consistent. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D99910
2021-04-08 01:14:59 +08:00
// Matches the semantics of vslide1up/slide1down. The first operand is the
// source vector, the second is the XLenVT scalar value. The third and fourth
[RISCV] Optimize INSERT_VECTOR_ELT sequences This patch optimizes the codegen for INSERT_VECTOR_ELT in various ways. Primarily, it removes the use of vslidedown during lowering, and the vector element is inserted entirely using vslideup with a custom VL and slide index. Additionally, lowering of i64-element vectors on RV32 has been optimized in several ways. When the 64-bit value to insert is the same as the sign-extension of the lower 32-bits, the codegen can follow the regular path. When this is not possible, a new sequence of two i32 vslide1up instructions is used to get the vector element into a vector. This sequence was suggested by @craig.topper. From there, the value is slid into the final position for more consistent lowering across RV32 and RV64. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D98250
2021-03-08 23:20:37 +08:00
// operands are the mask and VL operands.
VSLIDE1UP_VL,
[RISCV] Support vslide1up/down intrinsics for SEW=64 on RV32. This can't use our normal strategy of splatting the scalar and using a .vv operation instead of .vx. Instead this patch bitcasts the vector to the equivalent SEW=32 vector and inserts the scalar parts using two vslide1up/down. We do that unmasked and apply the mask separately at the end with a vmerge. For vslide1up there maybe some other options here like getting i64 into element 0 and using vslideup.vi with this vector as vd and the original source as vs1. Masking would still need to be done afterwards. That idea doesn't work for vslide1down. We need to slidedown and then insert a single scalar at vl-1 which we could do with a vslideup, but that assumes vl > 0 which I don't think we can assume. The i32 double slide1down implemented here is the best I could come up with and I just made vslide1up consistent. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D99910
2021-04-08 01:14:59 +08:00
VSLIDE1DOWN_VL,
[RISCV] Add support for selecting vid.v from build_vector This patch optimizes a build_vector "index sequence" and lowers it to the existing custom RISCVISD::VID node. This pattern is common in autovectorized code. The custom node was updated to allow it to be used by both scalable and fixed-length vectors, thus avoiding pattern duplication. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96332
2021-02-09 20:09:10 +08:00
// Matches the semantics of the vid.v instruction, with a mask and VL
// operand.
VID_VL,
[RISCV] Add support for RVV int<->fp & fp<->fp conversions This patch adds support for the full range of vector int-to-float, float-to-int, and float-to-float conversions on legal types. Many conversions are supported natively in RVV so are lowered with patterns. These include conversions between (element) types of the same size, and those that are half/double the size of the input. When conversions take place between types that are less than half or more than double the size we must lower them using sequences of instructions which go via intermediate types. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95447
2021-01-22 22:54:00 +08:00
// Matches the semantics of the vfcnvt.rod function (Convert double-width
// float to single-width float, rounding towards odd). Takes a double-width
[RISCV] Support fixed-length vector FP_ROUND & FP_EXTEND This patch extends the support for vector FP_ROUND and FP_EXTEND by including support for fixed-length vector types. Since fixed-length vectors use "VL" nodes and scalable vectors can use the standard nodes, there is slightly more to do in the fixed-length case. A helper function was introduced to try and reduce the divergent paths. It is expected that this function will similarly come in useful for lowering the int-to-fp and fp-to-int operations for fixed-length vectors. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97301
2021-02-23 23:25:26 +08:00
// float vector and produces a single-width float vector. Also has a mask and
// VL operand.
VFNCVT_ROD_VL,
[RISCV] Support scalable-vector integer reduction intrinsics This patch adds support for the integer reduction intrinsics supported by RVV. This excludes "mul" which has no corresponding instruction. The reduction instructions in RVV have slightly complicated type constraints given they always produce a single "M1" vector register. They are lowered to custom nodes including the second "scalar" reduction operand to simplify the patterns and in the hope that they can be useful for future DAG combines. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95620
2021-01-28 23:31:27 +08:00
// These nodes match the semantics of the corresponding RVV vector reduction
// instructions. They produce a vector result which is the reduction
// performed over the first vector operand plus the first element of the
// second vector operand. The first operand is an unconstrained vector type,
// and the result and second operand's types are expected to be the
// corresponding full-width LMUL=1 type for the first operand:
// nxv8i8 = vecreduce_add nxv32i8, nxv8i8
// nxv2i32 = vecreduce_add nxv8i32, nxv2i32
// The different in types does introduce extra vsetvli instructions but
// similarly it reduces the number of registers consumed per reduction.
[RISCV] Add support for fixed vector reductions. I've included tests that require type legalization to split the vector. The i64 version of these scalarizes on RV32 due to type legalization visiting the result before the vector type. So we have to abort our custom expansion to avoid creating target specific nodes with an illegal type. Then type legalization ends up scalarizing. We might be able to fix this by doing custom splitting for large vectors in our handler to get down to a legal type. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D98102
2021-03-10 01:33:00 +08:00
// Also has a mask and VL operand.
VECREDUCE_ADD_VL,
VECREDUCE_UMAX_VL,
VECREDUCE_SMAX_VL,
VECREDUCE_UMIN_VL,
VECREDUCE_SMIN_VL,
VECREDUCE_AND_VL,
VECREDUCE_OR_VL,
VECREDUCE_XOR_VL,
VECREDUCE_FADD_VL,
VECREDUCE_SEQ_FADD_VL,
[RISCV] Add support for fmin/fmax vector reductions Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D101518
2021-04-29 18:35:33 +08:00
VECREDUCE_FMIN_VL,
VECREDUCE_FMAX_VL,
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
[RISCV] Add support for integer fixed vector setcc I believe I've covered all orderings of splat operands here. Better canonicalization in lowering might help reduce this. I did not handle the immediate adjustments needed for set(u)gt/set(u)lt. Testing here is limited to byte types because the scalable vector type used for masks for the store is calculated assuming 8 byte elements. But for the setcc its based on the element count of the container type for the setcc input. So they don't agree. We'll need to enhanced D96352 to handle this I think. Differential Revision: https://reviews.llvm.org/D96443
2021-02-13 01:20:54 +08:00
// Vector binary and unary ops with a mask as a third operand, and VL as a
// fourth operand.
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
// FIXME: Can we replace these with ISD::VP_*?
ADD_VL,
AND_VL,
MUL_VL,
OR_VL,
SDIV_VL,
SHL_VL,
SREM_VL,
SRA_VL,
SRL_VL,
SUB_VL,
UDIV_VL,
UREM_VL,
XOR_VL,
FADD_VL,
FSUB_VL,
FMUL_VL,
FDIV_VL,
FNEG_VL,
[RISCV] Add support for fixed vector fabs
2021-02-13 07:33:21 +08:00
FABS_VL,
[RISCV] Add support for fixed vector sqrt.
2021-02-13 07:10:18 +08:00
FSQRT_VL,
[RISCV] Add support for fixed vector FMA. Follow up to D95705. Does not include the commuting support from D95800. Differential Revision: https://reviews.llvm.org/D96103
2021-02-09 02:45:45 +08:00
FMA_VL,
[RISCV] Support fixed vector copysign. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D98394
2021-03-12 01:41:39 +08:00
FCOPYSIGN_VL,
[RISCV] Add support for integer fixed min/max This patch extends the initial fixed-length vector support to include smin, smax, umin, and umax. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96491
2021-02-11 20:45:38 +08:00
SMIN_VL,
SMAX_VL,
UMIN_VL,
UMAX_VL,
[RISCV] Custom lower vector F(MIN|MAX)NUM to vf(min|max) This patch adds support for both scalable- and fixed-length vector code lowering of the llvm.minnum and llvm.maxnum intrinsics to the equivalent RVV instructions. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D101035
2021-04-22 16:55:27 +08:00
FMINNUM_VL,
FMAXNUM_VL,
[RISCV] Add support for fixed vector MULHU/MULHS. This uses to division by constant optimization to use MULHU/MULHS. Reviewed By: frasercrmck, arcbbb Differential Revision: https://reviews.llvm.org/D96934
2021-02-19 01:09:33 +08:00
MULHS_VL,
MULHU_VL,
[RISCV] Support fixed-length vector i2fp/fp2i conversions This patch extends the support for scalable-vector int->fp and fp->int conversions by additionally handling fixed-length vectors. The existing scalable-vector lowering re-expresses widening/narrowing by x4+ conversions as standard nodes. The fixed-length vector support slots in at "the end" of this process by lowering the now equally-sized and widening/narrowing by x2 nodes to our custom VL versions. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97374
2021-02-24 18:18:28 +08:00
FP_TO_SINT_VL,
FP_TO_UINT_VL,
SINT_TO_FP_VL,
UINT_TO_FP_VL,
[RISCV] Support fixed-length vector FP_ROUND & FP_EXTEND This patch extends the support for vector FP_ROUND and FP_EXTEND by including support for fixed-length vector types. Since fixed-length vectors use "VL" nodes and scalable vectors can use the standard nodes, there is slightly more to do in the fixed-length case. A helper function was introduced to try and reduce the divergent paths. It is expected that this function will similarly come in useful for lowering the int-to-fp and fp-to-int operations for fixed-length vectors. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97301
2021-02-23 23:25:26 +08:00
FP_ROUND_VL,
FP_EXTEND_VL,
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
[RISCV] Add support for matching vwmul(u) and vwmacc(u) from fixed vectors. This adds a DAG combine to detect sext/zext inputs and emit a new ISD opcode. The extends will either be removed or replaced with narrower extends. Isel patterns are used to match add and widening mul to vwmacc similar to the recently added vmacc patterns. There's still some work to be to match vmulsu. We should also rewrite splats that were extended as scalars and then splatted. Reviewed By: arcbbb Differential Revision: https://reviews.llvm.org/D104802
2021-07-07 00:55:57 +08:00
// Widening instructions
VWMUL_VL,
VWMULU_VL,
[RISCV] Add support for integer fixed vector setcc I believe I've covered all orderings of splat operands here. Better canonicalization in lowering might help reduce this. I did not handle the immediate adjustments needed for set(u)gt/set(u)lt. Testing here is limited to byte types because the scalable vector type used for masks for the store is calculated assuming 8 byte elements. But for the setcc its based on the element count of the container type for the setcc input. So they don't agree. We'll need to enhanced D96352 to handle this I think. Differential Revision: https://reviews.llvm.org/D96443
2021-02-13 01:20:54 +08:00
// Vector compare producing a mask. Fourth operand is input mask. Fifth
// operand is VL.
SETCC_VL,
[RISCV] Add support for fixed vector vselect This patch adds support for fixed-length vector vselect. It does so by lowering them to a custom unmasked VSELECT_VL node with a vector length operand. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96768
2021-02-16 00:38:25 +08:00
// Vector select with an additional VL operand. This operation is unmasked.
VSELECT_VL,
[RISCV] Add support for fixed vector floating point setcc. This is annoying because the condition code legalization belongs to LegalizeDAG, but our custom handler runs in Legalize vector ops which occurs earlier. This adds some of the mask binary operations so that we can combine multiple compares that we need for expansion. I've also fixed up RISCVISelDAGToDAG.cpp to handle copies of masks. This patch contains a subset of the integer setcc patch as well. That patch is dependent on the integer binary ops patch. I'll rebase based on what order the patches go in. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D96567
2021-02-16 04:42:33 +08:00
// Mask binary operators.
VMAND_VL,
VMOR_VL,
VMXOR_VL,
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
// Set mask vector to all zeros or ones.
VMCLR_VL,
VMSET_VL,
[RISCV] Add support for VECTOR_REVERSE for scalable vector types. I've left mask registers to a future patch as we'll need to convert them to full vectors, shuffle, and then truncate. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D97609
2021-03-10 01:43:08 +08:00
// Matches the semantics of vrgather.vx and vrgather.vv with an extra operand
// for VL.
[RISCV] Add support for selecting vrgather.vx/vi for fixed vector splat shuffles. The test cases extract a fixed element from a vector and splat it into a vector. This gets DAG combined into a splat shuffle. I've used some very wide vectors in the test to make sure we have at least a couple tests where the element doesn't fit into the uimm5 immediate of vrgather.vi so we fall back to vrgather.vx. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D96186
2021-02-11 01:40:28 +08:00
VRGATHER_VX_VL,
[RISCV] Add support for VECTOR_REVERSE for scalable vector types. I've left mask registers to a future patch as we'll need to convert them to full vectors, shuffle, and then truncate. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D97609
2021-03-10 01:43:08 +08:00
VRGATHER_VV_VL,
VRGATHEREI16_VV_VL,
[RISCV] Add support for selecting vrgather.vx/vi for fixed vector splat shuffles. The test cases extract a fixed element from a vector and splat it into a vector. This gets DAG combined into a splat shuffle. I've used some very wide vectors in the test to make sure we have at least a couple tests where the element doesn't fit into the uimm5 immediate of vrgather.vi so we fall back to vrgather.vx. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D96186
2021-02-11 01:40:28 +08:00
[RISCV] Support fixed-length vector sign/zero extension This patch adds support for the custom lowering sign- and zero-extension of fixed-length vector types. It does so through custom nodes. Since the source and destination types are (necessarily) of different sizes, it is possible that the source type is legal whilst the larger destination type isn't. In this case the legalization makes heavy use of EXTRACT_SUBVECTOR. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97194
2021-02-17 17:08:24 +08:00
// Vector sign/zero extend with additional mask & VL operands.
VSEXT_VL,
VZEXT_VL,
[RISCV] Add support for matching vwmul(u) and vwmacc(u) from fixed vectors. This adds a DAG combine to detect sext/zext inputs and emit a new ISD opcode. The extends will either be removed or replaced with narrower extends. Isel patterns are used to match add and widening mul to vwmacc similar to the recently added vmacc patterns. There's still some work to be to match vmulsu. We should also rewrite splats that were extended as scalars and then splatted. Reviewed By: arcbbb Differential Revision: https://reviews.llvm.org/D104802
2021-07-07 00:55:57 +08:00
[RISCV] Support OR/XOR/AND reductions on vector masks This patch adds RVV codegen support for OR/XOR/AND reductions for both scalable- and fixed-length vector types. There are a few possible codegen strategies for each -- vmfirst.m, vmsbf.m, and vmsif.m could be used to some extent -- but the vpopc.m instruction was chosen since it produces the scalar result in one instruction, after which scalar instructions can finish off the computation. The reductions are lowered identically for both scalable- and fixed-length vectors, although some alternate strategies may be more optimal on fixed-length vectors since it's cheaper to get the length of those types. Other reduction types were not deemed to be relevant for mask vectors. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D100030
2021-04-07 17:03:22 +08:00
// vpopc.m with additional mask and VL operands.
VPOPC_VL,
[RISCV] Support fixed-length vector sign/zero extension This patch adds support for the custom lowering sign- and zero-extension of fixed-length vector types. It does so through custom nodes. Since the source and destination types are (necessarily) of different sizes, it is possible that the source type is legal whilst the larger destination type isn't. In this case the legalization makes heavy use of EXTRACT_SUBVECTOR. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97194
2021-02-17 17:08:24 +08:00
[RISCV] DAG nodes and pseudo instructions for CSR access New custom DAG nodes were added to represent operations on CSR. These nodes are lowered to corresponding pseudo instruction. Using the pseudo instructions allows to specify different scheduling information for operations on different system registers. It also make possible to specify dependencies of instructions on specific system registers. Differential Revision: https://reviews.llvm.org/D98936
2021-03-18 23:07:27 +08:00
// Reads value of CSR.
// The first operand is a chain pointer. The second specifies address of the
// required CSR. Two results are produced, the read value and the new chain
// pointer.
READ_CSR,
// Write value to CSR.
// The first operand is a chain pointer, the second specifies address of the
// required CSR and the third is the value to write. The result is the new
// chain pointer.
WRITE_CSR,
// Read and write value of CSR.
// The first operand is a chain pointer, the second specifies address of the
// required CSR and the third is the value to write. Two results are produced,
// the value read before the modification and the new chain pointer.
SWAP_CSR,
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
// Memory opcodes start here.
VLE_VL = ISD::FIRST_TARGET_MEMORY_OPCODE,
VSE_VL,
// WARNING: Do not add anything in the end unless you want the node to
// have memop! In fact, starting from FIRST_TARGET_MEMORY_OPCODE all
// opcodes will be thought as target memory ops!
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
};
[RISCV] Lower GREVI and GORCI as custom nodes This moves the recognition of GREVI and GORCI from TableGen patterns into a DAGCombine. This is done primarily to match "deeper" patterns in the future, like (grevi (grevi x, 1) 2) -> (grevi x, 3). TableGen is not best suited to matching patterns such as these as the compile time of the DAG matchers quickly gets out of hand due to the expansion of commutative permutations. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D91259
2020-11-11 21:41:55 +08:00
} // namespace RISCVISD
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
class RISCVTargetLowering : public TargetLowering {
const RISCVSubtarget &Subtarget;
public:
explicit RISCVTargetLowering(const TargetMachine &TM,
const RISCVSubtarget &STI);
[RISCV] Define vadd/vsub/vrsub intrinsics and lower to V instructions. This patch is based on the proposal from Roger Ferrer Ibanez. http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html Differential Revision: https://reviews.llvm.org/D93013
2020-12-11 15:16:08 +08:00
const RISCVSubtarget &getSubtarget() const { return Subtarget; }
[RISCV] Codegen for i8, i16, and i32 atomicrmw with RV32A Introduce a new RISCVExpandPseudoInsts pass to expand atomic pseudo-instructions after register allocation. This is necessary in order to ensure that register spills aren't introduced between LL and SC, thus breaking the forward progress guarantee for the operation. AArch64 does something similar for CmpXchg (though only at O0), and Mips is moving towards this approach (see D31287). See also [this mailing list post](http://lists.llvm.org/pipermail/llvm-dev/2016-May/099490.html) from James Knight, which summarises the issues with lowering to ll/sc in IR or pre-RA. See the [accompanying RFC thread](http://lists.llvm.org/pipermail/llvm-dev/2018-June/123993.html) for an overview of the lowering strategy. Differential Revision: https://reviews.llvm.org/D47882 llvm-svn: 342534
2018-09-19 18:54:22 +08:00
bool getTgtMemIntrinsic(IntrinsicInfo &Info, const CallInst &I,
MachineFunction &MF,
unsigned Intrinsic) const override;
[RISCV] Implement isLegalAddressingMode for RISC-V This has no impact on codegen for the current RISC-V unit tests or my small benchmark set and very minor changes in a few programs in the GCC torture suite. Based on this, I haven't been able to produce a representative test program that demonstrates a benefit from isLegalAddressingMode. I'm committing the patch anyway, on the basis that presenting accurate information to the target-independent code is preferable to relying on incorrect generic assumptions. llvm-svn: 330932
2018-04-26 20:13:48 +08:00
bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM, Type *Ty,
unsigned AS,
Instruction *I = nullptr) const override;
[RISCV] Implement isLegalICmpImmediate I'm unable to construct a representative test case that demonstrates the advantage, but it seems sensible to report accurate target-specific information regardless. llvm-svn: 330938
2018-04-26 21:15:17 +08:00
bool isLegalICmpImmediate(int64_t Imm) const override;
[RISCV] Implement isLegalAddImmediate This causes a trivial improvement in the recently added lsr-legaladdimm.ll test case. llvm-svn: 330937
2018-04-26 21:00:37 +08:00
bool isLegalAddImmediate(int64_t Imm) const override;
[RISCV] Implement isTruncateFree Adapted from ARM's implementation introduced in r313533 and r314280. llvm-svn: 330940
2018-04-26 21:37:00 +08:00
bool isTruncateFree(Type *SrcTy, Type *DstTy) const override;
bool isTruncateFree(EVT SrcVT, EVT DstVT) const override;
[RISCV] Implement isZextFree This returns true for 8-bit and 16-bit loads, allowing LBU/LHU to be selected and avoiding unnecessary masks. llvm-svn: 330943
2018-04-26 22:04:18 +08:00
bool isZExtFree(SDValue Val, EVT VT2) const override;
[TargetLowering][RISCV] Introduce isSExtCheaperThanZExt hook and implement for RISC-V DAGTypeLegalizer::PromoteSetCCOperands currently prefers to zero-extend operands when it is able to do so. For some targets this is more expensive than a sign-extension, which is also a valid choice. Introduce the isSExtCheaperThanZExt hook and use it in the new SExtOrZExtPromotedInteger helper. On RISC-V, we prefer sign-extension for FromTy == MVT::i32 and ToTy == MVT::i64, as it can be performed using a single instruction. Differential Revision: https://reviews.llvm.org/D52978 llvm-svn: 347977
2018-11-30 17:56:54 +08:00
bool isSExtCheaperThanZExt(EVT SrcVT, EVT DstVT) const override;
[RISCV] Make ctlz/cttz cheap to speculatively execute so CodeGenPrepare won't insert a zero check. Add additional isel patterns for ctzw/clzw instructions. Differential Revision: https://reviews.llvm.org/D91040
2020-11-10 02:05:51 +08:00
bool isCheapToSpeculateCttz() const override;
bool isCheapToSpeculateCtlz() const override;
[RISCV] Select +0.0 immediate using fmv.{w,d}.x / fcvt.d.w Floating point positive zero can be selected using fmv.w.x / fmv.d.x / fcvt.d.w and the zero source register. Differential Revision: https://reviews.llvm.org/D75729
2020-03-20 17:22:48 +08:00
bool isFPImmLegal(const APFloat &Imm, EVT VT,
bool ForCodeSize) const override;
[RISCV] Support Bit-Preserving FP in F/D Extensions Summary: This allows some integer bitwise operations to instead be performed by hardware fp instructions. This is correct because the RISC-V spec requires the F and D extensions to use the IEEE-754 standard representation, and fp register loads and stores to be bit-preserving. This is tested against the soft-float ABI, but with hardware float extensions enabled, so that the tests also ensure the optimisation also fires in this case. Reviewers: asb, luismarques Reviewed By: asb Subscribers: hiraditya, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, PkmX, jocewei, psnobl, benna, Jim, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62900 llvm-svn: 362790
2019-06-07 20:20:14 +08:00
[RISCV] Use softPromoteHalf legalization for fp16 without Zfh rather than PromoteFloat. The default legalization strategy is PromoteFloat which keeps half in single precision format through multiple floating point operations. Conversion to/from float is done at loads, stores, bitcasts, and other places that care about the exact size being 16 bits. This patches switches to the alternative method softPromoteHalf. This aims to keep the type in 16-bit format between every operation. So we promote to float and immediately round for any arithmetic operation. This should be closer to the IR semantics since we are rounding after each operation and not accumulating extra precision across multiple operations. X86 is the only other target that enables this today. See https://reviews.llvm.org/D73749 I had to update getRegisterTypeForCallingConv to force f16 to use f32 when the F extension is enabled. This way we can still pass it in the lower bits of an FPR for ilp32f and lp64f ABIs. The softPromoteHalf would otherwise always give i16 as the argument type. Reviewed By: asb, frasercrmck Differential Revision: https://reviews.llvm.org/D99148
2021-04-02 02:47:11 +08:00
bool softPromoteHalfType() const override { return true; }
/// Return the register type for a given MVT, ensuring vectors are treated
/// as a series of gpr sized integers.
MVT getRegisterTypeForCallingConv(LLVMContext &Context, CallingConv::ID CC,
EVT VT) const override;
/// Return the number of registers for a given MVT, ensuring vectors are
/// treated as a series of gpr sized integers.
unsigned getNumRegistersForCallingConv(LLVMContext &Context,
CallingConv::ID CC,
EVT VT) const override;
[RISCV] Return false from isShuffleMaskLegal except for splats. We don't support any other shuffles currently. This changes the bswap/bitreverse tests that check for this in their expansion code. Previously we expanded a byte swapping shuffle through memory. Now we're scalarizing and doing bit operations on scalars to swap bytes. In the future we can probably use vrgather.vx to do a byte swap shuffle.
2021-03-12 08:21:42 +08:00
/// Return true if the given shuffle mask can be codegen'd directly, or if it
/// should be stack expanded.
bool isShuffleMaskLegal(ArrayRef<int> M, EVT VT) const override;
[RISCV] Support Bit-Preserving FP in F/D Extensions Summary: This allows some integer bitwise operations to instead be performed by hardware fp instructions. This is correct because the RISC-V spec requires the F and D extensions to use the IEEE-754 standard representation, and fp register loads and stores to be bit-preserving. This is tested against the soft-float ABI, but with hardware float extensions enabled, so that the tests also ensure the optimisation also fires in this case. Reviewers: asb, luismarques Reviewed By: asb Subscribers: hiraditya, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, PkmX, jocewei, psnobl, benna, Jim, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62900 llvm-svn: 362790
2019-06-07 20:20:14 +08:00
bool hasBitPreservingFPLogic(EVT VT) const override;
[RISCV] Support fixed-length INSERT_VECTOR_ELT This patch enables support for lowering INSERT_VECTOR_ELT on fixed-length vector types. The strategy follows that for scalable vector types. This patch also includes a quick fix to prevent the compiler infinitely looping between lowering BUILD_VECTOR as VECTOR_SHUFFLE and back again. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97698
2021-03-01 23:31:56 +08:00
bool
shouldExpandBuildVectorWithShuffles(EVT VT,
unsigned DefinedValues) const override;
[RISCV] Implement isLegalAddressingMode for RISC-V This has no impact on codegen for the current RISC-V unit tests or my small benchmark set and very minor changes in a few programs in the GCC torture suite. Based on this, I haven't been able to produce a representative test program that demonstrates a benefit from isLegalAddressingMode. I'm committing the patch anyway, on the basis that presenting accurate information to the target-independent code is preferable to relying on incorrect generic assumptions. llvm-svn: 330932
2018-04-26 20:13:48 +08:00
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
// Provide custom lowering hooks for some operations.
SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override;
[RISCV] Custom-legalise 32-bit variable shifts on RV64 The previous DAG combiner-based approach had an issue with infinite loops between the target-dependent and target-independent combiner logic (see PR40333). Although this was worked around in rL351806, the combiner-based approach is still potentially brittle and can fail to select the 32-bit shift variant when profitable to do so, as demonstrated in the pr40333.ll test case. This patch instead introduces target-specific SelectionDAG nodes for SHLW/SRLW/SRAW and custom-lowers variable i32 shifts to them. pr40333.ll is a good example of how this approach can improve codegen. This adds DAG combine that does SimplifyDemandedBits on the operands (only lower 32-bits of first operand and lower 5 bits of second operand are read). This seems better than implementing SimplifyDemandedBitsForTargetNode as there is no guarantee that would be called (and it's not for e.g. the anyext return test cases). Also implements ComputeNumSignBitsForTargetNode. There are codegen changes in atomic-rmw.ll and atomic-cmpxchg.ll but the new instruction sequences are semantically equivalent. Differential Revision: https://reviews.llvm.org/D57085 llvm-svn: 352169
2019-01-25 13:04:00 +08:00
void ReplaceNodeResults(SDNode *N, SmallVectorImpl<SDValue> &Results,
SelectionDAG &DAG) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
[RISCV] Handle redundant SplitF64+BuildPairF64 pairs in a DAGCombine r343712 performed this optimisation during instruction selection. As Eli Friedman pointed out in post-commit review, implementing this as a DAGCombine might allow opportunities for further optimisations. llvm-svn: 343741
2018-10-04 07:30:16 +08:00
SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
[RISCV] Add implementation of targetShrinkDemandedConstant to optimize AND immediates. SimplifyDemandedBits can remove set bits from immediates from instructions like AND/OR/XOR. This can prevent them from being efficiently codegened on RISCV. This adds an initial version that tries to keep or form 12 bit sign extended immediates for AND operations to enable use of ANDI. If that doesn't work we'll try to create a 32 bit sign extended immediate to use LUI+ADDIW. More optimizations are possible for different size immediates or different operations. But this is a good starting point that already has test coverage. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D94628
2021-01-16 02:54:26 +08:00
bool targetShrinkDemandedConstant(SDValue Op, const APInt &DemandedBits,
const APInt &DemandedElts,
TargetLoweringOpt &TLO) const override;
[RISCV] Custom lower ISD::VSCALE. This patch custom lowers ISD::VSCALE into a csrr vlenb followed by a shift right by 3 followed by a multiply by the scale amount. I've added computeKnownBits support to indicate that the csrr vlenb always produces 3 trailng bits of 0s so the shift right is "exact". This allows the shift and multiply sequence to be nicely optimized into a single shift or removed completely when the scale amount is a power of 2. The non power of 2 case multiplying by 24 is still producing suboptimal code. We could remove the right shift and use a multiply by 3. Hopefully we can improve DAG combine to fix that since it's not unique to this sequence. This replaces D94144. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D94249
2021-01-14 09:14:45 +08:00
void computeKnownBitsForTargetNode(const SDValue Op,
KnownBits &Known,
const APInt &DemandedElts,
const SelectionDAG &DAG,
unsigned Depth) const override;
[RISCV] Custom-legalise 32-bit variable shifts on RV64 The previous DAG combiner-based approach had an issue with infinite loops between the target-dependent and target-independent combiner logic (see PR40333). Although this was worked around in rL351806, the combiner-based approach is still potentially brittle and can fail to select the 32-bit shift variant when profitable to do so, as demonstrated in the pr40333.ll test case. This patch instead introduces target-specific SelectionDAG nodes for SHLW/SRLW/SRAW and custom-lowers variable i32 shifts to them. pr40333.ll is a good example of how this approach can improve codegen. This adds DAG combine that does SimplifyDemandedBits on the operands (only lower 32-bits of first operand and lower 5 bits of second operand are read). This seems better than implementing SimplifyDemandedBitsForTargetNode as there is no guarantee that would be called (and it's not for e.g. the anyext return test cases). Also implements ComputeNumSignBitsForTargetNode. There are codegen changes in atomic-rmw.ll and atomic-cmpxchg.ll but the new instruction sequences are semantically equivalent. Differential Revision: https://reviews.llvm.org/D57085 llvm-svn: 352169
2019-01-25 13:04:00 +08:00
unsigned ComputeNumSignBitsForTargetNode(SDValue Op,
const APInt &DemandedElts,
const SelectionDAG &DAG,
unsigned Depth) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
// This method returns the name of a target specific DAG node.
const char *getTargetNodeName(unsigned Opcode) const override;
[RISCV] Add basic support for inline asm constraints llvm-svn: 322217
2018-01-11 04:05:09 +08:00
[RISCV] Support 'f' Inline Assembly Constraint Summary: This adds the 'f' inline assembly constraint, as supported by GCC. An 'f'-constrained operand is passed in a floating point register. Exactly which kind of floating-point register (32-bit or 64-bit) is decided based on the operand type and the available standard extensions (-f and -d, respectively). This patch adds support in both the clang frontend, and LLVM itself. Reviewers: asb, lewis-revill Reviewed By: asb Subscribers: hiraditya, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, MaskRay, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, PkmX, jocewei, psnobl, benna, Jim, s.egerton, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D65500 llvm-svn: 367403
2019-07-31 17:45:55 +08:00
ConstraintType getConstraintType(StringRef Constraint) const override;
[RISCV] Lower inline asm constraint A for RISC-V This allows arguments with the constraint A to be lowered to input nodes for RISC-V, which implies a memory address stored in a register. This patch adds the minimal amount of code required to get operands with the right constraints to compile. https://reviews.llvm.org/D54296 llvm-svn: 369095
2019-08-16 18:28:34 +08:00
unsigned getInlineAsmMemConstraint(StringRef ConstraintCode) const override;
[RISCV] Add basic support for inline asm constraints llvm-svn: 322217
2018-01-11 04:05:09 +08:00
std::pair<unsigned, const TargetRegisterClass *>
getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
StringRef Constraint, MVT VT) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
[RISCV] Lower inline asm constraints I, J & K for RISC-V This validates and lowers arguments to inline asm nodes which have the constraints I, J & K, with the following semantics (equivalent to GCC): I: Any 12-bit signed immediate. J: Immediate integer zero only. K: Any 5-bit unsigned immediate. Differential Revision: https://reviews.llvm.org/D54093 llvm-svn: 363054
2019-06-11 20:42:13 +08:00
void LowerAsmOperandForConstraint(SDValue Op, std::string &Constraint,
std::vector<SDValue> &Ops,
SelectionDAG &DAG) const override;
[RISCV] Implement lowering of ISD::SELECT Although ISD::SELECT_CC is a more natural match for RISCVISD::SELECT_CC (and ultimately the integer RISC-V conditional branch instructions), we choose to expand ISD::SELECT_CC and lower ISD::SELECT. The appropriate compare+branch will be created in the case where an ISD::SELECT condition value is created by an ISD::SETCC node, which operates on XLen types. Other datatypes such as floating point don't have conditional branch instructions, and lowering ISD::SELECT allows more flexibility for handling these cases. Differential Revision: https://reviews.llvm.org/D29937 llvm-svn: 318735
2017-11-21 15:51:32 +08:00
MachineBasicBlock *
EmitInstrWithCustomInserter(MachineInstr &MI,
MachineBasicBlock *BB) const override;
[RISCV] Define getSetCCResultType for setting vector setCC type To avoid trigger "No default SetCC type for vectors!" Assertion Differential Revision: https://reviews.llvm.org/D42675 llvm-svn: 324054
2018-02-02 10:43:18 +08:00
EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context,
EVT VT) const override;
[RISCV] Implement lowering of ISD::SELECT Although ISD::SELECT_CC is a more natural match for RISCVISD::SELECT_CC (and ultimately the integer RISC-V conditional branch instructions), we choose to expand ISD::SELECT_CC and lower ISD::SELECT. The appropriate compare+branch will be created in the case where an ISD::SELECT condition value is created by an ISD::SETCC node, which operates on XLen types. Other datatypes such as floating point don't have conditional branch instructions, and lowering ISD::SELECT allows more flexibility for handling these cases. Differential Revision: https://reviews.llvm.org/D29937 llvm-svn: 318735
2017-11-21 15:51:32 +08:00
[RISCV] Allow conversion of CC logic to bitwise logic Indicates in the TargetLowering interface that conversions from CC logic to bitwise logic are allowed. Adds tests that show the benefit when optimization opportunities are detected. Also adds tests that show that when the optimization is not applied correct code is generated (but opportunities for other optimizations remain). Differential Revision: https://reviews.llvm.org/D59596 Patch by Luís Marques. llvm-svn: 356740
2019-03-22 18:39:22 +08:00
bool convertSetCCLogicToBitwiseLogic(EVT VT) const override {
return VT.isScalarInteger();
}
[RISCV] Implement convertSelectOfConstantsToMath Summary: The current lowering of `select` on RISC-V uses a branch instruction to load a register with one or other value. This is inefficient, especially in the case of small constants that can be computed easily. By implementing the TargetLowering::convertSelectOfConstantsToMath hook, some of the simpler cases are covered that let us avoid introducing a branch in these cases. Reviewed By: luismarques Differential Revision: https://reviews.llvm.org/D79260
2020-05-02 22:05:12 +08:00
bool convertSelectOfConstantsToMath(EVT VT) const override { return true; }
[RISCV] Allow conversion of CC logic to bitwise logic Indicates in the TargetLowering interface that conversions from CC logic to bitwise logic are allowed. Adds tests that show the benefit when optimization opportunities are detected. Also adds tests that show that when the optimization is not applied correct code is generated (but opportunities for other optimizations remain). Differential Revision: https://reviews.llvm.org/D59596 Patch by Luís Marques. llvm-svn: 356740
2019-03-22 18:39:22 +08:00
[RISCV] Add codegen support for atomic load/stores with RV32A Fences are inserted according to table A.6 in the current draft of version 2.3 of the RISC-V Instruction Set Manual, which incorporates the memory model changes and definitions contributed by the RISC-V Memory Consistency Model task group. Instruction selection failures will now occur for 8/16/32-bit atomicrmw and cmpxchg operations when targeting RV32IA until lowering for these operations is added in a follow-on patch. Differential Revision: https://reviews.llvm.org/D47589 llvm-svn: 334591
2018-06-13 20:04:51 +08:00
bool shouldInsertFencesForAtomic(const Instruction *I) const override {
return isa<LoadInst>(I) || isa<StoreInst>(I);
}
[TargetLowering] Use IRBuilderBase instead of IRBuilder<> (NFC) Don't require a specific kind of IRBuilder for TargetLowering hooks. This allows us to drop the IRBuilder.h include from TargetLowering.h. Differential Revision: https://reviews.llvm.org/D103759
2021-06-06 04:56:19 +08:00
Instruction *emitLeadingFence(IRBuilderBase &Builder, Instruction *Inst,
[RISCV] Add codegen support for atomic load/stores with RV32A Fences are inserted according to table A.6 in the current draft of version 2.3 of the RISC-V Instruction Set Manual, which incorporates the memory model changes and definitions contributed by the RISC-V Memory Consistency Model task group. Instruction selection failures will now occur for 8/16/32-bit atomicrmw and cmpxchg operations when targeting RV32IA until lowering for these operations is added in a follow-on patch. Differential Revision: https://reviews.llvm.org/D47589 llvm-svn: 334591
2018-06-13 20:04:51 +08:00
AtomicOrdering Ord) const override;
[TargetLowering] Use IRBuilderBase instead of IRBuilder<> (NFC) Don't require a specific kind of IRBuilder for TargetLowering hooks. This allows us to drop the IRBuilder.h include from TargetLowering.h. Differential Revision: https://reviews.llvm.org/D103759
2021-06-06 04:56:19 +08:00
Instruction *emitTrailingFence(IRBuilderBase &Builder, Instruction *Inst,
[RISCV] Add codegen support for atomic load/stores with RV32A Fences are inserted according to table A.6 in the current draft of version 2.3 of the RISC-V Instruction Set Manual, which incorporates the memory model changes and definitions contributed by the RISC-V Memory Consistency Model task group. Instruction selection failures will now occur for 8/16/32-bit atomicrmw and cmpxchg operations when targeting RV32IA until lowering for these operations is added in a follow-on patch. Differential Revision: https://reviews.llvm.org/D47589 llvm-svn: 334591
2018-06-13 20:04:51 +08:00
AtomicOrdering Ord) const override;
[RISCV] Add an implementation of isFMAFasterThanFMulAndFAdd Start with an assumption that FMA is faster than Fmul+FAdd. If thats not true on some particular implementation we can add a tuning parameter in the future. I've update the fmuladd test cases and added new test cases for fast math flag based contraction. Differential Revision: https://reviews.llvm.org/D91987
2020-11-26 07:07:34 +08:00
bool isFMAFasterThanFMulAndFAdd(const MachineFunction &MF,
EVT VT) const override;
[RISCV] Do a sign-extension in a compare-and-swap of 32 bit in RV64A AtomicCmpSwapWithSuccess is legalised into an AtomicCmpSwap plus a comparison. This requires an extension of the value which, by default, is a zero-extension. When we later lower AtomicCmpSwap into a PseudoCmpXchg32 and then expanded in RISCVExpandPseudoInsts.cpp, the lr.w instruction does a sign-extension. This mismatch of extensions causes the comparison to fail when the compared value is negative. This change overrides TargetLowering::getExtendForAtomicOps for RISC-V so it does a sign-extension instead. Differential Revision: https://reviews.llvm.org/D58829 Patch by Ferran Pallarès Roca. llvm-svn: 355869
2019-03-12 05:41:22 +08:00
ISD::NodeType getExtendForAtomicOps() const override {
[LegalizeTypes][RISCV] Correctly sign-extend comparison for ATOMIC_CMP_XCHG Summary: Currently, the comparison argument used for ATOMIC_CMP_XCHG is legalised with GetPromotedInteger, which leaves the upper bits of the value undefind. Since this is used for comparing in an LR/SC loop with a full-width comparison, we must sign extend it. We introduce a new getExtendForAtomicCmpSwapArg to complement getExtendForAtomicOps, since many targets have compare-and-swap instructions (or pseudos) that correctly handle an any-extend input, and the existing function determines the extension of the result, whereas we are concerned with the input. This is related to https://reviews.llvm.org/D58829, which solved the issue for ATOMIC_CMP_SWAP_WITH_SUCCESS, but not the simpler ATOMIC_CMP_SWAP. Reviewers: asb, lenary, efriedma Reviewed By: asb Subscribers: arichardson, hiraditya, rbar, johnrusso, simoncook, sabuasal, niosHD, kito-cheng, shiva0217, MaskRay, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, jfb, PkmX, jocewei, psnobl, benna, Jim, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, evandro, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D74453
2020-04-01 22:50:47 +08:00
return ISD::SIGN_EXTEND;
}
ISD::NodeType getExtendForAtomicCmpSwapArg() const override {
[RISCV] Do a sign-extension in a compare-and-swap of 32 bit in RV64A AtomicCmpSwapWithSuccess is legalised into an AtomicCmpSwap plus a comparison. This requires an extension of the value which, by default, is a zero-extension. When we later lower AtomicCmpSwap into a PseudoCmpXchg32 and then expanded in RISCVExpandPseudoInsts.cpp, the lr.w instruction does a sign-extension. This mismatch of extensions causes the comparison to fail when the compared value is negative. This change overrides TargetLowering::getExtendForAtomicOps for RISC-V so it does a sign-extension instead. Differential Revision: https://reviews.llvm.org/D58829 Patch by Ferran Pallarès Roca. llvm-svn: 355869
2019-03-12 05:41:22 +08:00
return ISD::SIGN_EXTEND;
}
[RISCV] Custom lower SHL_PARTS, SRA_PARTS, SRL_PARTS When not optimizing for minimum size (-Oz) we custom lower wide shifts (SHL_PARTS, SRA_PARTS, SRL_PARTS) instead of expanding to a libcall. Differential Revision: https://reviews.llvm.org/D59477 llvm-svn: 358498
2019-04-16 22:38:32 +08:00
bool shouldExpandShift(SelectionDAG &DAG, SDNode *N) const override {
if (DAG.getMachineFunction().getFunction().hasMinSize())
return false;
return true;
}
[RISCV] Prevent re-ordering some adds after shifts Summary: DAGCombine will normally turn a `(shl (add x, c1), c2)` into `(add (shl x, c2), c1 << c2)`, where `c1` and `c2` are constants. This can be prevented by a callback in TargetLowering. On RISC-V, materialising the constant `c1 << c2` can be more expensive than materialising `c1`, because materialising the former may take more instructions, and may use a register, where materialising the latter would not. This patch implements the hook in RISCVTargetLowering to prevent this transform, in the cases where: - `c1` fits into the immediate field in an `addi` instruction. - `c1` takes fewer instructions to materialise than `c1 << c2`. In future, DAGCombine could do the check to see whether `c1` fits into an add immediate, which might simplify more targets hooks than just RISC-V. Reviewers: asb, luismarques, efriedma Reviewed By: asb Subscribers: xbolva00, lebedev.ri, craig.topper, lewis-revill, Jim, hiraditya, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, PkmX, jocewei, psnobl, benna, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62857 llvm-svn: 363736
2019-06-19 04:38:08 +08:00
bool isDesirableToCommuteWithShift(const SDNode *N,
CombineLevel Level) const override;
[RISCV] Custom lower SHL_PARTS, SRA_PARTS, SRL_PARTS When not optimizing for minimum size (-Oz) we custom lower wide shifts (SHL_PARTS, SRA_PARTS, SRL_PARTS) instead of expanding to a libcall. Differential Revision: https://reviews.llvm.org/D59477 llvm-svn: 358498
2019-04-16 22:38:32 +08:00
[RISCV] Specify registers used in DWARF exception handling Defines RISCV registers for getExceptionPointerRegister() and getExceptionSelectorRegister(). Differential Revision: https://reviews.llvm.org/D63411 Patch by Edward Jones. Modified by Alex Bradbury to add CHECK lines to exception-pointer-register.ll. llvm-svn: 365301
2019-07-08 17:16:47 +08:00
/// If a physical register, this returns the register that receives the
/// exception address on entry to an EH pad.
CodeGen: Use Register in TargetLowering
2020-04-08 22:29:30 +08:00
Register
[RISCV] Specify registers used in DWARF exception handling Defines RISCV registers for getExceptionPointerRegister() and getExceptionSelectorRegister(). Differential Revision: https://reviews.llvm.org/D63411 Patch by Edward Jones. Modified by Alex Bradbury to add CHECK lines to exception-pointer-register.ll. llvm-svn: 365301
2019-07-08 17:16:47 +08:00
getExceptionPointerRegister(const Constant *PersonalityFn) const override;
/// If a physical register, this returns the register that receives the
/// exception typeid on entry to a landing pad.
CodeGen: Use Register in TargetLowering
2020-04-08 22:29:30 +08:00
Register
[RISCV] Specify registers used in DWARF exception handling Defines RISCV registers for getExceptionPointerRegister() and getExceptionSelectorRegister(). Differential Revision: https://reviews.llvm.org/D63411 Patch by Edward Jones. Modified by Alex Bradbury to add CHECK lines to exception-pointer-register.ll. llvm-svn: 365301
2019-07-08 17:16:47 +08:00
getExceptionSelectorRegister(const Constant *PersonalityFn) const override;
[RISCV] Avoid generating AssertZext for LP64 ABI when lowering floating LibCall The patch fixed the issue that RV64 didn't clear the upper bits when return complex floating value with lp64 ABI. float _Complex complex_add(float _Complex a, float _Complex b) { return a + b; } RealResult = zero_extend(RealA + RealB) ImageResult = ImageA + ImageB Return (RealResult | (ImageResult << 32)) The patch introduces shouldExtendTypeInLibCall target hook to suppress the AssertZext generation when lowering floating LibCall. Thanks to Eli's comments from the Bugzilla https://bugs.llvm.org/show_bug.cgi?id=42820 Differential Revision: https://reviews.llvm.org/D65497 llvm-svn: 370275
2019-08-29 07:40:37 +08:00
bool shouldExtendTypeInLibCall(EVT Type) const override;
[RISCV] Use sign extend for i32 arguments and returns in makeLibCall on RV64. As far as I know 32 bits arguments and returns on RV64 are always sign extended to i64. So I think we should be taking this into account around libcalls. Reviewed By: luismarques Differential Revision: https://reviews.llvm.org/D95285
2021-01-26 01:16:45 +08:00
bool shouldSignExtendTypeInLibCall(EVT Type, bool IsSigned) const override;
[RISCV] Avoid generating AssertZext for LP64 ABI when lowering floating LibCall The patch fixed the issue that RV64 didn't clear the upper bits when return complex floating value with lp64 ABI. float _Complex complex_add(float _Complex a, float _Complex b) { return a + b; } RealResult = zero_extend(RealA + RealB) ImageResult = ImageA + ImageB Return (RealResult | (ImageResult << 32)) The patch introduces shouldExtendTypeInLibCall target hook to suppress the AssertZext generation when lowering floating LibCall. Thanks to Eli's comments from the Bugzilla https://bugs.llvm.org/show_bug.cgi?id=42820 Differential Revision: https://reviews.llvm.org/D65497 llvm-svn: 370275
2019-08-29 07:40:37 +08:00
[RISCV] Implement the TargetLowering::getRegisterByName hook Summary: The hook should work for any RISC-V register. Non-allocatable registers do not need to be reserved, for the remaining the hook will only succeed if you pass clang the -ffixed-xX flag. This builds upon D67185, which currently only allows reserving GPRs. Reviewers: asb, lenary Reviewed By: lenary Tags: #llvm Differential Revision: https://reviews.llvm.org/D69130
2019-11-04 19:21:51 +08:00
/// Returns the register with the specified architectural or ABI name. This
/// method is necessary to lower the llvm.read_register.* and
/// llvm.write_register.* intrinsics. Allocatable registers must be reserved
/// with the clang -ffixed-xX flag for access to be allowed.
CodeGen: Use LLT instead of EVT in getRegisterByName Only PPC seems to be using it, and only checks some simple cases and doesn't distinguish between FP. Just switch to using LLT to simplify use from GlobalISel.
2019-12-28 22:18:56 +08:00
Register getRegisterByName(const char *RegName, LLT VT,
[RISCV] Implement the TargetLowering::getRegisterByName hook Summary: The hook should work for any RISC-V register. Non-allocatable registers do not need to be reserved, for the remaining the hook will only succeed if you pass clang the -ffixed-xX flag. This builds upon D67185, which currently only allows reserving GPRs. Reviewers: asb, lenary Reviewed By: lenary Tags: #llvm Differential Revision: https://reviews.llvm.org/D69130
2019-11-04 19:21:51 +08:00
const MachineFunction &MF) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
// Lower incoming arguments, copy physregs into vregs
SDValue LowerFormalArguments(SDValue Chain, CallingConv::ID CallConv,
bool IsVarArg,
const SmallVectorImpl<ISD::InputArg> &Ins,
const SDLoc &DL, SelectionDAG &DAG,
SmallVectorImpl<SDValue> &InVals) const override;
Revert "[RISCV] Make CanLowerReturn protected for downstream maintenance" This reverts commit d775841d7d6ee3e8bbf3a420590be9bb19433eaa.
2020-05-12 18:49:17 +08:00
bool CanLowerReturn(CallingConv::ID CallConv, MachineFunction &MF,
bool IsVarArg,
const SmallVectorImpl<ISD::OutputArg> &Outs,
LLVMContext &Context) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
SDValue LowerReturn(SDValue Chain, CallingConv::ID CallConv, bool IsVarArg,
const SmallVectorImpl<ISD::OutputArg> &Outs,
const SmallVectorImpl<SDValue> &OutVals, const SDLoc &DL,
SelectionDAG &DAG) const override;
[RISCV] Initial support for function calls Note that this is just enough for simple function call examples to generate working code. Support for varargs etc follows in future patches. Differential Revision: https://reviews.llvm.org/D29936 llvm-svn: 317691
2017-11-08 21:41:21 +08:00
SDValue LowerCall(TargetLowering::CallLoweringInfo &CLI,
SmallVectorImpl<SDValue> &InVals) const override;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
bool shouldConvertConstantLoadToIntImm(const APInt &Imm,
Type *Ty) const override {
return true;
}
[RISCV] Implement mayBeEmittedAsTailCall for tail call optimization Implement TargetLowering callback mayBeEmittedAsTailCall for riscv in CodeGenPrepare, which will duplicate return instructions to enable tailcall optimization. Differential Revision: https://reviews.llvm.org/D73699
2020-02-18 23:53:26 +08:00
bool mayBeEmittedAsTailCall(const CallInst *CI) const override;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
bool shouldConsiderGEPOffsetSplit() const override { return true; }
[RISCV] Optimize multiplication by constant ... to shift/add or shift/sub. Do not enable it on riscv32 with the M extension where decomposeMulByConstant may not be an optimization. Reviewed By: luismarques, MaskRay Differential Revision: https://reviews.llvm.org/D82660
2020-07-08 09:48:46 +08:00
bool decomposeMulByConstant(LLVMContext &Context, EVT VT,
SDValue C) const override;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
TargetLowering::AtomicExpansionKind
shouldExpandAtomicRMWInIR(AtomicRMWInst *AI) const override;
[TargetLowering] Use IRBuilderBase instead of IRBuilder<> (NFC) Don't require a specific kind of IRBuilder for TargetLowering hooks. This allows us to drop the IRBuilder.h include from TargetLowering.h. Differential Revision: https://reviews.llvm.org/D103759
2021-06-06 04:56:19 +08:00
Value *emitMaskedAtomicRMWIntrinsic(IRBuilderBase &Builder, AtomicRMWInst *AI,
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
Value *AlignedAddr, Value *Incr,
Value *Mask, Value *ShiftAmt,
AtomicOrdering Ord) const override;
TargetLowering::AtomicExpansionKind
shouldExpandAtomicCmpXchgInIR(AtomicCmpXchgInst *CI) const override;
[TargetLowering] Use IRBuilderBase instead of IRBuilder<> (NFC) Don't require a specific kind of IRBuilder for TargetLowering hooks. This allows us to drop the IRBuilder.h include from TargetLowering.h. Differential Revision: https://reviews.llvm.org/D103759
2021-06-06 04:56:19 +08:00
Value *emitMaskedAtomicCmpXchgIntrinsic(IRBuilderBase &Builder,
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
AtomicCmpXchgInst *CI,
Value *AlignedAddr, Value *CmpVal,
Value *NewVal, Value *Mask,
AtomicOrdering Ord) const override;
[RISCV] Initial support of LoopVectorizer for RISC-V Vector. Define an option -riscv-vector-bits-max to specify the maximum vector bits for vectorizer. Loop vectorizer will use the value to check if it is safe to use the whole vector registers to vectorize the loop. It is not the optimum solution for loop vectorizing for scalable vector. It assumed the whole vector registers will be used to vectorize the code. If it is possible, we should configure vl to do vectorize instead of using whole vector registers. We only consider LMUL = 1 in this patch. This patch just an initial work for loop vectorizer for RISC-V Vector. Differential Revision: https://reviews.llvm.org/D95659
2021-01-27 15:00:46 +08:00
/// Returns true if the target allows unaligned memory accesses of the
/// specified type.
bool allowsMisalignedMemoryAccesses(
EVT VT, unsigned AddrSpace = 0, Align Alignment = Align(1),
MachineMemOperand::Flags Flags = MachineMemOperand::MONone,
bool *Fast = nullptr) const override;
[RISCV] Support inline asm for vector instructions. Types of fractional LMUL and LMUL=1 are all using VR register class. When using inline asm, it will use the first type in the register class as the type for the register. It is not necessary the same as the value type. We need to use INSERT_SUBVECTOR/EXTRACT_SUBVECToR/BITCAST to make it legal to put the value in the corresponding register class. Differential Revision: https://reviews.llvm.org/D97480
2021-03-09 08:53:02 +08:00
bool splitValueIntoRegisterParts(SelectionDAG &DAG, const SDLoc &DL,
SDValue Val, SDValue *Parts,
unsigned NumParts, MVT PartVT,
Optional<CallingConv::ID> CC) const override;
SDValue
joinRegisterPartsIntoValue(SelectionDAG &DAG, const SDLoc &DL,
const SDValue *Parts, unsigned NumParts,
MVT PartVT, EVT ValueVT,
Optional<CallingConv::ID> CC) const override;
[RISCV] Move instruction information into the RISCVII namespace (NFC) Move instruction attributes into the `RISCVII` namespace and add associated helper functions. Differential Revision: https://reviews.llvm.org/D102268
2021-05-12 01:17:26 +08:00
static RISCVII::VLMUL getLMUL(MVT VT);
static unsigned getRegClassIDForLMUL(RISCVII::VLMUL LMul);
[RISCV] Support extraction of misaligned subvectors This patch extends the support for RVV EXTRACT_SUBVECTOR to cover those which don't align to a vector register boundary. It accomplishes this by extracting the nearest register-sized subvector (a subregister operation), then sliding the vector down with VSLIDEDOWN and extracting the subvector from the first position (a COPY operation). Since this procedure involves the use of VSCALE and multiplication, the handling of such operations is done during lowering to simplify the implementation and make use of DAG combining. This necessitated moving some helper functions from RISCVISelDAGToDAG to RISCVTargetLowering. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96959
2021-02-18 20:48:14 +08:00
static unsigned getSubregIndexByMVT(MVT VT, unsigned Index);
static unsigned getRegClassIDForVecVT(MVT VT);
static std::pair<unsigned, unsigned>
decomposeSubvectorInsertExtractToSubRegs(MVT VecVT, MVT SubVecVT,
unsigned InsertExtractIdx,
const RISCVRegisterInfo *TRI);
[RISCV] Add new entry points to getContainerForFixedLengthVector While working on adding fixed-length vectors to the calling convention, it was necessary to be able to query for a fixed-length vector container type without access to an instance of SelectionDAG. This patch modifies the "main" getContainerForFixedLengthVector function to use an instance of TargetLowering rather than SelectionDAG, and preserves the SelectionDAG overload as a wrapper. An additional non-static version of the function was also added to simplify the common case in RISCVTargetLowering. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97925
2021-03-04 17:55:14 +08:00
MVT getContainerForFixedLengthVector(MVT VT) const;
[RISCV] Support extraction of misaligned subvectors This patch extends the support for RVV EXTRACT_SUBVECTOR to cover those which don't align to a vector register boundary. It accomplishes this by extracting the nearest register-sized subvector (a subregister operation), then sliding the vector down with VSLIDEDOWN and extracting the subvector from the first position (a COPY operation). Since this procedure involves the use of VSCALE and multiplication, the handling of such operations is done during lowering to simplify the implementation and make use of DAG combining. This necessitated moving some helper functions from RISCVISelDAGToDAG to RISCVTargetLowering. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96959
2021-02-18 20:48:14 +08:00
[RISCV] Support scalable-vector masked gather operations This patch supports the masked gather intrinsics in RVV. The RVV indexed load/store instructions only support the "unsigned unscaled" addressing mode; indices are implicitly zero-extended or truncated to XLEN and are treated as byte offsets. This ISA supports the intrinsics directly, but not the majority of various forms of the MGATHER SDNode that LLVM combines to. Any signed or scaled indexing is extended to the XLEN value type and scaled accordingly. This is done during DAG combining as widening the index types to XLEN may produce illegal vectors that require splitting, e.g. nxv16i8->nxv16i64. Support for scalable-vector CONCAT_VECTORS was added to avoid spilling via the stack when lowering split legalized index operands. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96263
2021-02-04 17:56:01 +08:00
bool shouldRemoveExtendFromGSIndex(EVT VT) const override;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
private:
[RISCV] Support vector types in combination with fastcc This patch extends the RISC-V lowering of the 'fastcc' calling convention to vector types, both fixed-length and scalable. Without this patch, any function passing or returning vector types by value would throw a compiler error. Vectors are handled in 'fastcc' much as they are in the default calling convention, the noticeable difference being the extended set of scalar GPR registers that can be used to pass vectors indirectly. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D102505
2021-05-14 00:34:41 +08:00
/// RISCVCCAssignFn - This target-specific function extends the default
/// CCValAssign with additional information used to lower RISC-V calling
/// conventions.
typedef bool RISCVCCAssignFn(const DataLayout &DL, RISCVABI::ABI,
unsigned ValNo, MVT ValVT, MVT LocVT,
CCValAssign::LocInfo LocInfo,
ISD::ArgFlagsTy ArgFlags, CCState &State,
bool IsFixed, bool IsRet, Type *OrigTy,
const RISCVTargetLowering &TLI,
Optional<unsigned> FirstMaskArgument);
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
void analyzeInputArgs(MachineFunction &MF, CCState &CCInfo,
[RISCV] Support vector types in combination with fastcc This patch extends the RISC-V lowering of the 'fastcc' calling convention to vector types, both fixed-length and scalable. Without this patch, any function passing or returning vector types by value would throw a compiler error. Vectors are handled in 'fastcc' much as they are in the default calling convention, the noticeable difference being the extended set of scalar GPR registers that can be used to pass vectors indirectly. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D102505
2021-05-14 00:34:41 +08:00
const SmallVectorImpl<ISD::InputArg> &Ins, bool IsRet,
RISCVCCAssignFn Fn) const;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
void analyzeOutputArgs(MachineFunction &MF, CCState &CCInfo,
const SmallVectorImpl<ISD::OutputArg> &Outs,
[RISCV] Support vector types in combination with fastcc This patch extends the RISC-V lowering of the 'fastcc' calling convention to vector types, both fixed-length and scalable. Without this patch, any function passing or returning vector types by value would throw a compiler error. Vectors are handled in 'fastcc' much as they are in the default calling convention, the noticeable difference being the extended set of scalar GPR registers that can be used to pass vectors indirectly. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D102505
2021-05-14 00:34:41 +08:00
bool IsRet, CallLoweringInfo *CLI,
RISCVCCAssignFn Fn) const;
[RISCV] Generate address sequences suitable for mcmodel=medium This patch adds an implementation of a PC-relative addressing sequence to be used when -mcmodel=medium is specified. With absolute addressing, a 'medium' codemodel may cause addresses to be out of range. This is because while 'medium' implies a 2 GiB addressing range, this 2 GiB can be at any offset as opposed to 'small', which implies the first 2 GiB only. Note that LLVM/Clang currently specifies code models differently to GCC, where small and medium imply the same functionality as GCC's medlow and medany respectively. Differential Revision: https://reviews.llvm.org/D54143 Patch by Lewis Revill. llvm-svn: 357393
2019-04-01 22:42:56 +08:00
template <class NodeTy>
[RISCV] Add lowering of addressing sequences for PIC This patch allows lowering of PIC addresses by using PC-relative addressing for DSO-local symbols and accessing the address through the global offset table for non-DSO-local symbols. Differential Revision: https://reviews.llvm.org/D55303 llvm-svn: 363058
2019-06-11 20:57:47 +08:00
SDValue getAddr(NodeTy *N, SelectionDAG &DAG, bool IsLocal = true) const;
[RISCV] Generate address sequences suitable for mcmodel=medium This patch adds an implementation of a PC-relative addressing sequence to be used when -mcmodel=medium is specified. With absolute addressing, a 'medium' codemodel may cause addresses to be out of range. This is because while 'medium' implies a 2 GiB addressing range, this 2 GiB can be at any offset as opposed to 'small', which implies the first 2 GiB only. Note that LLVM/Clang currently specifies code models differently to GCC, where small and medium imply the same functionality as GCC's medlow and medany respectively. Differential Revision: https://reviews.llvm.org/D54143 Patch by Lewis Revill. llvm-svn: 357393
2019-04-01 22:42:56 +08:00
[RISCV] Add lowering of global TLS addresses This patch adds lowering for global TLS addresses for the TLS models of InitialExec, GlobalDynamic, LocalExec and LocalDynamic. LocalExec support required using a 4-operand add instruction, which uses the fourth operand to express a relocation on the symbol. The necessary fixup is emitted when the instruction is emitted. Differential Revision: https://reviews.llvm.org/D55305 llvm-svn: 363771
2019-06-19 16:40:59 +08:00
SDValue getStaticTLSAddr(GlobalAddressSDNode *N, SelectionDAG &DAG,
bool UseGOT) const;
SDValue getDynamicTLSAddr(GlobalAddressSDNode *N, SelectionDAG &DAG) const;
[RISCV] Codegen support for memory operations on global addresses Differential Revision: https://reviews.llvm.org/D39103 llvm-svn: 317688
2017-11-08 21:24:21 +08:00
SDValue lowerGlobalAddress(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support and tests for a variety of additional LLVM IR constructs Previous patches primarily ensured that codegen was possible for the standard RISC-V instructions. However, there are a number of IR inputs that wouldn't be appropriately lowered. This patch both adds test cases and supports lowering for a number of these cases: * Improved sext/zext/trunc support * Support for setcc variants that don't map directly to RISC-V instructions * Lowering mul, and hence support for external symbols * addc, adde, subc, sube * mulhs, srem, mulhu, urem, udiv, sdiv * {srl,sra,shl}_parts * brind * br_jt * bswap, ctlz, cttz, ctpop * rotl, rotr * BlockAddress operands Differential Revision: https://reviews.llvm.org/D29938 llvm-svn: 318737
2017-11-21 16:11:03 +08:00
SDValue lowerBlockAddress(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add codegen for RV32F floating point load/store As part of this, add support for load/store from the constant pool. This is used to materialise f32 constants. llvm-svn: 327979
2018-03-20 21:26:12 +08:00
SDValue lowerConstantPool(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Basic jump table lowering This patch enables jump table lowering in the RISC-V backend. In addition to the test case included, the new lowering was tested by compiling the OCaml runtime and running it under qemu. Differential Revision: https://reviews.llvm.org/D92097
2020-11-25 22:01:19 +08:00
SDValue lowerJumpTable(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add lowering of global TLS addresses This patch adds lowering for global TLS addresses for the TLS models of InitialExec, GlobalDynamic, LocalExec and LocalDynamic. LocalExec support required using a 4-operand add instruction, which uses the fourth operand to express a relocation on the symbol. The necessary fixup is emitted when the instruction is emitted. Differential Revision: https://reviews.llvm.org/D55305 llvm-svn: 363771
2019-06-19 16:40:59 +08:00
SDValue lowerGlobalTLSAddress(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Implement lowering of ISD::SELECT Although ISD::SELECT_CC is a more natural match for RISCVISD::SELECT_CC (and ultimately the integer RISC-V conditional branch instructions), we choose to expand ISD::SELECT_CC and lower ISD::SELECT. The appropriate compare+branch will be created in the case where an ISD::SELECT condition value is created by an ISD::SETCC node, which operates on XLen types. Other datatypes such as floating point don't have conditional branch instructions, and lowering ISD::SELECT allows more flexibility for handling these cases. Differential Revision: https://reviews.llvm.org/D29937 llvm-svn: 318735
2017-11-21 15:51:32 +08:00
SDValue lowerSELECT(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add RISCVISD::BR_CC similar to RISCVISD::SELECT_CC. This allows me to introduce similar combines for branches as we have recently added for SELECT_CC. Some of them are less useful for standalone setccs and only help branch instructions. By having a BR_CC node its easier to only affect branches. I'm using CondCodeSDNode to make isel patterns easier to write so we can refer to the codes by name. SELECT_CC uses a constant instead. I've translated the condition code just like SELECT_CC so we need less patterns for the swapped conditions. This includes special cases for X < 1 and X > -1 that get translated to blez and bgez by using a 0 constant. computeKnownBitsForTargetNode support for SELECT_CC is added to allow MaskedValueIsZero to work for cases where the true and false values of the SELECT_CC are setccs and the result of the SELECT_CC is used by a BR_CC. This was needed to avoid regressions in some of the overflow tests. Reviewed By: luismarques Differential Revision: https://reviews.llvm.org/D98159
2021-03-16 02:44:59 +08:00
SDValue lowerBRCOND(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support for varargs Includes support for expanding va_copy. Also adds support for using 'aligned' registers when necessary for vararg calls, and ensure the frame pointer always points to the bottom of the vararg spill region. This is necessary to ensure that the saved return address and stack pointer are always available at fixed known offsets of the frame pointer. Differential Revision: https://reviews.llvm.org/D40805 llvm-svn: 322215
2018-01-11 03:41:03 +08:00
SDValue lowerVASTART(SDValue Op, SelectionDAG &DAG) const;
[RISCV][NFC] Fix naming of RISCVISelLowering::{LowerRETURNADDR,LowerFRAMEADDR} Rename to lowerRETURNADDR, lowerFRAMEADDR in order to be consistent with the LLVM coding style and the other functions in this file. llvm-svn: 343752
2018-10-04 13:27:50 +08:00
SDValue lowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Custom lower SHL_PARTS, SRA_PARTS, SRL_PARTS When not optimizing for minimum size (-Oz) we custom lower wide shifts (SHL_PARTS, SRA_PARTS, SRL_PARTS) instead of expanding to a libcall. Differential Revision: https://reviews.llvm.org/D59477 llvm-svn: 358498
2019-04-16 22:38:32 +08:00
SDValue lowerShiftLeftParts(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerShiftRightParts(SDValue Op, SelectionDAG &DAG, bool IsSRA) const;
[RISCV][SelectionDAG] Introduce an ISD::SPLAT_VECTOR_PARTS node that can represent a splat of 2 i32 values into a nxvXi64 vector for riscv32. On riscv32, i64 isn't a legal scalar type but we would like to support scalable vectors of i64. This patch introduces a new node that can represent a splat made of multiple scalar values. I've used this new node to solve the current crashes we experience when getConstant is used after type legalization. For RISCV, we are now default expanding SPLAT_VECTOR to SPLAT_VECTOR_PARTS when needed and then handling the SPLAT_VECTOR_PARTS later during LegalizeOps. I've remove the special case I previously put in for ABS for D97991 as the default expansion is now able to succesfully use getConstant. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D98004
2021-03-11 01:46:16 +08:00
SDValue lowerSPLAT_VECTOR_PARTS(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Lower splats of non-constant i1s as SETCCs This patch adds support for splatting i1 types to fixed-length or scalable vector types. It does so by lowering the operation to a SETCC of the equivalent i8 type. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D101465
2021-04-28 23:11:57 +08:00
SDValue lowerVectorMaskSplat(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add ISel patterns for scalable mask exts & truncs Original patch by @rogfer01. This patch adds support for sign-, zero-, and any-extension from scalable mask vector types to integer vector types, as well as truncation in the opposite direction. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D94590
2021-01-13 20:27:05 +08:00
SDValue lowerVectorMaskExt(SDValue Op, SelectionDAG &DAG,
int64_t ExtTrueVal) const;
SDValue lowerVectorMaskTrunc(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add RVV insertelt/extractelt scalable-vector patterns Original patch by @rogfer01. This patch adds support for insertelt and extractelt operations on scalable vectors. Special care must be taken on RV32 when dealing with i64 vectors as there are no straightforward ways to insert a 64-bit element without a register of that size. To that end, both are custom-lowered to different sequences. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D94615
2021-01-12 01:42:05 +08:00
SDValue lowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerEXTRACT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support llvm.thread.pointer Fixes https://bugs.llvm.org/show_bug.cgi?id=45303 (clang crashed on __builtin_thread_pointer) Reviewed By: lenary, MaskRay, luismarques Differential Revision: https://reviews.llvm.org/D76828
2020-03-28 08:27:00 +08:00
SDValue LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Define vsadd/vsaddu/vssub/vssubu intrinsics. We work with @rogfer01 from BSC to come out this patch. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: ShihPo Hung <shihpo.hung@sifive.com> Co-Authored-by: Monk Chiang <monk.chiang@sifive.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D93366
2020-12-17 13:45:52 +08:00
SDValue LowerINTRINSIC_W_CHAIN(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support scalable-vector integer reduction intrinsics This patch adds support for the integer reduction intrinsics supported by RVV. This excludes "mul" which has no corresponding instruction. The reduction instructions in RVV have slightly complicated type constraints given they always produce a single "M1" vector register. They are lowered to custom nodes including the second "scalar" reduction operand to simplify the patterns and in the hope that they can be useful for future DAG combines. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95620
2021-01-28 23:31:27 +08:00
SDValue lowerVECREDUCE(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support OR/XOR/AND reductions on vector masks This patch adds RVV codegen support for OR/XOR/AND reductions for both scalable- and fixed-length vector types. There are a few possible codegen strategies for each -- vmfirst.m, vmsbf.m, and vmsif.m could be used to some extent -- but the vpopc.m instruction was chosen since it produces the scalar result in one instruction, after which scalar instructions can finish off the computation. The reductions are lowered identically for both scalable- and fixed-length vectors, although some alternate strategies may be more optimal on fixed-length vectors since it's cheaper to get the length of those types. Other reduction types were not deemed to be relevant for mask vectors. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D100030
2021-04-07 17:03:22 +08:00
SDValue lowerVectorMaskVECREDUCE(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support the scalable-vector fadd reduction intrinsic This patch adds support for both the fadd reduction intrinsic, in both the ordered and unordered modes. The fmin and fmax intrinsics are not currently supported due to a discrepancy between the LLVM semantics and the RVV ISA behaviour with regards to signaling NaNs. This behaviour is likely fixed in version 2.3 of the RISC-V F/D/Q extension, but until then the intrinsics can be left unsupported. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95870
2021-02-02 22:34:51 +08:00
SDValue lowerFPVECREDUCE(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support insertion of misaligned subvectors This patch extends the support for RVV INSERT_SUBVECTOR to cover those which don't align to a vector register boundary. Like the support for EXTRACT_SUBVECTOR in D96959, it accomplishes this by extracting the nearest register-sized subvector (a subregister operation), then sliding the vector down with VSLIDEDOWN, inserting the subvector to the first position, and sliding the vector back up again afterwards. Unlike subvector extraction, for vectors that occupy less than a full vector register we must preserve the untouched elements. We do this by lowering to an LMUL=1 INSERT_SUBVECTOR using the above method and lowering that to a VSLIDEUP with a zero offset. This uses a tail-undisturbed policy and so has the effect of "sliding in" the subvector elements while preserving the surrounding ones. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96972
2021-02-19 00:48:49 +08:00
SDValue lowerINSERT_SUBVECTOR(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support extraction of misaligned subvectors This patch extends the support for RVV EXTRACT_SUBVECTOR to cover those which don't align to a vector register boundary. It accomplishes this by extracting the nearest register-sized subvector (a subregister operation), then sliding the vector down with VSLIDEDOWN and extracting the subvector from the first position (a COPY operation). Since this procedure involves the use of VSCALE and multiplication, the handling of such operations is done during lowering to simplify the implementation and make use of DAG combining. This necessitated moving some helper functions from RISCVISelDAGToDAG to RISCVTargetLowering. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96959
2021-02-18 20:48:14 +08:00
SDValue lowerEXTRACT_SUBVECTOR(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add support for the stepvector intrinsic This adds almost everything required for supporting the new stepvector intrinsic on RVV. It is lowered to the existing VID_VL SDNode. The only exception is a limitation that RV32 cannot yet lower the intrinsic on i64 vectors. This is because the step operand is (currently) required to be at least as large as the vector element type. I will look into patching that out and loosening the requirement to only an integer pointer type. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D99594
2021-03-30 21:49:25 +08:00
SDValue lowerSTEP_VECTOR(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add support for VECTOR_REVERSE for scalable vector types. I've left mask registers to a future patch as we'll need to convert them to full vectors, shuffle, and then truncate. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D97609
2021-03-10 01:43:08 +08:00
SDValue lowerVECTOR_REVERSE(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Optimize fixed vector ABS. Fix crash on scalable vector ABS for SEW=64 with RV32. The default fixed vector expansion uses sra+xor+add since it can't see that smax is legal due to our custom handling. So we select smax(X, sub(0, X)) manually. Scalable vectors are able to use the smax expansion automatically for most cases. It crashes in one case because getConstant can't build a SPLAT_VECTOR for nxvXi64 when i64 scalars aren't legal. So we manually emit a SPLAT_VECTOR_I64 for that case. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D97991
2021-03-10 00:51:01 +08:00
SDValue lowerABS(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Lower scalable vector masked loads to intrinsics to match fixed vectors and reduce isel patterns. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D98840
2021-03-20 01:39:33 +08:00
SDValue lowerMLOAD(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerMSTORE(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support fixed vector copysign. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D98394
2021-03-12 01:41:39 +08:00
SDValue lowerFixedLengthVectorFCOPYSIGNToRVV(SDValue Op,
SelectionDAG &DAG) const;
[RISCV] Add support for fixed vector masked gather/scatter. I've split the gather/scatter custom handler to avoid complicating it with even more differences between gather/scatter. Tests are the scalable vector tests with the vscale removed and dropped the tests that used vector.insert. We're probably not as thorough on the splitting cases since we use 128 for VLEN here but scalable vector use a known min size of 64. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D98991
2021-03-23 00:54:17 +08:00
SDValue lowerMGATHER(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerMSCATTER(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
SDValue lowerFixedLengthVectorLoadToRVV(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerFixedLengthVectorStoreToRVV(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add support for integer fixed vector setcc I believe I've covered all orderings of splat operands here. Better canonicalization in lowering might help reduce this. I did not handle the immediate adjustments needed for set(u)gt/set(u)lt. Testing here is limited to byte types because the scalable vector type used for masks for the store is calculated assuming 8 byte elements. But for the setcc its based on the element count of the container type for the setcc input. So they don't agree. We'll need to enhanced D96352 to handle this I think. Differential Revision: https://reviews.llvm.org/D96443
2021-02-13 01:20:54 +08:00
SDValue lowerFixedLengthVectorSetccToRVV(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add support for fixed vector mask logic operations. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D96741
2021-02-17 01:26:22 +08:00
SDValue lowerFixedLengthVectorLogicOpToRVV(SDValue Op, SelectionDAG &DAG,
unsigned MaskOpc,
unsigned VecOpc) const;
[RISCV] Protect the SHL/SRA/SRL handlers in LowerOperation against being called for an illegal i32 shift amount. It seems it is possible for DAG combine to create a shl with an i64 result type and an i32 shift amount. This is ok before type legalization since the type don't need to match in SelectionDAG. This results in type legalization calling LowerOperation to legalize just the amount. We weren't expecting this so we asserted for not finding a fixed vector shift. To fix this, I've added a check for the fixed vector case and returned SDValue() to get the default type legalizer. I've factored all shifts together and added a fixed vector specific handler to avoid repeating similar code for each in LowerOperation. The particular case I found was exposed by D104581, but the bad shift is created after that patch triggers.
2021-06-29 14:09:58 +08:00
SDValue lowerFixedLengthVectorShiftToRVV(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add support for fixed vector vselect This patch adds support for fixed-length vector vselect. It does so by lowering them to a custom unmasked VSELECT_VL node with a vector length operand. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96768
2021-02-16 00:38:25 +08:00
SDValue lowerFixedLengthVectorSelectToRVV(SDValue Op,
SelectionDAG &DAG) const;
[RISCV] Add support for fixed vector mask logic operations. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D96741
2021-02-17 01:26:22 +08:00
SDValue lowerToScalableOp(SDValue Op, SelectionDAG &DAG, unsigned NewOpc,
bool HasMask = true) const;
[RISCV][VP] Lower VP ISD nodes to RVV instructions This patch supports all of the current set of VP integer binary intrinsics by lowering them to to RVV instructions. It does so by using the existing RISCVISD *_VL custom nodes as an intermediate layer. Both scalable and fixed-length vectors are supported by using this method. One notable change to the existing vector codegen strategy is that scalable all-ones and all-zeros mask SPLAT_VECTORs are now lowered to RISCVISD VMSET_VL and VMCLR_VL nodes to match their fixed-length BUILD_VECTOR counterparts. This allows them to reuse the existing "all-ones" VL patterns. To reduce the size of the phabricator diff, some tests are intentionally left out and will be added later if the patch is accepted. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D101826
2021-04-29 23:58:56 +08:00
SDValue lowerVPOp(SDValue Op, SelectionDAG &DAG, unsigned RISCVISDOpc) const;
[RISCV] Support fixed-length vector sign/zero extension This patch adds support for the custom lowering sign- and zero-extension of fixed-length vector types. It does so through custom nodes. Since the source and destination types are (necessarily) of different sizes, it is possible that the source type is legal whilst the larger destination type isn't. In this case the legalization makes heavy use of EXTRACT_SUBVECTOR. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D97194
2021-02-17 17:08:24 +08:00
SDValue lowerFixedLengthVectorExtendToRVV(SDValue Op, SelectionDAG &DAG,
unsigned ExtendOpc) const;
[RISCV] Custom lowering of FLT_ROUNDS_ Differential Revision: https://reviews.llvm.org/D90854
2020-11-05 14:59:52 +08:00
SDValue lowerGET_ROUNDING(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Custom lowering of SET_ROUNDING Differential Revision: https://reviews.llvm.org/D91242
2020-11-11 00:51:34 +08:00
SDValue lowerSET_ROUNDING(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Lower the tail pseudoinstruction This patch lowers the tail pseudoinstruction. This has been modeled after ARM's tail call opt. llvm-svn: 333137
2018-05-24 06:44:08 +08:00
[RISCV] Transform unaligned RVV vector loads/stores to aligned ones This patch adds support for loading and storing unaligned vectors via an equivalently-sized i8 vector type, which has support in the RVV specification for byte-aligned access. This offers a more optimal path for handling of unaligned fixed-length vector accesses, which are currently scalarized. It also prevents crashing when `LegalizeDAG` sees an unaligned scalable-vector load/store operation. Future work could be to investigate loading/storing via the largest vector element type for the given alignment, in case that would be more optimal on hardware. For instance, a 4-byte-aligned nxv2i64 vector load could loaded as nxv4i32 instead of as nxv16i8. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D104032
2021-06-09 22:17:21 +08:00
SDValue expandUnalignedRVVLoad(SDValue Op, SelectionDAG &DAG) const;
SDValue expandUnalignedRVVStore(SDValue Op, SelectionDAG &DAG) const;
[RISCV][NFC] IsEligibleForTailCallOptimization -> isEligibleForTailCallOptimization Also clang-format the modified hunks. llvm-svn: 354584
2019-02-21 22:31:41 +08:00
bool isEligibleForTailCallOptimization(
CCState &CCInfo, CallLoweringInfo &CLI, MachineFunction &MF,
const SmallVector<CCValAssign, 16> &ArgLocs) const;
[RISCV] Codegen for i8, i16, and i32 atomicrmw with RV32A Introduce a new RISCVExpandPseudoInsts pass to expand atomic pseudo-instructions after register allocation. This is necessary in order to ensure that register spills aren't introduced between LL and SC, thus breaking the forward progress guarantee for the operation. AArch64 does something similar for CmpXchg (though only at O0), and Mips is moving towards this approach (see D31287). See also [this mailing list post](http://lists.llvm.org/pipermail/llvm-dev/2016-May/099490.html) from James Knight, which summarises the issues with lowering to ll/sc in IR or pre-RA. See the [accompanying RFC thread](http://lists.llvm.org/pipermail/llvm-dev/2018-June/123993.html) for an overview of the lowering strategy. Differential Revision: https://reviews.llvm.org/D47882 llvm-svn: 342534
2018-09-19 18:54:22 +08:00
[RISCV] Add support for -ffixed-xX flags This adds support for reserving GPRs such that the compiler will not choose a register for register allocation. The implementation follows the same design as for AArch64; each reserved register becomes a target feature and used for getting the reserved registers for a given MachineFunction. The backend checks that it does not need to write to any reserved register; if it does a relevant error is generated. Differential Revision: https://reviews.llvm.org/D67185
2019-10-23 04:25:01 +08:00
/// Generate error diagnostics if any register used by CC has been marked
/// reserved.
void validateCCReservedRegs(
const SmallVectorImpl<std::pair<llvm::Register, llvm::SDValue>> &Regs,
MachineFunction &MF) const;
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
bool useRVVForFixedLengthVectorVT(MVT VT) const;
[RISCV] Prevent store combining from infinitely looping RVV code generation does not successfully custom-lower BUILD_VECTOR in all cases. When it resorts to default expansion it may, on occasion, be expanded to scalar stores through the stack. Unfortunately these stores may then be picked up by the post-legalization DAGCombiner which merges them again. The merged store uses a BUILD_VECTOR which is then expanded, and so on. This patch addresses the issue by overriding the `mergeStoresAfterLegalization` hook. A lack of granularity in this method (being passed the scalar type) means we opt out in almost all cases when RVV fixed-length vector support is enabled. The only exception to this rule are mask vectors, which are always either custom-lowered or are expanded to a load from a constant pool. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D102913
2021-05-21 20:00:19 +08:00
[VP][SelectionDAG] Add a target-configurable EVL operand type This patch adds a way for the target to configure the type it uses for the explicit vector length operands of VP SDNodes. The type must be a legal integer type (there is still no target-independent legalization of this operand) and must currently be at least as big as i32, the type used by the IR intrinsics. An implicit zero-extension takes place on targets which choose a larger type. All VP nodes should be created with this type used for the EVL operand. This allows 64-bit RISC-V to avoid custom legalization of all VP nodes, keeping them in their target-independent form for that bit longer. Reviewed By: simoll Differential Revision: https://reviews.llvm.org/D103027
2021-05-24 22:24:54 +08:00
MVT getVPExplicitVectorLengthTy() const override;
[RISCV] Prevent store combining from infinitely looping RVV code generation does not successfully custom-lower BUILD_VECTOR in all cases. When it resorts to default expansion it may, on occasion, be expanded to scalar stores through the stack. Unfortunately these stores may then be picked up by the post-legalization DAGCombiner which merges them again. The merged store uses a BUILD_VECTOR which is then expanded, and so on. This patch addresses the issue by overriding the `mergeStoresAfterLegalization` hook. A lack of granularity in this method (being passed the scalar type) means we opt out in almost all cases when RVV fixed-length vector support is enabled. The only exception to this rule are mask vectors, which are always either custom-lowered or are expanded to a load from a constant pool. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D102913
2021-05-21 20:00:19 +08:00
/// RVV code generation for fixed length vectors does not lower all
/// BUILD_VECTORs. This makes BUILD_VECTOR legalisation a source of stores to
/// merge. However, merging them creates a BUILD_VECTOR that is just as
/// illegal as the original, thus leading to an infinite legalisation loop.
/// NOTE: Once BUILD_VECTOR can be custom lowered for all legal vector types,
/// this override can be removed.
bool mergeStoresAfterLegalization(EVT VT) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
};
[RISCV] Define vadd/vsub/vrsub intrinsics and lower to V instructions. This patch is based on the proposal from Roger Ferrer Ibanez. http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html Differential Revision: https://reviews.llvm.org/D93013
2020-12-11 15:16:08 +08:00
[RISCV] Initial support of LoopVectorizer for RISC-V Vector. Define an option -riscv-vector-bits-max to specify the maximum vector bits for vectorizer. Loop vectorizer will use the value to check if it is safe to use the whole vector registers to vectorize the loop. It is not the optimum solution for loop vectorizing for scalable vector. It assumed the whole vector registers will be used to vectorize the code. If it is possible, we should configure vl to do vectorize instead of using whole vector registers. We only consider LMUL = 1 in this patch. This patch just an initial work for loop vectorizer for RISC-V Vector. Differential Revision: https://reviews.llvm.org/D95659
2021-01-27 15:00:46 +08:00
namespace RISCV {
// We use 64 bits as the known part in the scalable vector types.
static constexpr unsigned RVVBitsPerBlock = 64;
[RISCV] Remove superfluous semicolon. NFC
2021-02-11 03:20:07 +08:00
} // namespace RISCV
[RISCV] Initial support of LoopVectorizer for RISC-V Vector. Define an option -riscv-vector-bits-max to specify the maximum vector bits for vectorizer. Loop vectorizer will use the value to check if it is safe to use the whole vector registers to vectorize the loop. It is not the optimum solution for loop vectorizing for scalable vector. It assumed the whole vector registers will be used to vectorize the code. If it is possible, we should configure vl to do vectorize instead of using whole vector registers. We only consider LMUL = 1 in this patch. This patch just an initial work for loop vectorizer for RISC-V Vector. Differential Revision: https://reviews.llvm.org/D95659
2021-01-27 15:00:46 +08:00
[RISCV] Define vadd/vsub/vrsub intrinsics and lower to V instructions. This patch is based on the proposal from Roger Ferrer Ibanez. http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html Differential Revision: https://reviews.llvm.org/D93013
2020-12-11 15:16:08 +08:00
namespace RISCVVIntrinsicsTable {
struct RISCVVIntrinsicInfo {
[RISCV] Reduce field sizes in searchable tables to reduce binary size.
2021-01-28 04:22:36 +08:00
unsigned IntrinsicID;
[RISCV] Starting fixing issues that prevent us from testing vXi64 intrinsics on RV32. Currently we crash in type legalization any time an intrinsic uses a scalar i64 on RV32. This patch adds support for type legalizing this to prevent crashing. I don't promise that it uses the best possible codegen just that it is functional. This first version handles 3 cases. vmv.v.x intrinsic, vmv.s.x intrinsic and intrinsics that take a scalar input, splat it and then do some operation. For vmv.v.x we'll either rely on hardware sign extension for constants or we'll convert it to multiple splats and bit manipulation. For vmv.s.x we use a really unoptimal sequence inspired by what we do for an INSERT_VECTOR_ELT. For the third case we'll either try to use the .vi form for constants or convert to a complicated splat and bitmanip and use the .vv form of the operation. I've renamed the ExtendOperand field to SplatOperand now use it specifically for the third case. The first two cases are handled by custom lowering specifically for those intrinsics. I haven't updated all tests yet, but I tried to cover a subset that includes single-width, widening, and narrowing. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D97895
2021-03-11 01:37:25 +08:00
uint8_t SplatOperand;
[RISCV] Define vadd/vsub/vrsub intrinsics and lower to V instructions. This patch is based on the proposal from Roger Ferrer Ibanez. http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html Differential Revision: https://reviews.llvm.org/D93013
2020-12-11 15:16:08 +08:00
};
using namespace RISCV;
#define GET_RISCVVIntrinsicsTable_DECL
#include "RISCVGenSearchableTables.inc"
} // end namespace RISCVVIntrinsicsTable
[RISCV] Implement vlseg intrinsics. For Zvlsseg, we need continuous vector registers for the values. We need to define new register classes for the different combinations of (number of fields and LMUL). For example, when the number of fields(NF) = 3, LMUL = 2, the values will be assigned to (V0M2, V2M2, V4M2), (V2M2, V4M2, V6M2), (V4M2, V6M2, V8M2), ... We define the vlseg intrinsics with multiple outputs. There is no way to describe the codegen patterns with multiple outputs in the tablegen files. We do the codegen in RISCVISelDAGToDAG and use EXTRACT_SUBREG to extract the values of output. The multiple scalable vector values will be put into a struct. This patch is depended on the support for scalable vector struct. Differential Revision: https://reviews.llvm.org/D94229
2020-12-31 17:14:15 +08:00
[RISCV] Localize RISCVZvlssegTable to RISCVISelDAGToDAG.cpp, the only place it is used.
2021-02-18 03:37:06 +08:00
} // end namespace llvm
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
#endif