maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/lib/Target/RISCV/RISCVISelLowering.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

457 lines
18 KiB
C
Raw Normal View History

[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
//===-- RISCVISelLowering.h - RISCV DAG Lowering Interface ------*- C++ -*-===//
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 16:50:56 +08:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
//
//===----------------------------------------------------------------------===//
//
// This file defines the interfaces that RISCV uses to lower LLVM code into a
// selection DAG.
//
//===----------------------------------------------------------------------===//
#ifndef LLVM_LIB_TARGET_RISCV_RISCVISELLOWERING_H
#define LLVM_LIB_TARGET_RISCV_RISCVISELLOWERING_H
#include "RISCV.h"
#include "llvm/CodeGen/SelectionDAG.h"
Fix a bunch more layering of CodeGen headers that are in Target All these headers already depend on CodeGen headers so moving them into CodeGen fixes the layering (since CodeGen depends on Target, not the other way around). llvm-svn: 318490
2017-11-17 09:07:10 +08:00
#include "llvm/CodeGen/TargetLowering.h"
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
namespace llvm {
class RISCVSubtarget;
namespace RISCVISD {
enum NodeType : unsigned {
FIRST_NUMBER = ISD::BUILTIN_OP_END,
[RISCV] Initial support for function calls Note that this is just enough for simple function call examples to generate working code. Support for varargs etc follows in future patches. Differential Revision: https://reviews.llvm.org/D29936 llvm-svn: 317691
2017-11-08 21:41:21 +08:00
RET_FLAG,
[RISCV] Add support for _interrupt attribute - Save/restore only registers that are used. This includes Callee saved registers and Caller saved registers (arguments and temporaries) for integer and FP registers. - If there is a call in the interrupt handler, save/restore all Caller saved registers (arguments and temporaries) and all FP registers. - Emit special return instructions depending on "interrupt" attribute type. Based on initial patch by Zhaoshi Zheng. Reviewers: asb Reviewed By: asb Subscribers: rkruppe, the_o, MartinMosbeck, brucehoult, rbar, johnrusso, simoncook, sabuasal, niosHD, kito-cheng, shiva0217, zzheng, edward-jones, mgrang, rogfer01, llvm-commits Differential Revision: https://reviews.llvm.org/D48411 llvm-svn: 338047
2018-07-27 01:49:43 +08:00
URET_FLAG,
SRET_FLAG,
MRET_FLAG,
[RISCV] Implement lowering of ISD::SELECT Although ISD::SELECT_CC is a more natural match for RISCVISD::SELECT_CC (and ultimately the integer RISC-V conditional branch instructions), we choose to expand ISD::SELECT_CC and lower ISD::SELECT. The appropriate compare+branch will be created in the case where an ISD::SELECT condition value is created by an ISD::SETCC node, which operates on XLen types. Other datatypes such as floating point don't have conditional branch instructions, and lowering ISD::SELECT allows more flexibility for handling these cases. Differential Revision: https://reviews.llvm.org/D29937 llvm-svn: 318735
2017-11-21 15:51:32 +08:00
CALL,
[RISCV] Optimize select_cc after fp compare expansion Some FP compares expand to a sequence ending with (xor X, 1) to invert the result. If the consumer is a select_cc we can likely get rid of this xor by fixing up the select_cc condition. This patch combines (select_cc (xor X, 1), 0, setne, trueV, falseV) - (select_cc X, 0, seteq, trueV, falseV) if we can prove X is 0/1. Reviewed By: lenary Differential Revision: https://reviews.llvm.org/D94546
2021-01-15 05:41:35 +08:00
/// Select with condition operator - This selects between a true value and
/// a false value (ops #3 and #4) based on the boolean result of comparing
/// the lhs and rhs (ops #0 and #1) of a conditional expression with the
/// condition code in op #2, a XLenVT constant from the ISD::CondCode enum.
/// The lhs and rhs are XLenVT integers. The true and false values can be
/// integer or floating point.
[RISCV] Codegen support for RV32D floating point load/store, fadd.d, calling conv fadd.d is required in order to force floating point registers to be used in test code, as parameters are passed in integer registers in the soft float ABI. Much of this patch is concerned with support for passing f64 on RV32D with a soft-float ABI. Similar to Mips, introduce pseudoinstructions to build an f64 out of a pair of i32 and to split an f64 to a pair of i32. BUILD_PAIR and EXTRACT_ELEMENT can't be used, as a BITCAST to i64 would be necessary, but i64 is not a legal type. llvm-svn: 329871
2018-04-12 13:34:25 +08:00
SELECT_CC,
BuildPairF64,
[RISCV] Lower the tail pseudoinstruction This patch lowers the tail pseudoinstruction. This has been modeled after ARM's tail call opt. llvm-svn: 333137
2018-05-24 06:44:08 +08:00
SplitF64,
[RISCV] Custom-legalise 32-bit variable shifts on RV64 The previous DAG combiner-based approach had an issue with infinite loops between the target-dependent and target-independent combiner logic (see PR40333). Although this was worked around in rL351806, the combiner-based approach is still potentially brittle and can fail to select the 32-bit shift variant when profitable to do so, as demonstrated in the pr40333.ll test case. This patch instead introduces target-specific SelectionDAG nodes for SHLW/SRLW/SRAW and custom-lowers variable i32 shifts to them. pr40333.ll is a good example of how this approach can improve codegen. This adds DAG combine that does SimplifyDemandedBits on the operands (only lower 32-bits of first operand and lower 5 bits of second operand are read). This seems better than implementing SimplifyDemandedBitsForTargetNode as there is no guarantee that would be called (and it's not for e.g. the anyext return test cases). Also implements ComputeNumSignBitsForTargetNode. There are codegen changes in atomic-rmw.ll and atomic-cmpxchg.ll but the new instruction sequences are semantically equivalent. Differential Revision: https://reviews.llvm.org/D57085 llvm-svn: 352169
2019-01-25 13:04:00 +08:00
TAIL,
// RV64I shifts, directly matching the semantics of the named RISC-V
// instructions.
SLLW,
SRAW,
[RISCV] Custom-legalise i32 SDIV/UDIV/UREM on RV64M Follow the same custom legalisation strategy as used in D57085 for variable-length shifts (see that patch summary for more discussion). Although we may lose out on some late-stage DAG combines, I think this custom legalisation strategy is ultimately easier to reason about. There are some codegen changes in rv64m-exhaustive-w-insts.ll but they are all neutral in terms of the number of instructions. Differential Revision: https://reviews.llvm.org/D57096 llvm-svn: 352171
2019-01-25 13:11:34 +08:00
SRLW,
// 32-bit operations from RV64M that can't be simply matched with a pattern
[RISCV] Custom type legalize i8/i16 UDIV/UREM/SDIV on RV64 so we can use divuw/remuw/divw. This makes our i8/i16 codegen more similar to the i32 codegen. I've also added computeKnownBits support for DIVUW/REMUW so that we can remove zero extending ANDs from the output. Without this we end up turning DIVUW/REMUW back into DIVU/REMU via some isel patterns. Reviewed By: frasercrmck, luismarques Differential Revision: https://reviews.llvm.org/D95322
2021-01-26 01:35:14 +08:00
// at instruction selection time. These have undefined behavior for division
// by 0 or overflow (divw) like their target independent counterparts.
[RISCV] Custom-legalise i32 SDIV/UDIV/UREM on RV64M Follow the same custom legalisation strategy as used in D57085 for variable-length shifts (see that patch summary for more discussion). Although we may lose out on some late-stage DAG combines, I think this custom legalisation strategy is ultimately easier to reason about. There are some codegen changes in rv64m-exhaustive-w-insts.ll but they are all neutral in terms of the number of instructions. Differential Revision: https://reviews.llvm.org/D57096 llvm-svn: 352171
2019-01-25 13:11:34 +08:00
DIVW,
DIVUW,
[RISCV] Add RV64F codegen support This requires a little extra work due tothe fact i32 is not a legal type. When call lowering happens post-legalisation (e.g. when an intrinsic was inserted during legalisation). A bitcast from f32 to i32 can't be introduced. This is similar to the challenges with RV32D. To handle this, we introduce target-specific DAG nodes that perform bitcast+anyext for f32->i64 and trunc+bitcast for i64->f32. Differential Revision: https://reviews.llvm.org/D53235 llvm-svn: 352807
2019-02-01 06:48:38 +08:00
REMUW,
[RISCV] Add RISCVISD::ROLW/RORW use those for custom legalizing i32 rotl/rotr on RV64IZbb. This should result in better utilization of RORIW since we don't need to look for a SIGN_EXTEND_INREG that may not exist. Also remove rotl/rotr isel matching to GREVI and just prefer RORI. This is to keep consistency so we don't have to match ROLW/RORW to GREVIW as well. I imagine RORI/RORIW performance will be the same or better than GREVI. Differential Revision: https://reviews.llvm.org/D91449
2020-11-21 02:11:34 +08:00
// RV64IB rotates, directly matching the semantics of the named RISC-V
// instructions.
ROLW,
RORW,
[RISCV] Custom lower fshl/fshr with Zbt extension. We need to add a mask to the shift amount for these operations to use the FSR/FSL instructions. We were previously doing this in isel patterns, but custom lowering will make the mask visible to optimizations earlier.
2021-02-01 09:37:44 +08:00
// RV64IB/RV32IB funnel shifts, with the semantics of the named RISC-V
// instructions, but the same operand order as fshl/fshr intrinsics.
FSR,
FSL,
[RISCV] Custom type legalize i32 fshl/fshr on RV64 with Zbt. This adds custom opcodes for FSLW/FSRW so we can type legalize fshl/fshr without needing to match a sign_extend_inreg. I've used the operand order from fshl/fshr to make the isel pattern similar to the non-W form. It was also hard to decide another order since the register instruction has the shift amount as the second operand, but the immediate instruction has it as the third operand. Differential Revision: https://reviews.llvm.org/D91479
2020-11-26 01:43:16 +08:00
// RV64IB funnel shifts, with the semantics of the named RISC-V instructions,
// but the same operand order as fshl/fshr intrinsics.
FSRW,
FSLW,
[RISCV] Support Zfh half-precision floating-point extension. Support "Zfh" extension according to https://github.com/riscv/riscv-isa-manual/blob/zfh/src/zfh.tex Differential Revision: https://reviews.llvm.org/D90738
2020-07-03 22:57:59 +08:00
// FPR<->GPR transfer operations when the FPR is smaller than XLEN, needed as
// XLEN is the only legal integer width.
//
[RISCV] Merge FMV_H_X_RV32/FMV_H_X_RV64 into a single opcode. Same with FMV_X_ANYEXTH_RV32/RV64 Rather than having a different opcode for RV32 and RV64. Let's just say the integer type is XLenVT and use a single opcode for both modes. Differential Revision: https://reviews.llvm.org/D92538
2020-12-04 02:29:10 +08:00
// FMV_H_X matches the semantics of the FMV.H.X.
// FMV_X_ANYEXTH is similar to FMV.X.H but has an any-extended result.
[RISCV] Support Zfh half-precision floating-point extension. Support "Zfh" extension according to https://github.com/riscv/riscv-isa-manual/blob/zfh/src/zfh.tex Differential Revision: https://reviews.llvm.org/D90738
2020-07-03 22:57:59 +08:00
// FMV_W_X_RV64 matches the semantics of the FMV.W.X.
[RISCV] Add RV64F codegen support This requires a little extra work due tothe fact i32 is not a legal type. When call lowering happens post-legalisation (e.g. when an intrinsic was inserted during legalisation). A bitcast from f32 to i32 can't be introduced. This is similar to the challenges with RV32D. To handle this, we introduce target-specific DAG nodes that perform bitcast+anyext for f32->i64 and trunc+bitcast for i64->f32. Differential Revision: https://reviews.llvm.org/D53235 llvm-svn: 352807
2019-02-01 06:48:38 +08:00
// FMV_X_ANYEXTW_RV64 is similar to FMV.X.W but has an any-extended result.
[RISCV] Support Zfh half-precision floating-point extension. Support "Zfh" extension according to https://github.com/riscv/riscv-isa-manual/blob/zfh/src/zfh.tex Differential Revision: https://reviews.llvm.org/D90738
2020-07-03 22:57:59 +08:00
//
[RISCV] Add RV64F codegen support This requires a little extra work due tothe fact i32 is not a legal type. When call lowering happens post-legalisation (e.g. when an intrinsic was inserted during legalisation). A bitcast from f32 to i32 can't be introduced. This is similar to the challenges with RV32D. To handle this, we introduce target-specific DAG nodes that perform bitcast+anyext for f32->i64 and trunc+bitcast for i64->f32. Differential Revision: https://reviews.llvm.org/D53235 llvm-svn: 352807
2019-02-01 06:48:38 +08:00
// This is a more convenient semantic for producing dagcombines that remove
// unnecessary GPR->FPR->GPR moves.
[RISCV] Merge FMV_H_X_RV32/FMV_H_X_RV64 into a single opcode. Same with FMV_X_ANYEXTH_RV32/RV64 Rather than having a different opcode for RV32 and RV64. Let's just say the integer type is XLenVT and use a single opcode for both modes. Differential Revision: https://reviews.llvm.org/D92538
2020-12-04 02:29:10 +08:00
FMV_H_X,
FMV_X_ANYEXTH,
[RISCV] Add RV64F codegen support This requires a little extra work due tothe fact i32 is not a legal type. When call lowering happens post-legalisation (e.g. when an intrinsic was inserted during legalisation). A bitcast from f32 to i32 can't be introduced. This is similar to the challenges with RV32D. To handle this, we introduce target-specific DAG nodes that perform bitcast+anyext for f32->i64 and trunc+bitcast for i64->f32. Differential Revision: https://reviews.llvm.org/D53235 llvm-svn: 352807
2019-02-01 06:48:38 +08:00
FMV_W_X_RV64,
[RISCV] Support @llvm.readcyclecounter() Intrinsic On RISC-V, the `cycle` CSR holds a 64-bit count of the number of clock cycles executed by the core, from an arbitrary point in the past. This matches the intended semantics of `@llvm.readcyclecounter()`, which we currently leave to the default lowering (to the constant 0). With this patch, we will now correctly lower this intrinsic to the intended semantics, using the user-space instruction `rdcycle`. On 64-bit targets, we can directly lower to this instruction. On 32-bit targets, we need to do more, as `rdcycle` only returns the low 32-bits of the `cycle` CSR. In this case, we perform a custom lowering, based on the PowerPC lowering, using `rdcycleh` to obtain the high 32-bits of the `cycle` CSR. This custom lowering inserts a new basic block which detects overflow in the high 32-bits of the `cycle` CSR during reading (because multiple instructions are required to read). The emitted assembly matches the suggested assembly in the RISC-V specification. Differential Revision: https://reviews.llvm.org/D64125 llvm-svn: 365201
2019-07-05 20:35:21 +08:00
FMV_X_ANYEXTW_RV64,
// READ_CYCLE_WIDE - A read of the 64-bit cycle CSR on a 32-bit target
// (returns (Lo, Hi)). It takes a chain operand.
[RISCV] Lower GREVI and GORCI as custom nodes This moves the recognition of GREVI and GORCI from TableGen patterns into a DAGCombine. This is done primarily to match "deeper" patterns in the future, like (grevi (grevi x, 1) 2) -> (grevi x, 3). TableGen is not best suited to matching patterns such as these as the compile time of the DAG matchers quickly gets out of hand due to the expansion of commutative permutations. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D91259
2020-11-11 21:41:55 +08:00
READ_CYCLE_WIDE,
// Generalized Reverse and Generalized Or-Combine - directly matching the
// semantics of the named RISC-V instructions. Lowered as custom nodes as
// TableGen chokes when faced with commutative permutations in deeply-nested
// DAGs. Each node takes an input operand and a TargetConstant immediate
// shift amount, and outputs a bit-manipulated version of input. All operands
// are of type XLenVT.
GREVI,
GREVIW,
GORCI,
GORCIW,
[RISCV] Add intrinsics for vmv.x.s and vmv.s.x This adds intrinsics for vmv.x.s and vmv.s.x. I've used stricter type constraints on these intrinsics than what we've been doing on the arithmetic intrinsics so far. This will allow us to not need to pass the scalar type to the Intrinsic::getDeclaration call when creating these intrinsics. A custom ISD is used for vmv.x.s in order to implement the change in computeNumSignBitsForTargetNode which can remove sign extends on the result. I also modified the MC layer description of these instructions to show the tied source/dest operand. This is different than what we do for masked instructions where we drop the tied source operand when converting to MC. But it is a more accurate description of the instruction. We can't do this for masked instructions since we use the same MC instruction for masked and unmasked. Tools like llvm-mca operate in the MC layer and rely on ins/outs and Uses/Defs for analysis so I don't know if we'll be able to maintain the current behavior for masked instructions. So I went with the accurate description here since it was easy. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D93365
2020-12-19 01:50:23 +08:00
// Vector Extension
[RISCV] Add support for splat fixed length build_vectors using RVV. Building on the fixed vector support from D95705 I've added ISD nodes for vmv.v.x and vfmv.v.f and switched to lowering the intrinsics to it. This allows us to share the same isel patterns for both. This doesn't handle splats of i64 on RV32 yet. The build_vector gets converted to a vXi32 build_vector+bitcast during type legalization. Not sure the best way to handle this at the moment. Differential Revision: https://reviews.llvm.org/D96108
2021-02-09 02:59:38 +08:00
// VMV_V_X_VL matches the semantics of vmv.v.x but includes an extra operand
// for the VL value to be used for the operation.
VMV_V_X_VL,
// VFMV_V_F_VL matches the semantics of vfmv.v.f but includes an extra operand
// for the VL value to be used for the operation.
VFMV_V_F_VL,
[RISCV] Optimize sign-extended EXTRACT_VECTOR_ELT nodes This patch custom-legalizes all integer EXTRACT_VECTOR_ELT nodes where SEW < XLEN to VMV_S_X nodes to help the compiler infer sign bits from the result. This allows us to eliminate redundant sign extensions. For parity, all integer EXTRACT_VECTOR_ELT nodes are legalized this way so that we don't need TableGen patterns for some and not others. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95741
2021-01-31 00:53:32 +08:00
// VMV_X_S matches the semantics of vmv.x.s. The result is always XLenVT sign
// extended from the vector element size.
[RISCV] Add intrinsics for vmv.x.s and vmv.s.x This adds intrinsics for vmv.x.s and vmv.s.x. I've used stricter type constraints on these intrinsics than what we've been doing on the arithmetic intrinsics so far. This will allow us to not need to pass the scalar type to the Intrinsic::getDeclaration call when creating these intrinsics. A custom ISD is used for vmv.x.s in order to implement the change in computeNumSignBitsForTargetNode which can remove sign extends on the result. I also modified the MC layer description of these instructions to show the tied source/dest operand. This is different than what we do for masked instructions where we drop the tied source operand when converting to MC. But it is a more accurate description of the instruction. We can't do this for masked instructions since we use the same MC instruction for masked and unmasked. Tools like llvm-mca operate in the MC layer and rely on ins/outs and Uses/Defs for analysis so I don't know if we'll be able to maintain the current behavior for masked instructions. So I went with the accurate description here since it was easy. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D93365
2020-12-19 01:50:23 +08:00
VMV_X_S,
[RISCV] Add ISel support for RVV vector/scalar forms This patch extends the SDNode ISel support for RVV from only the vector/vector instructions to include the vector/scalar and vector/immediate forms. It uses splat_vector to carry the scalar in each case, except when XLEN<SEW (RV32 SEW=64) when a custom node `SPLAT_VECTOR_I64` is used for type-legalization and to encode the fact that the value is sign-extended to SEW. When the scalar is a full 64-bit value we use a sequence to materialize the constant into the vector register. The non-intrinsic ISel patterns have also been split into their own file. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D93312
2020-12-15 21:05:32 +08:00
// Splats an i64 scalar to a vector type (with element type i64) where the
// scalar is a sign-extended i32.
SPLAT_VECTOR_I64,
[RISCV] Custom lower ISD::VSCALE. This patch custom lowers ISD::VSCALE into a csrr vlenb followed by a shift right by 3 followed by a multiply by the scale amount. I've added computeKnownBits support to indicate that the csrr vlenb always produces 3 trailng bits of 0s so the shift right is "exact". This allows the shift and multiply sequence to be nicely optimized into a single shift or removed completely when the scale amount is a power of 2. The non power of 2 case multiplying by 24 is still producing suboptimal code. We could remove the right shift and use a multiply by 3. Hopefully we can improve DAG combine to fix that since it's not unique to this sequence. This replaces D94144. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D94249
2021-01-14 09:14:45 +08:00
// Read VLENB CSR
READ_VLENB,
[RISCV] Add scalable vector truncate patterns Original patch by @rogfer01. This patch supports vector truncates, which on RVV must be done in a series of instructions truncating by one power-of-two at a time. This is done through custom-lowering and a custom node to avoid LLVM re-combining the split TRUNCATE nodes. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D94796
2021-01-16 01:04:52 +08:00
// Truncates a RVV integer vector by one power-of-two.
TRUNCATE_VECTOR,
[RISCV] Add a VL output to vleff intrinsics. The fault-only-first-load instructions can reduce VL if an element other than element 0 triggers a memory fault. This can be used to vectorize loops with data dependent exit conditions like strcmp or strlen. This patch adds a VL output to these intrinsics so that the new VL value can be captured by software. This will be expanded to 'csrr gpr, vl' after the vleff instruction during SelectionDAG. By doing this with one intrinsic we are able to guarantee that the csrr reads the VL value produced by the vleff instruction. Having it as a separate intrinsic would make it impossible to guarantee ordering without making every other vector intrinsic have side effects. The intrinsics are expanded during lowering into two ISD nodes that are glued together. These ISD nodes will go through isel separately, but should maintain the glue so that they get emitted adjacently by InstrEmitter. I've only ran the chain through the vleff instruction, allowing the READ_VL to be deleted if it is unused. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D94286
2021-01-22 09:08:41 +08:00
// Unit-stride fault-only-first load
VLEFF,
VLEFF_MASK,
[RISCV] Add RVV insertelt/extractelt scalable-vector patterns Original patch by @rogfer01. This patch adds support for insertelt and extractelt operations on scalable vectors. Special care must be taken on RV32 when dealing with i64 vectors as there are no straightforward ways to insert a 64-bit element without a register of that size. To that end, both are custom-lowered to different sequences. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D94615
2021-01-12 01:42:05 +08:00
// Matches the semantics of vslideup/vslidedown. The first operand is the
// pass-thru operand, the second is the source vector, and the third is the
// XLenVT index (either constant or non-constant).
VSLIDEUP,
VSLIDEDOWN,
[RISCV] Add support for selecting vid.v from build_vector This patch optimizes a build_vector "index sequence" and lowers it to the existing custom RISCVISD::VID node. This pattern is common in autovectorized code. The custom node was updated to allow it to be used by both scalable and fixed-length vectors, thus avoiding pattern duplication. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96332
2021-02-09 20:09:10 +08:00
// Matches the semantics of the vid.v instruction, with a mask and VL
// operand.
VID_VL,
[RISCV] Add support for RVV int<->fp & fp<->fp conversions This patch adds support for the full range of vector int-to-float, float-to-int, and float-to-float conversions on legal types. Many conversions are supported natively in RVV so are lowered with patterns. These include conversions between (element) types of the same size, and those that are half/double the size of the input. When conversions take place between types that are less than half or more than double the size we must lower them using sequences of instructions which go via intermediate types. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95447
2021-01-22 22:54:00 +08:00
// Matches the semantics of the vfcnvt.rod function (Convert double-width
// float to single-width float, rounding towards odd). Takes a double-width
// float vector and produces a single-width float vector.
VFNCVT_ROD,
[RISCV] Support scalable-vector integer reduction intrinsics This patch adds support for the integer reduction intrinsics supported by RVV. This excludes "mul" which has no corresponding instruction. The reduction instructions in RVV have slightly complicated type constraints given they always produce a single "M1" vector register. They are lowered to custom nodes including the second "scalar" reduction operand to simplify the patterns and in the hope that they can be useful for future DAG combines. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95620
2021-01-28 23:31:27 +08:00
// These nodes match the semantics of the corresponding RVV vector reduction
// instructions. They produce a vector result which is the reduction
// performed over the first vector operand plus the first element of the
// second vector operand. The first operand is an unconstrained vector type,
// and the result and second operand's types are expected to be the
// corresponding full-width LMUL=1 type for the first operand:
// nxv8i8 = vecreduce_add nxv32i8, nxv8i8
// nxv2i32 = vecreduce_add nxv8i32, nxv2i32
// The different in types does introduce extra vsetvli instructions but
// similarly it reduces the number of registers consumed per reduction.
VECREDUCE_ADD,
VECREDUCE_UMAX,
VECREDUCE_SMAX,
VECREDUCE_UMIN,
VECREDUCE_SMIN,
VECREDUCE_AND,
VECREDUCE_OR,
VECREDUCE_XOR,
[RISCV] Support the scalable-vector fadd reduction intrinsic This patch adds support for both the fadd reduction intrinsic, in both the ordered and unordered modes. The fmin and fmax intrinsics are not currently supported due to a discrepancy between the LLVM semantics and the RVV ISA behaviour with regards to signaling NaNs. This behaviour is likely fixed in version 2.3 of the RISC-V F/D/Q extension, but until then the intrinsics can be left unsupported. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95870
2021-02-02 22:34:51 +08:00
VECREDUCE_FADD,
VECREDUCE_SEQ_FADD,
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
[RISCV] Add support for integer fixed vector setcc I believe I've covered all orderings of splat operands here. Better canonicalization in lowering might help reduce this. I did not handle the immediate adjustments needed for set(u)gt/set(u)lt. Testing here is limited to byte types because the scalable vector type used for masks for the store is calculated assuming 8 byte elements. But for the setcc its based on the element count of the container type for the setcc input. So they don't agree. We'll need to enhanced D96352 to handle this I think. Differential Revision: https://reviews.llvm.org/D96443
2021-02-13 01:20:54 +08:00
// Vector binary and unary ops with a mask as a third operand, and VL as a
// fourth operand.
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
// FIXME: Can we replace these with ISD::VP_*?
ADD_VL,
AND_VL,
MUL_VL,
OR_VL,
SDIV_VL,
SHL_VL,
SREM_VL,
SRA_VL,
SRL_VL,
SUB_VL,
UDIV_VL,
UREM_VL,
XOR_VL,
FADD_VL,
FSUB_VL,
FMUL_VL,
FDIV_VL,
FNEG_VL,
[RISCV] Add support for fixed vector sqrt.
2021-02-13 07:10:18 +08:00
FSQRT_VL,
[RISCV] Add support for fixed vector FMA. Follow up to D95705. Does not include the commuting support from D95800. Differential Revision: https://reviews.llvm.org/D96103
2021-02-09 02:45:45 +08:00
FMA_VL,
[RISCV] Add support for integer fixed min/max This patch extends the initial fixed-length vector support to include smin, smax, umin, and umax. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D96491
2021-02-11 20:45:38 +08:00
SMIN_VL,
SMAX_VL,
UMIN_VL,
UMAX_VL,
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
[RISCV] Add support for integer fixed vector setcc I believe I've covered all orderings of splat operands here. Better canonicalization in lowering might help reduce this. I did not handle the immediate adjustments needed for set(u)gt/set(u)lt. Testing here is limited to byte types because the scalable vector type used for masks for the store is calculated assuming 8 byte elements. But for the setcc its based on the element count of the container type for the setcc input. So they don't agree. We'll need to enhanced D96352 to handle this I think. Differential Revision: https://reviews.llvm.org/D96443
2021-02-13 01:20:54 +08:00
// Vector compare producing a mask. Fourth operand is input mask. Fifth
// operand is VL.
SETCC_VL,
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
// Set mask vector to all zeros or ones.
VMCLR_VL,
VMSET_VL,
[RISCV] Add support for selecting vrgather.vx/vi for fixed vector splat shuffles. The test cases extract a fixed element from a vector and splat it into a vector. This gets DAG combined into a splat shuffle. I've used some very wide vectors in the test to make sure we have at least a couple tests where the element doesn't fit into the uimm5 immediate of vrgather.vi so we fall back to vrgather.vx. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D96186
2021-02-11 01:40:28 +08:00
// Matches the semantics of vrgather.vx with an extra operand for VL.
VRGATHER_VX_VL,
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
// Memory opcodes start here.
VLE_VL = ISD::FIRST_TARGET_MEMORY_OPCODE,
VSE_VL,
// WARNING: Do not add anything in the end unless you want the node to
// have memop! In fact, starting from FIRST_TARGET_MEMORY_OPCODE all
// opcodes will be thought as target memory ops!
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
};
[RISCV] Lower GREVI and GORCI as custom nodes This moves the recognition of GREVI and GORCI from TableGen patterns into a DAGCombine. This is done primarily to match "deeper" patterns in the future, like (grevi (grevi x, 1) 2) -> (grevi x, 3). TableGen is not best suited to matching patterns such as these as the compile time of the DAG matchers quickly gets out of hand due to the expansion of commutative permutations. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D91259
2020-11-11 21:41:55 +08:00
} // namespace RISCVISD
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
class RISCVTargetLowering : public TargetLowering {
const RISCVSubtarget &Subtarget;
public:
explicit RISCVTargetLowering(const TargetMachine &TM,
const RISCVSubtarget &STI);
[RISCV] Define vadd/vsub/vrsub intrinsics and lower to V instructions. This patch is based on the proposal from Roger Ferrer Ibanez. http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html Differential Revision: https://reviews.llvm.org/D93013
2020-12-11 15:16:08 +08:00
const RISCVSubtarget &getSubtarget() const { return Subtarget; }
[RISCV] Codegen for i8, i16, and i32 atomicrmw with RV32A Introduce a new RISCVExpandPseudoInsts pass to expand atomic pseudo-instructions after register allocation. This is necessary in order to ensure that register spills aren't introduced between LL and SC, thus breaking the forward progress guarantee for the operation. AArch64 does something similar for CmpXchg (though only at O0), and Mips is moving towards this approach (see D31287). See also [this mailing list post](http://lists.llvm.org/pipermail/llvm-dev/2016-May/099490.html) from James Knight, which summarises the issues with lowering to ll/sc in IR or pre-RA. See the [accompanying RFC thread](http://lists.llvm.org/pipermail/llvm-dev/2018-June/123993.html) for an overview of the lowering strategy. Differential Revision: https://reviews.llvm.org/D47882 llvm-svn: 342534
2018-09-19 18:54:22 +08:00
bool getTgtMemIntrinsic(IntrinsicInfo &Info, const CallInst &I,
MachineFunction &MF,
unsigned Intrinsic) const override;
[RISCV] Implement isLegalAddressingMode for RISC-V This has no impact on codegen for the current RISC-V unit tests or my small benchmark set and very minor changes in a few programs in the GCC torture suite. Based on this, I haven't been able to produce a representative test program that demonstrates a benefit from isLegalAddressingMode. I'm committing the patch anyway, on the basis that presenting accurate information to the target-independent code is preferable to relying on incorrect generic assumptions. llvm-svn: 330932
2018-04-26 20:13:48 +08:00
bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM, Type *Ty,
unsigned AS,
Instruction *I = nullptr) const override;
[RISCV] Implement isLegalICmpImmediate I'm unable to construct a representative test case that demonstrates the advantage, but it seems sensible to report accurate target-specific information regardless. llvm-svn: 330938
2018-04-26 21:15:17 +08:00
bool isLegalICmpImmediate(int64_t Imm) const override;
[RISCV] Implement isLegalAddImmediate This causes a trivial improvement in the recently added lsr-legaladdimm.ll test case. llvm-svn: 330937
2018-04-26 21:00:37 +08:00
bool isLegalAddImmediate(int64_t Imm) const override;
[RISCV] Implement isTruncateFree Adapted from ARM's implementation introduced in r313533 and r314280. llvm-svn: 330940
2018-04-26 21:37:00 +08:00
bool isTruncateFree(Type *SrcTy, Type *DstTy) const override;
bool isTruncateFree(EVT SrcVT, EVT DstVT) const override;
[RISCV] Implement isZextFree This returns true for 8-bit and 16-bit loads, allowing LBU/LHU to be selected and avoiding unnecessary masks. llvm-svn: 330943
2018-04-26 22:04:18 +08:00
bool isZExtFree(SDValue Val, EVT VT2) const override;
[TargetLowering][RISCV] Introduce isSExtCheaperThanZExt hook and implement for RISC-V DAGTypeLegalizer::PromoteSetCCOperands currently prefers to zero-extend operands when it is able to do so. For some targets this is more expensive than a sign-extension, which is also a valid choice. Introduce the isSExtCheaperThanZExt hook and use it in the new SExtOrZExtPromotedInteger helper. On RISC-V, we prefer sign-extension for FromTy == MVT::i32 and ToTy == MVT::i64, as it can be performed using a single instruction. Differential Revision: https://reviews.llvm.org/D52978 llvm-svn: 347977
2018-11-30 17:56:54 +08:00
bool isSExtCheaperThanZExt(EVT SrcVT, EVT DstVT) const override;
[RISCV] Make ctlz/cttz cheap to speculatively execute so CodeGenPrepare won't insert a zero check. Add additional isel patterns for ctzw/clzw instructions. Differential Revision: https://reviews.llvm.org/D91040
2020-11-10 02:05:51 +08:00
bool isCheapToSpeculateCttz() const override;
bool isCheapToSpeculateCtlz() const override;
[RISCV] Select +0.0 immediate using fmv.{w,d}.x / fcvt.d.w Floating point positive zero can be selected using fmv.w.x / fmv.d.x / fcvt.d.w and the zero source register. Differential Revision: https://reviews.llvm.org/D75729
2020-03-20 17:22:48 +08:00
bool isFPImmLegal(const APFloat &Imm, EVT VT,
bool ForCodeSize) const override;
[RISCV] Support Bit-Preserving FP in F/D Extensions Summary: This allows some integer bitwise operations to instead be performed by hardware fp instructions. This is correct because the RISC-V spec requires the F and D extensions to use the IEEE-754 standard representation, and fp register loads and stores to be bit-preserving. This is tested against the soft-float ABI, but with hardware float extensions enabled, so that the tests also ensure the optimisation also fires in this case. Reviewers: asb, luismarques Reviewed By: asb Subscribers: hiraditya, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, PkmX, jocewei, psnobl, benna, Jim, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62900 llvm-svn: 362790
2019-06-07 20:20:14 +08:00
bool hasBitPreservingFPLogic(EVT VT) const override;
[RISCV] Implement isLegalAddressingMode for RISC-V This has no impact on codegen for the current RISC-V unit tests or my small benchmark set and very minor changes in a few programs in the GCC torture suite. Based on this, I haven't been able to produce a representative test program that demonstrates a benefit from isLegalAddressingMode. I'm committing the patch anyway, on the basis that presenting accurate information to the target-independent code is preferable to relying on incorrect generic assumptions. llvm-svn: 330932
2018-04-26 20:13:48 +08:00
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
// Provide custom lowering hooks for some operations.
SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override;
[RISCV] Custom-legalise 32-bit variable shifts on RV64 The previous DAG combiner-based approach had an issue with infinite loops between the target-dependent and target-independent combiner logic (see PR40333). Although this was worked around in rL351806, the combiner-based approach is still potentially brittle and can fail to select the 32-bit shift variant when profitable to do so, as demonstrated in the pr40333.ll test case. This patch instead introduces target-specific SelectionDAG nodes for SHLW/SRLW/SRAW and custom-lowers variable i32 shifts to them. pr40333.ll is a good example of how this approach can improve codegen. This adds DAG combine that does SimplifyDemandedBits on the operands (only lower 32-bits of first operand and lower 5 bits of second operand are read). This seems better than implementing SimplifyDemandedBitsForTargetNode as there is no guarantee that would be called (and it's not for e.g. the anyext return test cases). Also implements ComputeNumSignBitsForTargetNode. There are codegen changes in atomic-rmw.ll and atomic-cmpxchg.ll but the new instruction sequences are semantically equivalent. Differential Revision: https://reviews.llvm.org/D57085 llvm-svn: 352169
2019-01-25 13:04:00 +08:00
void ReplaceNodeResults(SDNode *N, SmallVectorImpl<SDValue> &Results,
SelectionDAG &DAG) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
[RISCV] Handle redundant SplitF64+BuildPairF64 pairs in a DAGCombine r343712 performed this optimisation during instruction selection. As Eli Friedman pointed out in post-commit review, implementing this as a DAGCombine might allow opportunities for further optimisations. llvm-svn: 343741
2018-10-04 07:30:16 +08:00
SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
[RISCV] Add implementation of targetShrinkDemandedConstant to optimize AND immediates. SimplifyDemandedBits can remove set bits from immediates from instructions like AND/OR/XOR. This can prevent them from being efficiently codegened on RISCV. This adds an initial version that tries to keep or form 12 bit sign extended immediates for AND operations to enable use of ANDI. If that doesn't work we'll try to create a 32 bit sign extended immediate to use LUI+ADDIW. More optimizations are possible for different size immediates or different operations. But this is a good starting point that already has test coverage. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D94628
2021-01-16 02:54:26 +08:00
bool targetShrinkDemandedConstant(SDValue Op, const APInt &DemandedBits,
const APInt &DemandedElts,
TargetLoweringOpt &TLO) const override;
[RISCV] Custom lower ISD::VSCALE. This patch custom lowers ISD::VSCALE into a csrr vlenb followed by a shift right by 3 followed by a multiply by the scale amount. I've added computeKnownBits support to indicate that the csrr vlenb always produces 3 trailng bits of 0s so the shift right is "exact". This allows the shift and multiply sequence to be nicely optimized into a single shift or removed completely when the scale amount is a power of 2. The non power of 2 case multiplying by 24 is still producing suboptimal code. We could remove the right shift and use a multiply by 3. Hopefully we can improve DAG combine to fix that since it's not unique to this sequence. This replaces D94144. Reviewed By: HsiangKai Differential Revision: https://reviews.llvm.org/D94249
2021-01-14 09:14:45 +08:00
void computeKnownBitsForTargetNode(const SDValue Op,
KnownBits &Known,
const APInt &DemandedElts,
const SelectionDAG &DAG,
unsigned Depth) const override;
[RISCV] Custom-legalise 32-bit variable shifts on RV64 The previous DAG combiner-based approach had an issue with infinite loops between the target-dependent and target-independent combiner logic (see PR40333). Although this was worked around in rL351806, the combiner-based approach is still potentially brittle and can fail to select the 32-bit shift variant when profitable to do so, as demonstrated in the pr40333.ll test case. This patch instead introduces target-specific SelectionDAG nodes for SHLW/SRLW/SRAW and custom-lowers variable i32 shifts to them. pr40333.ll is a good example of how this approach can improve codegen. This adds DAG combine that does SimplifyDemandedBits on the operands (only lower 32-bits of first operand and lower 5 bits of second operand are read). This seems better than implementing SimplifyDemandedBitsForTargetNode as there is no guarantee that would be called (and it's not for e.g. the anyext return test cases). Also implements ComputeNumSignBitsForTargetNode. There are codegen changes in atomic-rmw.ll and atomic-cmpxchg.ll but the new instruction sequences are semantically equivalent. Differential Revision: https://reviews.llvm.org/D57085 llvm-svn: 352169
2019-01-25 13:04:00 +08:00
unsigned ComputeNumSignBitsForTargetNode(SDValue Op,
const APInt &DemandedElts,
const SelectionDAG &DAG,
unsigned Depth) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
// This method returns the name of a target specific DAG node.
const char *getTargetNodeName(unsigned Opcode) const override;
[RISCV] Add basic support for inline asm constraints llvm-svn: 322217
2018-01-11 04:05:09 +08:00
[RISCV] Support 'f' Inline Assembly Constraint Summary: This adds the 'f' inline assembly constraint, as supported by GCC. An 'f'-constrained operand is passed in a floating point register. Exactly which kind of floating-point register (32-bit or 64-bit) is decided based on the operand type and the available standard extensions (-f and -d, respectively). This patch adds support in both the clang frontend, and LLVM itself. Reviewers: asb, lewis-revill Reviewed By: asb Subscribers: hiraditya, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, MaskRay, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, PkmX, jocewei, psnobl, benna, Jim, s.egerton, cfe-commits, llvm-commits Tags: #clang, #llvm Differential Revision: https://reviews.llvm.org/D65500 llvm-svn: 367403
2019-07-31 17:45:55 +08:00
ConstraintType getConstraintType(StringRef Constraint) const override;
[RISCV] Lower inline asm constraint A for RISC-V This allows arguments with the constraint A to be lowered to input nodes for RISC-V, which implies a memory address stored in a register. This patch adds the minimal amount of code required to get operands with the right constraints to compile. https://reviews.llvm.org/D54296 llvm-svn: 369095
2019-08-16 18:28:34 +08:00
unsigned getInlineAsmMemConstraint(StringRef ConstraintCode) const override;
[RISCV] Add basic support for inline asm constraints llvm-svn: 322217
2018-01-11 04:05:09 +08:00
std::pair<unsigned, const TargetRegisterClass *>
getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
StringRef Constraint, MVT VT) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
[RISCV] Lower inline asm constraints I, J & K for RISC-V This validates and lowers arguments to inline asm nodes which have the constraints I, J & K, with the following semantics (equivalent to GCC): I: Any 12-bit signed immediate. J: Immediate integer zero only. K: Any 5-bit unsigned immediate. Differential Revision: https://reviews.llvm.org/D54093 llvm-svn: 363054
2019-06-11 20:42:13 +08:00
void LowerAsmOperandForConstraint(SDValue Op, std::string &Constraint,
std::vector<SDValue> &Ops,
SelectionDAG &DAG) const override;
[RISCV] Implement lowering of ISD::SELECT Although ISD::SELECT_CC is a more natural match for RISCVISD::SELECT_CC (and ultimately the integer RISC-V conditional branch instructions), we choose to expand ISD::SELECT_CC and lower ISD::SELECT. The appropriate compare+branch will be created in the case where an ISD::SELECT condition value is created by an ISD::SETCC node, which operates on XLen types. Other datatypes such as floating point don't have conditional branch instructions, and lowering ISD::SELECT allows more flexibility for handling these cases. Differential Revision: https://reviews.llvm.org/D29937 llvm-svn: 318735
2017-11-21 15:51:32 +08:00
MachineBasicBlock *
EmitInstrWithCustomInserter(MachineInstr &MI,
MachineBasicBlock *BB) const override;
[RISCV] Define getSetCCResultType for setting vector setCC type To avoid trigger "No default SetCC type for vectors!" Assertion Differential Revision: https://reviews.llvm.org/D42675 llvm-svn: 324054
2018-02-02 10:43:18 +08:00
EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context,
EVT VT) const override;
[RISCV] Implement lowering of ISD::SELECT Although ISD::SELECT_CC is a more natural match for RISCVISD::SELECT_CC (and ultimately the integer RISC-V conditional branch instructions), we choose to expand ISD::SELECT_CC and lower ISD::SELECT. The appropriate compare+branch will be created in the case where an ISD::SELECT condition value is created by an ISD::SETCC node, which operates on XLen types. Other datatypes such as floating point don't have conditional branch instructions, and lowering ISD::SELECT allows more flexibility for handling these cases. Differential Revision: https://reviews.llvm.org/D29937 llvm-svn: 318735
2017-11-21 15:51:32 +08:00
[RISCV] Allow conversion of CC logic to bitwise logic Indicates in the TargetLowering interface that conversions from CC logic to bitwise logic are allowed. Adds tests that show the benefit when optimization opportunities are detected. Also adds tests that show that when the optimization is not applied correct code is generated (but opportunities for other optimizations remain). Differential Revision: https://reviews.llvm.org/D59596 Patch by Luís Marques. llvm-svn: 356740
2019-03-22 18:39:22 +08:00
bool convertSetCCLogicToBitwiseLogic(EVT VT) const override {
return VT.isScalarInteger();
}
[RISCV] Implement convertSelectOfConstantsToMath Summary: The current lowering of `select` on RISC-V uses a branch instruction to load a register with one or other value. This is inefficient, especially in the case of small constants that can be computed easily. By implementing the TargetLowering::convertSelectOfConstantsToMath hook, some of the simpler cases are covered that let us avoid introducing a branch in these cases. Reviewed By: luismarques Differential Revision: https://reviews.llvm.org/D79260
2020-05-02 22:05:12 +08:00
bool convertSelectOfConstantsToMath(EVT VT) const override { return true; }
[RISCV] Allow conversion of CC logic to bitwise logic Indicates in the TargetLowering interface that conversions from CC logic to bitwise logic are allowed. Adds tests that show the benefit when optimization opportunities are detected. Also adds tests that show that when the optimization is not applied correct code is generated (but opportunities for other optimizations remain). Differential Revision: https://reviews.llvm.org/D59596 Patch by Luís Marques. llvm-svn: 356740
2019-03-22 18:39:22 +08:00
[RISCV] Add codegen support for atomic load/stores with RV32A Fences are inserted according to table A.6 in the current draft of version 2.3 of the RISC-V Instruction Set Manual, which incorporates the memory model changes and definitions contributed by the RISC-V Memory Consistency Model task group. Instruction selection failures will now occur for 8/16/32-bit atomicrmw and cmpxchg operations when targeting RV32IA until lowering for these operations is added in a follow-on patch. Differential Revision: https://reviews.llvm.org/D47589 llvm-svn: 334591
2018-06-13 20:04:51 +08:00
bool shouldInsertFencesForAtomic(const Instruction *I) const override {
return isa<LoadInst>(I) || isa<StoreInst>(I);
}
Instruction *emitLeadingFence(IRBuilder<> &Builder, Instruction *Inst,
AtomicOrdering Ord) const override;
Instruction *emitTrailingFence(IRBuilder<> &Builder, Instruction *Inst,
AtomicOrdering Ord) const override;
[RISCV] Add an implementation of isFMAFasterThanFMulAndFAdd Start with an assumption that FMA is faster than Fmul+FAdd. If thats not true on some particular implementation we can add a tuning parameter in the future. I've update the fmuladd test cases and added new test cases for fast math flag based contraction. Differential Revision: https://reviews.llvm.org/D91987
2020-11-26 07:07:34 +08:00
bool isFMAFasterThanFMulAndFAdd(const MachineFunction &MF,
EVT VT) const override;
[RISCV] Do a sign-extension in a compare-and-swap of 32 bit in RV64A AtomicCmpSwapWithSuccess is legalised into an AtomicCmpSwap plus a comparison. This requires an extension of the value which, by default, is a zero-extension. When we later lower AtomicCmpSwap into a PseudoCmpXchg32 and then expanded in RISCVExpandPseudoInsts.cpp, the lr.w instruction does a sign-extension. This mismatch of extensions causes the comparison to fail when the compared value is negative. This change overrides TargetLowering::getExtendForAtomicOps for RISC-V so it does a sign-extension instead. Differential Revision: https://reviews.llvm.org/D58829 Patch by Ferran Pallarès Roca. llvm-svn: 355869
2019-03-12 05:41:22 +08:00
ISD::NodeType getExtendForAtomicOps() const override {
[LegalizeTypes][RISCV] Correctly sign-extend comparison for ATOMIC_CMP_XCHG Summary: Currently, the comparison argument used for ATOMIC_CMP_XCHG is legalised with GetPromotedInteger, which leaves the upper bits of the value undefind. Since this is used for comparing in an LR/SC loop with a full-width comparison, we must sign extend it. We introduce a new getExtendForAtomicCmpSwapArg to complement getExtendForAtomicOps, since many targets have compare-and-swap instructions (or pseudos) that correctly handle an any-extend input, and the existing function determines the extension of the result, whereas we are concerned with the input. This is related to https://reviews.llvm.org/D58829, which solved the issue for ATOMIC_CMP_SWAP_WITH_SUCCESS, but not the simpler ATOMIC_CMP_SWAP. Reviewers: asb, lenary, efriedma Reviewed By: asb Subscribers: arichardson, hiraditya, rbar, johnrusso, simoncook, sabuasal, niosHD, kito-cheng, shiva0217, MaskRay, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, jfb, PkmX, jocewei, psnobl, benna, Jim, s.egerton, pzheng, sameer.abuasal, apazos, luismarques, evandro, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D74453
2020-04-01 22:50:47 +08:00
return ISD::SIGN_EXTEND;
}
ISD::NodeType getExtendForAtomicCmpSwapArg() const override {
[RISCV] Do a sign-extension in a compare-and-swap of 32 bit in RV64A AtomicCmpSwapWithSuccess is legalised into an AtomicCmpSwap plus a comparison. This requires an extension of the value which, by default, is a zero-extension. When we later lower AtomicCmpSwap into a PseudoCmpXchg32 and then expanded in RISCVExpandPseudoInsts.cpp, the lr.w instruction does a sign-extension. This mismatch of extensions causes the comparison to fail when the compared value is negative. This change overrides TargetLowering::getExtendForAtomicOps for RISC-V so it does a sign-extension instead. Differential Revision: https://reviews.llvm.org/D58829 Patch by Ferran Pallarès Roca. llvm-svn: 355869
2019-03-12 05:41:22 +08:00
return ISD::SIGN_EXTEND;
}
[RISCV] Custom lower SHL_PARTS, SRA_PARTS, SRL_PARTS When not optimizing for minimum size (-Oz) we custom lower wide shifts (SHL_PARTS, SRA_PARTS, SRL_PARTS) instead of expanding to a libcall. Differential Revision: https://reviews.llvm.org/D59477 llvm-svn: 358498
2019-04-16 22:38:32 +08:00
bool shouldExpandShift(SelectionDAG &DAG, SDNode *N) const override {
if (DAG.getMachineFunction().getFunction().hasMinSize())
return false;
return true;
}
[RISCV] Prevent re-ordering some adds after shifts Summary: DAGCombine will normally turn a `(shl (add x, c1), c2)` into `(add (shl x, c2), c1 << c2)`, where `c1` and `c2` are constants. This can be prevented by a callback in TargetLowering. On RISC-V, materialising the constant `c1 << c2` can be more expensive than materialising `c1`, because materialising the former may take more instructions, and may use a register, where materialising the latter would not. This patch implements the hook in RISCVTargetLowering to prevent this transform, in the cases where: - `c1` fits into the immediate field in an `addi` instruction. - `c1` takes fewer instructions to materialise than `c1 << c2`. In future, DAGCombine could do the check to see whether `c1` fits into an add immediate, which might simplify more targets hooks than just RISC-V. Reviewers: asb, luismarques, efriedma Reviewed By: asb Subscribers: xbolva00, lebedev.ri, craig.topper, lewis-revill, Jim, hiraditya, rbar, johnrusso, simoncook, apazos, sabuasal, niosHD, kito-cheng, shiva0217, jrtc27, zzheng, edward-jones, rogfer01, MartinMosbeck, brucehoult, the_o, rkruppe, PkmX, jocewei, psnobl, benna, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D62857 llvm-svn: 363736
2019-06-19 04:38:08 +08:00
bool isDesirableToCommuteWithShift(const SDNode *N,
CombineLevel Level) const override;
[RISCV] Custom lower SHL_PARTS, SRA_PARTS, SRL_PARTS When not optimizing for minimum size (-Oz) we custom lower wide shifts (SHL_PARTS, SRA_PARTS, SRL_PARTS) instead of expanding to a libcall. Differential Revision: https://reviews.llvm.org/D59477 llvm-svn: 358498
2019-04-16 22:38:32 +08:00
[RISCV] Specify registers used in DWARF exception handling Defines RISCV registers for getExceptionPointerRegister() and getExceptionSelectorRegister(). Differential Revision: https://reviews.llvm.org/D63411 Patch by Edward Jones. Modified by Alex Bradbury to add CHECK lines to exception-pointer-register.ll. llvm-svn: 365301
2019-07-08 17:16:47 +08:00
/// If a physical register, this returns the register that receives the
/// exception address on entry to an EH pad.
CodeGen: Use Register in TargetLowering
2020-04-08 22:29:30 +08:00
Register
[RISCV] Specify registers used in DWARF exception handling Defines RISCV registers for getExceptionPointerRegister() and getExceptionSelectorRegister(). Differential Revision: https://reviews.llvm.org/D63411 Patch by Edward Jones. Modified by Alex Bradbury to add CHECK lines to exception-pointer-register.ll. llvm-svn: 365301
2019-07-08 17:16:47 +08:00
getExceptionPointerRegister(const Constant *PersonalityFn) const override;
/// If a physical register, this returns the register that receives the
/// exception typeid on entry to a landing pad.
CodeGen: Use Register in TargetLowering
2020-04-08 22:29:30 +08:00
Register
[RISCV] Specify registers used in DWARF exception handling Defines RISCV registers for getExceptionPointerRegister() and getExceptionSelectorRegister(). Differential Revision: https://reviews.llvm.org/D63411 Patch by Edward Jones. Modified by Alex Bradbury to add CHECK lines to exception-pointer-register.ll. llvm-svn: 365301
2019-07-08 17:16:47 +08:00
getExceptionSelectorRegister(const Constant *PersonalityFn) const override;
[RISCV] Avoid generating AssertZext for LP64 ABI when lowering floating LibCall The patch fixed the issue that RV64 didn't clear the upper bits when return complex floating value with lp64 ABI. float _Complex complex_add(float _Complex a, float _Complex b) { return a + b; } RealResult = zero_extend(RealA + RealB) ImageResult = ImageA + ImageB Return (RealResult | (ImageResult << 32)) The patch introduces shouldExtendTypeInLibCall target hook to suppress the AssertZext generation when lowering floating LibCall. Thanks to Eli's comments from the Bugzilla https://bugs.llvm.org/show_bug.cgi?id=42820 Differential Revision: https://reviews.llvm.org/D65497 llvm-svn: 370275
2019-08-29 07:40:37 +08:00
bool shouldExtendTypeInLibCall(EVT Type) const override;
[RISCV] Use sign extend for i32 arguments and returns in makeLibCall on RV64. As far as I know 32 bits arguments and returns on RV64 are always sign extended to i64. So I think we should be taking this into account around libcalls. Reviewed By: luismarques Differential Revision: https://reviews.llvm.org/D95285
2021-01-26 01:16:45 +08:00
bool shouldSignExtendTypeInLibCall(EVT Type, bool IsSigned) const override;
[RISCV] Avoid generating AssertZext for LP64 ABI when lowering floating LibCall The patch fixed the issue that RV64 didn't clear the upper bits when return complex floating value with lp64 ABI. float _Complex complex_add(float _Complex a, float _Complex b) { return a + b; } RealResult = zero_extend(RealA + RealB) ImageResult = ImageA + ImageB Return (RealResult | (ImageResult << 32)) The patch introduces shouldExtendTypeInLibCall target hook to suppress the AssertZext generation when lowering floating LibCall. Thanks to Eli's comments from the Bugzilla https://bugs.llvm.org/show_bug.cgi?id=42820 Differential Revision: https://reviews.llvm.org/D65497 llvm-svn: 370275
2019-08-29 07:40:37 +08:00
[RISCV] Implement the TargetLowering::getRegisterByName hook Summary: The hook should work for any RISC-V register. Non-allocatable registers do not need to be reserved, for the remaining the hook will only succeed if you pass clang the -ffixed-xX flag. This builds upon D67185, which currently only allows reserving GPRs. Reviewers: asb, lenary Reviewed By: lenary Tags: #llvm Differential Revision: https://reviews.llvm.org/D69130
2019-11-04 19:21:51 +08:00
/// Returns the register with the specified architectural or ABI name. This
/// method is necessary to lower the llvm.read_register.* and
/// llvm.write_register.* intrinsics. Allocatable registers must be reserved
/// with the clang -ffixed-xX flag for access to be allowed.
CodeGen: Use LLT instead of EVT in getRegisterByName Only PPC seems to be using it, and only checks some simple cases and doesn't distinguish between FP. Just switch to using LLT to simplify use from GlobalISel.
2019-12-28 22:18:56 +08:00
Register getRegisterByName(const char *RegName, LLT VT,
[RISCV] Implement the TargetLowering::getRegisterByName hook Summary: The hook should work for any RISC-V register. Non-allocatable registers do not need to be reserved, for the remaining the hook will only succeed if you pass clang the -ffixed-xX flag. This builds upon D67185, which currently only allows reserving GPRs. Reviewers: asb, lenary Reviewed By: lenary Tags: #llvm Differential Revision: https://reviews.llvm.org/D69130
2019-11-04 19:21:51 +08:00
const MachineFunction &MF) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
// Lower incoming arguments, copy physregs into vregs
SDValue LowerFormalArguments(SDValue Chain, CallingConv::ID CallConv,
bool IsVarArg,
const SmallVectorImpl<ISD::InputArg> &Ins,
const SDLoc &DL, SelectionDAG &DAG,
SmallVectorImpl<SDValue> &InVals) const override;
Revert "[RISCV] Make CanLowerReturn protected for downstream maintenance" This reverts commit d775841d7d6ee3e8bbf3a420590be9bb19433eaa.
2020-05-12 18:49:17 +08:00
bool CanLowerReturn(CallingConv::ID CallConv, MachineFunction &MF,
bool IsVarArg,
const SmallVectorImpl<ISD::OutputArg> &Outs,
LLVMContext &Context) const override;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
SDValue LowerReturn(SDValue Chain, CallingConv::ID CallConv, bool IsVarArg,
const SmallVectorImpl<ISD::OutputArg> &Outs,
const SmallVectorImpl<SDValue> &OutVals, const SDLoc &DL,
SelectionDAG &DAG) const override;
[RISCV] Initial support for function calls Note that this is just enough for simple function call examples to generate working code. Support for varargs etc follows in future patches. Differential Revision: https://reviews.llvm.org/D29936 llvm-svn: 317691
2017-11-08 21:41:21 +08:00
SDValue LowerCall(TargetLowering::CallLoweringInfo &CLI,
SmallVectorImpl<SDValue> &InVals) const override;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
bool shouldConvertConstantLoadToIntImm(const APInt &Imm,
Type *Ty) const override {
return true;
}
[RISCV] Implement mayBeEmittedAsTailCall for tail call optimization Implement TargetLowering callback mayBeEmittedAsTailCall for riscv in CodeGenPrepare, which will duplicate return instructions to enable tailcall optimization. Differential Revision: https://reviews.llvm.org/D73699
2020-02-18 23:53:26 +08:00
bool mayBeEmittedAsTailCall(const CallInst *CI) const override;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
bool shouldConsiderGEPOffsetSplit() const override { return true; }
[RISCV] Optimize multiplication by constant ... to shift/add or shift/sub. Do not enable it on riscv32 with the M extension where decomposeMulByConstant may not be an optimization. Reviewed By: luismarques, MaskRay Differential Revision: https://reviews.llvm.org/D82660
2020-07-08 09:48:46 +08:00
bool decomposeMulByConstant(LLVMContext &Context, EVT VT,
SDValue C) const override;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
TargetLowering::AtomicExpansionKind
shouldExpandAtomicRMWInIR(AtomicRMWInst *AI) const override;
Value *emitMaskedAtomicRMWIntrinsic(IRBuilder<> &Builder, AtomicRMWInst *AI,
Value *AlignedAddr, Value *Incr,
Value *Mask, Value *ShiftAmt,
AtomicOrdering Ord) const override;
TargetLowering::AtomicExpansionKind
shouldExpandAtomicCmpXchgInIR(AtomicCmpXchgInst *CI) const override;
Value *emitMaskedAtomicCmpXchgIntrinsic(IRBuilder<> &Builder,
AtomicCmpXchgInst *CI,
Value *AlignedAddr, Value *CmpVal,
Value *NewVal, Value *Mask,
AtomicOrdering Ord) const override;
[RISCV] Initial support of LoopVectorizer for RISC-V Vector. Define an option -riscv-vector-bits-max to specify the maximum vector bits for vectorizer. Loop vectorizer will use the value to check if it is safe to use the whole vector registers to vectorize the loop. It is not the optimum solution for loop vectorizing for scalable vector. It assumed the whole vector registers will be used to vectorize the code. If it is possible, we should configure vl to do vectorize instead of using whole vector registers. We only consider LMUL = 1 in this patch. This patch just an initial work for loop vectorizer for RISC-V Vector. Differential Revision: https://reviews.llvm.org/D95659
2021-01-27 15:00:46 +08:00
/// Returns true if the target allows unaligned memory accesses of the
/// specified type.
bool allowsMisalignedMemoryAccesses(
EVT VT, unsigned AddrSpace = 0, Align Alignment = Align(1),
MachineMemOperand::Flags Flags = MachineMemOperand::MONone,
bool *Fast = nullptr) const override;
[RISCV] Make visibility of overridden methods in RISCVISelLowering match the parent Currently, some fairly arbitrary subset of overriden methods in RISCVISelLowering are private rather than public (which is the visibility they have in TargetLowering). I suspect this is a holdover from too closely copying another backend. D78545 pointed out this can be difficult for some downstream patches, and nobody has come forward to suggest a reason for keeping the visibility as-is. This commit simply makes all overridden methods match the public visiblity of the parent. Differential Revision: https://reviews.llvm.org/D79928
2020-06-10 16:16:09 +08:00
private:
void analyzeInputArgs(MachineFunction &MF, CCState &CCInfo,
const SmallVectorImpl<ISD::InputArg> &Ins,
bool IsRet) const;
void analyzeOutputArgs(MachineFunction &MF, CCState &CCInfo,
const SmallVectorImpl<ISD::OutputArg> &Outs,
bool IsRet, CallLoweringInfo *CLI) const;
[RISCV] Generate address sequences suitable for mcmodel=medium This patch adds an implementation of a PC-relative addressing sequence to be used when -mcmodel=medium is specified. With absolute addressing, a 'medium' codemodel may cause addresses to be out of range. This is because while 'medium' implies a 2 GiB addressing range, this 2 GiB can be at any offset as opposed to 'small', which implies the first 2 GiB only. Note that LLVM/Clang currently specifies code models differently to GCC, where small and medium imply the same functionality as GCC's medlow and medany respectively. Differential Revision: https://reviews.llvm.org/D54143 Patch by Lewis Revill. llvm-svn: 357393
2019-04-01 22:42:56 +08:00
template <class NodeTy>
[RISCV] Add lowering of addressing sequences for PIC This patch allows lowering of PIC addresses by using PC-relative addressing for DSO-local symbols and accessing the address through the global offset table for non-DSO-local symbols. Differential Revision: https://reviews.llvm.org/D55303 llvm-svn: 363058
2019-06-11 20:57:47 +08:00
SDValue getAddr(NodeTy *N, SelectionDAG &DAG, bool IsLocal = true) const;
[RISCV] Generate address sequences suitable for mcmodel=medium This patch adds an implementation of a PC-relative addressing sequence to be used when -mcmodel=medium is specified. With absolute addressing, a 'medium' codemodel may cause addresses to be out of range. This is because while 'medium' implies a 2 GiB addressing range, this 2 GiB can be at any offset as opposed to 'small', which implies the first 2 GiB only. Note that LLVM/Clang currently specifies code models differently to GCC, where small and medium imply the same functionality as GCC's medlow and medany respectively. Differential Revision: https://reviews.llvm.org/D54143 Patch by Lewis Revill. llvm-svn: 357393
2019-04-01 22:42:56 +08:00
[RISCV] Add lowering of global TLS addresses This patch adds lowering for global TLS addresses for the TLS models of InitialExec, GlobalDynamic, LocalExec and LocalDynamic. LocalExec support required using a 4-operand add instruction, which uses the fourth operand to express a relocation on the symbol. The necessary fixup is emitted when the instruction is emitted. Differential Revision: https://reviews.llvm.org/D55305 llvm-svn: 363771
2019-06-19 16:40:59 +08:00
SDValue getStaticTLSAddr(GlobalAddressSDNode *N, SelectionDAG &DAG,
bool UseGOT) const;
SDValue getDynamicTLSAddr(GlobalAddressSDNode *N, SelectionDAG &DAG) const;
[RISCV] Codegen support for memory operations on global addresses Differential Revision: https://reviews.llvm.org/D39103 llvm-svn: 317688
2017-11-08 21:24:21 +08:00
SDValue lowerGlobalAddress(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support and tests for a variety of additional LLVM IR constructs Previous patches primarily ensured that codegen was possible for the standard RISC-V instructions. However, there are a number of IR inputs that wouldn't be appropriately lowered. This patch both adds test cases and supports lowering for a number of these cases: * Improved sext/zext/trunc support * Support for setcc variants that don't map directly to RISC-V instructions * Lowering mul, and hence support for external symbols * addc, adde, subc, sube * mulhs, srem, mulhu, urem, udiv, sdiv * {srl,sra,shl}_parts * brind * br_jt * bswap, ctlz, cttz, ctpop * rotl, rotr * BlockAddress operands Differential Revision: https://reviews.llvm.org/D29938 llvm-svn: 318737
2017-11-21 16:11:03 +08:00
SDValue lowerBlockAddress(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add codegen for RV32F floating point load/store As part of this, add support for load/store from the constant pool. This is used to materialise f32 constants. llvm-svn: 327979
2018-03-20 21:26:12 +08:00
SDValue lowerConstantPool(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Basic jump table lowering This patch enables jump table lowering in the RISC-V backend. In addition to the test case included, the new lowering was tested by compiling the OCaml runtime and running it under qemu. Differential Revision: https://reviews.llvm.org/D92097
2020-11-25 22:01:19 +08:00
SDValue lowerJumpTable(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add lowering of global TLS addresses This patch adds lowering for global TLS addresses for the TLS models of InitialExec, GlobalDynamic, LocalExec and LocalDynamic. LocalExec support required using a 4-operand add instruction, which uses the fourth operand to express a relocation on the symbol. The necessary fixup is emitted when the instruction is emitted. Differential Revision: https://reviews.llvm.org/D55305 llvm-svn: 363771
2019-06-19 16:40:59 +08:00
SDValue lowerGlobalTLSAddress(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Implement lowering of ISD::SELECT Although ISD::SELECT_CC is a more natural match for RISCVISD::SELECT_CC (and ultimately the integer RISC-V conditional branch instructions), we choose to expand ISD::SELECT_CC and lower ISD::SELECT. The appropriate compare+branch will be created in the case where an ISD::SELECT condition value is created by an ISD::SETCC node, which operates on XLen types. Other datatypes such as floating point don't have conditional branch instructions, and lowering ISD::SELECT allows more flexibility for handling these cases. Differential Revision: https://reviews.llvm.org/D29937 llvm-svn: 318735
2017-11-21 15:51:32 +08:00
SDValue lowerSELECT(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support for varargs Includes support for expanding va_copy. Also adds support for using 'aligned' registers when necessary for vararg calls, and ensure the frame pointer always points to the bottom of the vararg spill region. This is necessary to ensure that the saved return address and stack pointer are always available at fixed known offsets of the frame pointer. Differential Revision: https://reviews.llvm.org/D40805 llvm-svn: 322215
2018-01-11 03:41:03 +08:00
SDValue lowerVASTART(SDValue Op, SelectionDAG &DAG) const;
[RISCV][NFC] Fix naming of RISCVISelLowering::{LowerRETURNADDR,LowerFRAMEADDR} Rename to lowerRETURNADDR, lowerFRAMEADDR in order to be consistent with the LLVM coding style and the other functions in this file. llvm-svn: 343752
2018-10-04 13:27:50 +08:00
SDValue lowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Custom lower SHL_PARTS, SRA_PARTS, SRL_PARTS When not optimizing for minimum size (-Oz) we custom lower wide shifts (SHL_PARTS, SRA_PARTS, SRL_PARTS) instead of expanding to a libcall. Differential Revision: https://reviews.llvm.org/D59477 llvm-svn: 358498
2019-04-16 22:38:32 +08:00
SDValue lowerShiftLeftParts(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerShiftRightParts(SDValue Op, SelectionDAG &DAG, bool IsSRA) const;
[RISCV] Add ISel support for RVV vector/scalar forms This patch extends the SDNode ISel support for RVV from only the vector/vector instructions to include the vector/scalar and vector/immediate forms. It uses splat_vector to carry the scalar in each case, except when XLEN<SEW (RV32 SEW=64) when a custom node `SPLAT_VECTOR_I64` is used for type-legalization and to encode the fact that the value is sign-extended to SEW. When the scalar is a full 64-bit value we use a sequence to materialize the constant into the vector register. The non-intrinsic ISel patterns have also been split into their own file. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D93312
2020-12-15 21:05:32 +08:00
SDValue lowerSPLATVECTOR(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add ISel patterns for scalable mask exts & truncs Original patch by @rogfer01. This patch adds support for sign-, zero-, and any-extension from scalable mask vector types to integer vector types, as well as truncation in the opposite direction. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D94590
2021-01-13 20:27:05 +08:00
SDValue lowerVectorMaskExt(SDValue Op, SelectionDAG &DAG,
int64_t ExtTrueVal) const;
SDValue lowerVectorMaskTrunc(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add RVV insertelt/extractelt scalable-vector patterns Original patch by @rogfer01. This patch adds support for insertelt and extractelt operations on scalable vectors. Special care must be taken on RV32 when dealing with i64 vectors as there are no straightforward ways to insert a 64-bit element without a register of that size. To that end, both are custom-lowered to different sequences. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: Fraser Cormack <fraser@codeplay.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D94615
2021-01-12 01:42:05 +08:00
SDValue lowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerEXTRACT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support llvm.thread.pointer Fixes https://bugs.llvm.org/show_bug.cgi?id=45303 (clang crashed on __builtin_thread_pointer) Reviewed By: lenary, MaskRay, luismarques Differential Revision: https://reviews.llvm.org/D76828
2020-03-28 08:27:00 +08:00
SDValue LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Define vsadd/vsaddu/vssub/vssubu intrinsics. We work with @rogfer01 from BSC to come out this patch. Authored-by: Roger Ferrer Ibanez <rofirrim@gmail.com> Co-Authored-by: ShihPo Hung <shihpo.hung@sifive.com> Co-Authored-by: Monk Chiang <monk.chiang@sifive.com> Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D93366
2020-12-17 13:45:52 +08:00
SDValue LowerINTRINSIC_W_CHAIN(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support scalable-vector integer reduction intrinsics This patch adds support for the integer reduction intrinsics supported by RVV. This excludes "mul" which has no corresponding instruction. The reduction instructions in RVV have slightly complicated type constraints given they always produce a single "M1" vector register. They are lowered to custom nodes including the second "scalar" reduction operand to simplify the patterns and in the hope that they can be useful for future DAG combines. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95620
2021-01-28 23:31:27 +08:00
SDValue lowerVECREDUCE(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Support the scalable-vector fadd reduction intrinsic This patch adds support for both the fadd reduction intrinsic, in both the ordered and unordered modes. The fmin and fmax intrinsics are not currently supported due to a discrepancy between the LLVM semantics and the RVV ISA behaviour with regards to signaling NaNs. This behaviour is likely fixed in version 2.3 of the RISC-V F/D/Q extension, but until then the intrinsics can be left unsupported. Reviewed By: craig.topper Differential Revision: https://reviews.llvm.org/D95870
2021-02-02 22:34:51 +08:00
SDValue lowerFPVECREDUCE(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
SDValue lowerFixedLengthVectorLoadToRVV(SDValue Op, SelectionDAG &DAG) const;
SDValue lowerFixedLengthVectorStoreToRVV(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add support for integer fixed vector setcc I believe I've covered all orderings of splat operands here. Better canonicalization in lowering might help reduce this. I did not handle the immediate adjustments needed for set(u)gt/set(u)lt. Testing here is limited to byte types because the scalable vector type used for masks for the store is calculated assuming 8 byte elements. But for the setcc its based on the element count of the container type for the setcc input. So they don't agree. We'll need to enhanced D96352 to handle this I think. Differential Revision: https://reviews.llvm.org/D96443
2021-02-13 01:20:54 +08:00
SDValue lowerFixedLengthVectorSetccToRVV(SDValue Op, SelectionDAG &DAG) const;
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
SDValue lowerToScalableOp(SDValue Op, SelectionDAG &DAG,
unsigned NewOpc) const;
[RISCV] Lower the tail pseudoinstruction This patch lowers the tail pseudoinstruction. This has been modeled after ARM's tail call opt. llvm-svn: 333137
2018-05-24 06:44:08 +08:00
[RISCV][NFC] IsEligibleForTailCallOptimization -> isEligibleForTailCallOptimization Also clang-format the modified hunks. llvm-svn: 354584
2019-02-21 22:31:41 +08:00
bool isEligibleForTailCallOptimization(
CCState &CCInfo, CallLoweringInfo &CLI, MachineFunction &MF,
const SmallVector<CCValAssign, 16> &ArgLocs) const;
[RISCV] Codegen for i8, i16, and i32 atomicrmw with RV32A Introduce a new RISCVExpandPseudoInsts pass to expand atomic pseudo-instructions after register allocation. This is necessary in order to ensure that register spills aren't introduced between LL and SC, thus breaking the forward progress guarantee for the operation. AArch64 does something similar for CmpXchg (though only at O0), and Mips is moving towards this approach (see D31287). See also [this mailing list post](http://lists.llvm.org/pipermail/llvm-dev/2016-May/099490.html) from James Knight, which summarises the issues with lowering to ll/sc in IR or pre-RA. See the [accompanying RFC thread](http://lists.llvm.org/pipermail/llvm-dev/2018-June/123993.html) for an overview of the lowering strategy. Differential Revision: https://reviews.llvm.org/D47882 llvm-svn: 342534
2018-09-19 18:54:22 +08:00
[RISCV] Add support for -ffixed-xX flags This adds support for reserving GPRs such that the compiler will not choose a register for register allocation. The implementation follows the same design as for AArch64; each reserved register becomes a target feature and used for getting the reserved registers for a given MachineFunction. The backend checks that it does not need to write to any reserved register; if it does a relevant error is generated. Differential Revision: https://reviews.llvm.org/D67185
2019-10-23 04:25:01 +08:00
/// Generate error diagnostics if any register used by CC has been marked
/// reserved.
void validateCCReservedRegs(
const SmallVectorImpl<std::pair<llvm::Register, llvm::SDValue>> &Regs,
MachineFunction &MF) const;
[RISCV] Add initial support for converting fixed vectors to scalable vectors during lowering to use RVV instructions. This is an alternative to D95563. This is modeled after a similar feature for AArch64's SVE that uses predicated scalable vector instructions.a Rather than use predication, this patch uses an explicit VL operand. I've limited it to always use LMUL=1 for now, but we can improve this in the future. This requires a bunch of new ISD opcodes to carry the VL operand. I think we can probably lower intrinsics to these ISD opcodes to cut down on the size of the isel table. Which is why I've added patterns for all integer/float types and not just LMUL=1. I'm only testing one vector width right now, but the width is programmable via the command line. Reviewed By: frasercrmck Differential Revision: https://reviews.llvm.org/D95705
2021-02-09 02:32:32 +08:00
bool useRVVForFixedLengthVectorVT(MVT VT) const;
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
};
[RISCV] Define vadd/vsub/vrsub intrinsics and lower to V instructions. This patch is based on the proposal from Roger Ferrer Ibanez. http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html Differential Revision: https://reviews.llvm.org/D93013
2020-12-11 15:16:08 +08:00
[RISCV] Initial support of LoopVectorizer for RISC-V Vector. Define an option -riscv-vector-bits-max to specify the maximum vector bits for vectorizer. Loop vectorizer will use the value to check if it is safe to use the whole vector registers to vectorize the loop. It is not the optimum solution for loop vectorizing for scalable vector. It assumed the whole vector registers will be used to vectorize the code. If it is possible, we should configure vl to do vectorize instead of using whole vector registers. We only consider LMUL = 1 in this patch. This patch just an initial work for loop vectorizer for RISC-V Vector. Differential Revision: https://reviews.llvm.org/D95659
2021-01-27 15:00:46 +08:00
namespace RISCV {
// We use 64 bits as the known part in the scalable vector types.
static constexpr unsigned RVVBitsPerBlock = 64;
[RISCV] Remove superfluous semicolon. NFC
2021-02-11 03:20:07 +08:00
} // namespace RISCV
[RISCV] Initial support of LoopVectorizer for RISC-V Vector. Define an option -riscv-vector-bits-max to specify the maximum vector bits for vectorizer. Loop vectorizer will use the value to check if it is safe to use the whole vector registers to vectorize the loop. It is not the optimum solution for loop vectorizing for scalable vector. It assumed the whole vector registers will be used to vectorize the code. If it is possible, we should configure vl to do vectorize instead of using whole vector registers. We only consider LMUL = 1 in this patch. This patch just an initial work for loop vectorizer for RISC-V Vector. Differential Revision: https://reviews.llvm.org/D95659
2021-01-27 15:00:46 +08:00
[RISCV] Define vadd/vsub/vrsub intrinsics and lower to V instructions. This patch is based on the proposal from Roger Ferrer Ibanez. http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html Differential Revision: https://reviews.llvm.org/D93013
2020-12-11 15:16:08 +08:00
namespace RISCVVIntrinsicsTable {
struct RISCVVIntrinsicInfo {
[RISCV] Reduce field sizes in searchable tables to reduce binary size.
2021-01-28 04:22:36 +08:00
unsigned IntrinsicID;
uint8_t ExtendedOperand;
[RISCV] Define vadd/vsub/vrsub intrinsics and lower to V instructions. This patch is based on the proposal from Roger Ferrer Ibanez. http://lists.llvm.org/pipermail/llvm-dev/2020-October/145850.html Differential Revision: https://reviews.llvm.org/D93013
2020-12-11 15:16:08 +08:00
};
using namespace RISCV;
#define GET_RISCVVIntrinsicsTable_DECL
#include "RISCVGenSearchableTables.inc"
} // end namespace RISCVVIntrinsicsTable
[RISCV] Implement vlseg intrinsics. For Zvlsseg, we need continuous vector registers for the values. We need to define new register classes for the different combinations of (number of fields and LMUL). For example, when the number of fields(NF) = 3, LMUL = 2, the values will be assigned to (V0M2, V2M2, V4M2), (V2M2, V4M2, V6M2), (V4M2, V6M2, V8M2), ... We define the vlseg intrinsics with multiple outputs. There is no way to describe the codegen patterns with multiple outputs in the tablegen files. We do the codegen in RISCVISelDAGToDAG and use EXTRACT_SUBREG to extract the values of output. The multiple scalable vector values will be put into a struct. This patch is depended on the support for scalable vector struct. Differential Revision: https://reviews.llvm.org/D94229
2020-12-31 17:14:15 +08:00
namespace RISCVZvlssegTable {
struct RISCVZvlsseg {
[RISCV] Reduce field sizes in searchable tables to reduce binary size.
2021-01-28 04:22:36 +08:00
unsigned IntrinsicID;
uint8_t SEW;
uint8_t LMUL;
uint8_t IndexLMUL;
uint16_t Pseudo;
[RISCV] Implement vlseg intrinsics. For Zvlsseg, we need continuous vector registers for the values. We need to define new register classes for the different combinations of (number of fields and LMUL). For example, when the number of fields(NF) = 3, LMUL = 2, the values will be assigned to (V0M2, V2M2, V4M2), (V2M2, V4M2, V6M2), (V4M2, V6M2, V8M2), ... We define the vlseg intrinsics with multiple outputs. There is no way to describe the codegen patterns with multiple outputs in the tablegen files. We do the codegen in RISCVISelDAGToDAG and use EXTRACT_SUBREG to extract the values of output. The multiple scalable vector values will be put into a struct. This patch is depended on the support for scalable vector struct. Differential Revision: https://reviews.llvm.org/D94229
2020-12-31 17:14:15 +08:00
};
using namespace RISCV;
#define GET_RISCVZvlssegTable_DECL
#include "RISCVGenSearchableTables.inc"
} // namespace RISCVZvlssegTable
[RISCV] Initial codegen support for ALU operations This adds the minimum necessary to support codegen for simple ALU operations on RV32. Prolog and epilog insertion, support for memory operations etc etc follow in future patches. Leave guessInstructionProperties=1 until https://reviews.llvm.org/D37065 is reviewed and lands. Differential Revision: https://reviews.llvm.org/D29933 llvm-svn: 316188
2017-10-20 05:37:38 +08:00
}
#endif