Commit Graph

121 Commits

Author SHA1 Message Date
Chandler Carruth c1bedac3bd [cleanup] Hoist an if-else chain on ISD opcodes (really designed for
switches) into a switch, and sink them into a dispatch function that can
return the result rather than awkward variable setting with breaks.

llvm-svn: 212166
2014-07-02 06:23:34 +00:00
Chandler Carruth 722289f311 [cleanup] Remove dead 'break;' statements that I meant to nuke in
r212158 but missed.

Thanks to Craig for spotting the goof!

llvm-svn: 212159
2014-07-02 04:39:34 +00:00
Chandler Carruth 2746c2861f [cleanup] Hoist the promotion dispatch logic into the promote function
so that we can use return to express it more cleanly and avoid so many
nested switch statements.

llvm-svn: 212158
2014-07-02 03:07:15 +00:00
Chandler Carruth 1cfa895c4a [cleanup] Nuke the 'VectorOp' bit of the promote method names.
This doesn't add any information for methods in the VectorLegalizer
class that clearly take SDAG operations to legalize.

llvm-svn: 212157
2014-07-02 03:07:11 +00:00
Chandler Carruth 68adf1568a [x86] Clean up and modernize the doxygen and API comments for the vector
operation legalization code.

llvm-svn: 212155
2014-07-02 02:16:57 +00:00
Benjamin Kramer f3ad23551d SDAG: Legalize vector BSWAP into a shuffle if the shuffle is legal but the bswap not.
- On ARM/ARM64 we get a vrev because the shuffle matching code is really smart. We still unroll anything that's not v4i32 though.
- On X86 we get a pshufb with SSSE3. Required more cleverness in isShuffleMaskLegal.
- On PPC we get a vperm for v8i16 and v4i32. v2i64 is unrolled.

llvm-svn: 209123
2014-05-19 13:12:38 +00:00
Craig Topper 8c0b4d0791 Convert more SelectionDAG functions to use ArrayRef.
llvm-svn: 207397
2014-04-28 05:57:50 +00:00
Craig Topper 48d114bed1 Convert SelectionDAG::getNode methods to use ArrayRef<SDValue>.
llvm-svn: 207327
2014-04-26 18:35:24 +00:00
Adam Nemet 24381f1cb7 [VectorLegalizer/X86] Don't unvectorize fp_to_uint for v8f32->v8i16
Rather than LegalizeAction::Expand, this needs LegalizeAction::Promote to get
promoted to fp_to_sint v8f32->v8i32.  This is a legal operation on AVX.

For that to work properly, we also need to teach the legalizer about the
specific promotion required here.  The default vector promotion uses
bitcasting to a vector type of the same total size.  We want to promote the
vector element type, effectively widening the operation and then truncating
the result.  This is analogous to the current logic of how int_to_fp is
promoted.

The change also factors out some code from the int_to_fp promotion code to
ValueType::widenIntegerVectorElementType.  This is now shared between
int_to_fp and fp_to_int.

There is no longer need for the custom lowering of fp_to_sint f32->v8i16 in
X86.  It can now go through the new target-independent fp_to_*int promotion
logic.

I also checked that no other target uses Promote for these ops yet, so there
shouldn't be any unexpected change in behavior.

Fixes <rdar://problem/16202247>

llvm-svn: 204058
2014-03-17 17:06:14 +00:00
Benjamin Kramer b6d0bd48bd [C++11] Replace llvm::next and llvm::prior with std::next and std::prev.
Remove the old functions.

llvm-svn: 202636
2014-03-02 12:27:27 +00:00
Hal Finkel 5c968d9440 Expand vector bswap in LegalizeVectorOps
ISD::BSWAP was missing from the list of node types that should be expanded
element-wise.

llvm-svn: 200705
2014-02-03 17:27:25 +00:00
Richard Sandiford 39c1ce4dc1 Keep TBAA info when rewriting SelectionDAG loads and stores
Most SelectionDAG code drops the TBAA info when creating a new form of a
load and store (e.g. during legalization, or when converting a plain
load to an extending one).  This patch tries to catch all cases where
the TBAA information can legitimately be carried over.

The patch adds alternative forms of getLoad() and getExtLoad() that take
a MachineMemOperand instead of individual fields.  (The corresponding
getTruncStore() already exists.)  The idea is to use the MachineMemOperand
forms when all fields are carried over (size, pointer info, isVolatile,
isNonTemporal, alignment and TBAA info).  If some adjustment is being
made, e.g. to narrow the load, then we still pass the individual fields
but also pass the TBAA info.

llvm-svn: 193517
2013-10-28 11:17:59 +00:00
Matt Arsenault bc08ddba58 Remove pointless assertion after r190376
llvm-svn: 190565
2013-09-12 01:07:49 +00:00
Matt Arsenault d232222f34 Don't use getSetCCResultType for creating a vselect
The vselect mask isn't a setcc.

This breaks in the case when the result of getSetCCResultType
is larger than the vector operands

e.g. %tmp = select i1 %cmp <2 x i8> %a, <2 x i8> %b
when getSetCCResultType returns <2 x i32>, the assertion
that the (MaskTy.getSizeInBits() == Op1.getValueType().getSizeInBits())
is hit.

No test since I don't think I can hit this with any of the current
targets. The R600/SI implementation would break, since it returns a
vector of i1 for this, but it doesn't reach ExpandSELECT for other
reasons.

llvm-svn: 190376
2013-09-10 00:41:56 +00:00
Tom Stellard 838e2344ec SelectionDAG: Remove unnecessary uses of TargetLowering::getPointerTy()
If we have a binary operation like ISD:ADD, we can set the result type
equal to the result type of one of its operands rather than using
TargetLowering::getPointerTy().

Also, any use of DAG.getIntPtrConstant(C) as an operand for a binary
operation can be replaced with:
DAG.getConstant(C, OtherOperand.getValueType());

llvm-svn: 189227
2013-08-26 15:06:10 +00:00
Tom Stellard 1b2c2d8414 SelectionDAG: Make sure stores are always added to the LegalizedNodes list
When truncated vector stores were being custom lowered in
VectorLegalizer::LegalizeOp(), the old (illegal) and new (legal) node pair
was not being added to LegalizedNodes list.  Instead of the legalized
result being passed to VectorLegalizer::TranslateLegalizeResult(),
the result was being passed back into VectorLegalizer::LegalizeOp(),
which ended up adding a (new, new) pair to the list instead.

This was causing an assertion failure when a custom lowered truncated
vector store was the last instruction a basic block and the VectorLegalizer
was unable to find it in the LegalizedNodes list when updating the
DAG root.

llvm-svn: 188953
2013-08-21 22:42:58 +00:00
Hal Finkel 0c5c01aa4a Add a llvm.copysign intrinsic
This adds a llvm.copysign intrinsic; We already have Libfunc recognition for
copysign (which is turned into the FCOPYSIGN SDAG node). In order to
autovectorize calls to copysign in the loop vectorizer, we need a corresponding
intrinsic as well.

In addition to the expected changes to the language reference, the loop
vectorizer, BasicTTI, and the SDAG builder (the intrinsic is transformed into
an FCOPYSIGN node, just like the function call), this also adds FCOPYSIGN to a
few lists in LegalizeVector{Ops,Types} so that vector copysigns can be
expanded.

In TargetLoweringBase::initActions, I've made the default action for FCOPYSIGN
be Expand for vector types. This seems correct for all in-tree targets, and I
think is the right thing to do because, previously, there was no way to generate
vector-values FCOPYSIGN nodes (and most targets don't specify an action for
vector-typed FCOPYSIGN).

llvm-svn: 188728
2013-08-19 23:35:46 +00:00
Hal Finkel 171817ee8a Add ISD::FROUND for libm round()
All libm floating-point rounding functions, except for round(), had their own
ISD nodes. Recent PowerPC cores have an instruction for round(), and so here I'm
adding ISD::FROUND so that round() can be custom lowered as well.

For the most part, this is straightforward. I've added an intrinsic
and a matching ISD node just like those for nearbyint() and friends. The
SelectionDAG pattern I've named frnd (because ISD::FP_ROUND has already claimed
fround).

This will be used by the PowerPC backend in a follow-up commit.

llvm-svn: 187926
2013-08-07 22:49:12 +00:00
Tom Stellard d42c594960 TargetLowering: Add getVectorIdxTy() function v2
This virtual function can be implemented by targets to specify the type
to use for the index operand of INSERT_VECTOR_ELT, EXTRACT_VECTOR_ELT,
INSERT_SUBVECTOR, EXTRACT_SUBVECTOR.  The default implementation returns
the result from TargetLowering::getPointerTy()

The previous code was using TargetLowering::getPointerTy() for vector
indices, because this is guaranteed to be legal on all targets.  However,
using TargetLowering::getPointerTy() can be a problem for targets with
pointer sizes that differ across address spaces.  On such targets,
when vectors need to be loaded or stored to an address space other than the
default 'zero' address space (which is the address space assumed by
TargetLowering::getPointerTy()), having an index that
is a different size than the pointer can lead to inefficient
pointer calculations, (e.g. 64-bit adds for a 32-bit address space).

There is no intended functionality change with this patch.

llvm-svn: 187748
2013-08-05 22:22:01 +00:00
Stephen Lin cfe7f352c7 Remove trailing whitespace from SelectionDAG/*.cpp
llvm-svn: 185780
2013-07-08 00:37:03 +00:00
Matt Arsenault d2f0332a29 Introduce getSelect usage and use more getSelectCC
llvm-svn: 184012
2013-06-14 22:04:37 +00:00
Benjamin Kramer 351d53c225 Remove double semicolons.
llvm-svn: 182778
2013-05-28 16:31:26 +00:00
Andrew Trick ef9de2a739 Track IR ordering of SelectionDAG nodes 2/4.
Change SelectionDAG::getXXXNode() interfaces as well as call sites of
these functions to pass in SDLoc instead of DebugLoc.

llvm-svn: 182703
2013-05-25 02:42:55 +00:00
Matt Arsenault 75865923c9 Add LLVMContext argument to getSetCCResultType
llvm-svn: 182180
2013-05-18 00:21:46 +00:00
Matt Arsenault a5733dc97e Fix vselect when getSetCCResultType returns a different type from the operands
llvm-svn: 181348
2013-05-07 20:24:18 +00:00
Nadav Rotem b7f90bd97b SelectionDAG compile time improvement.
One of the phases of SelectionDAG is LegalizeVectors. We don't need to sort the DAG and copy nodes around if there are no vector ops.

Speeds up the compilation time of SelectionDAG on a big scalar workload by ~8%.

llvm-svn: 175929
2013-02-22 23:33:30 +00:00
Michael Liao 7fb39669ef Fix PR15267
- When extloading from a vector with non-byte-addressable element, e.g.
  <4 x i1>, the current logic breaks. Extend the current logic to
  fix the case where the element type is not byte-addressable by loading
  all bytes, bit-extracting/packing each element.

llvm-svn: 175642
2013-02-20 18:04:21 +00:00
Preston Gurd 0959bb707d This patch aims to reduce compile time in LegalizeTypes by using SmallDenseMap,
with an initial number of elements,  instead of DenseMap, which has
zero initial elements, in order to avoid the copying of elements
when the size changes and to avoid allocating space every time
LegalizeTypes is run. This patch will not affect the memory footprint,
because DenseMap will increase the element size to 64
when the first element is added.

Patch by Wan Xiaofei.

llvm-svn: 173448
2013-01-25 15:18:54 +00:00
Benjamin Kramer 5ea0349ef5 When lowering an inreg sext first shift left, then right arithmetically.
Shifting right two times will only yield zero. Should fix
SingleSource/UnitTests/SignlessTypes/factor.

llvm-svn: 172322
2013-01-12 19:06:44 +00:00
Nadav Rotem dbe5c72d03 PPC: Implement efficient lowering of sign_extend_inreg.
llvm-svn: 172269
2013-01-11 22:57:48 +00:00
Patrik Hagglund fd41b5b969 Change TargetLowering::getTypeToPromoteTo to take and return MVTs,
instead of EVTs.

llvm-svn: 170529
2012-12-19 11:21:04 +00:00
Patrik Hagglund d7cdcf8cb5 Change TargetLowering::getTruncStoreAction to take MVTs, instead of EVTs.
llvm-svn: 170510
2012-12-19 08:28:51 +00:00
Patrik Hagglund e98b7a0389 Revert EVT->MVT changes, r169836-169851, due to buildbot failures.
llvm-svn: 169854
2012-12-11 11:14:33 +00:00
Patrik Hagglund ffb60f7c08 Change TargetLowering::getTypeToPromoteTo to take and return MVTs,
instead of EVTs.

llvm-svn: 169844
2012-12-11 09:54:23 +00:00
Patrik Hagglund 7ffcd226dd Change TargetLowering::getTruncStoreAction to take MVTs, instead of EVTs.
llvm-svn: 169841
2012-12-11 09:42:24 +00:00
Eli Friedman 30834940ec Mark FP_EXTEND form v2f32 to v2f64 as "expand" for ARM NEON. Patch by Pete Couperus.
llvm-svn: 168240
2012-11-17 01:52:46 +00:00
Eli Friedman e6385e61b5 Mark FP_ROUND for converting NEON v2f64 to v2f32 as expand. Add a missing
case to vector legalization so this actually works.

Patch by Pete Couperus.  Fixes PR12540.

llvm-svn: 168107
2012-11-15 22:44:27 +00:00
Nadav Rotem 10f6b8802b Fix a typo.
llvm-svn: 163094
2012-09-02 12:21:50 +00:00
Nadav Rotem 500d691d4a Generate better select code by allowing the target to use scalar select, and not sign-extend.
llvm-svn: 163086
2012-09-02 08:20:07 +00:00
Pete Cooper 2455e9c4a5 Only legalise a VSELECT in to bitwise operations if the vector mask bool is zeros or all ones. A vector bool with just ones isn't suitable for masking with.
No test case unfortunately as i couldn't find a target which fit all
the conditions needed to hit this code.

llvm-svn: 163075
2012-09-01 22:27:48 +00:00
Nadav Rotem ea973bda26 Currently targets that do not support selects with scalar conditions and vector operands - scalarize the code. ARM is such a target
because it does not support CMOV of vectors. To implement this efficientlyi, we broadcast the condition bit and use a sequence of NAND-OR
to select between the two operands. This is the same sequence we use for targets that don't have vector BLENDs (like SSE2).

rdar://12201387

llvm-svn: 162926
2012-08-30 19:17:29 +00:00
Craig Topper 2da13f9ef8 Add FMA to switch statement in VectorLegalizer::LegalizeOp so that it can be expanded when it isn't legal.
llvm-svn: 162894
2012-08-30 07:34:22 +00:00
Jim Grosbach e0c10d8b86 'Promote' vector [su]int_to_fp should widen elements.
Teach vector legalization how to honor Promote for int to float
conversions. The code checking whether to promote the operation knew
to look at the operand, but the actual promotion code didn't. This
fixes that. The operand is promoted up via [zs]ext.

rdar://11762659

llvm-svn: 159378
2012-06-28 21:03:44 +00:00
Nadav Rotem 02ef0c3524 When emulating vselect using OR/AND/XOR make sure to bitcast the result back to the original type.
llvm-svn: 154764
2012-04-15 15:08:09 +00:00
Craig Topper ee4dab5f1f Convert assert(0) to llvm_unreachable
llvm-svn: 149816
2012-02-05 08:31:47 +00:00
Chandler Carruth 637cc6a8aa Initial CodeGen support for CTTZ/CTLZ where a zero input produces an
undefined result. This adds new ISD nodes for the new semantics,
selecting them when the LLVM intrinsic indicates that the undef behavior
is desired. The new nodes expand trivially to the old nodes, so targets
don't actually need to do anything to support these new nodes besides
indicating that they should be expanded. I've done this for all the
operand types that I could figure out for all the targets. Owners of
various targets, please review and let me know if any of these are
incorrect.

Note that the expand behavior is *conservatively correct*, and exactly
matches LLVM's current behavior with these operations. Ideally this
patch will not change behavior in any way. For example the regtest suite
finds the exact same instruction sequences coming out of the code
generator. That's why there are no new tests here -- all of this is
being exercised by the existing test suite.

Thanks to Duncan Sands for reviewing the various bits of this patch and
helping me get the wrinkles ironed out with expanding for each target.
Also thanks to Chris for clarifying through all the discussions that
this is indeed the approach he was looking for. That said, there are
likely still rough spots. Further review much appreciated.

llvm-svn: 146466
2011-12-13 01:56:10 +00:00
Nadav Rotem 8824472a25 Improve code generation for vselect on SSE2:
When checking the availability of instructions using the TLI, a 'promoted'
instruction IS available. It means that the value is bitcasted to another type
for which there is an operation. The correct check for the availablity of an
instruction is to check if it should be expanded.

llvm-svn: 142542
2011-10-19 20:43:16 +00:00
Nadav Rotem 75c2229f41 Fix a bug in the legalization of vector anyext-load and trunc-store. Mem Index starts with zero.
llvm-svn: 142434
2011-10-18 22:32:43 +00:00
Duncan Sands d278d35b13 Fix a bunch of unused variable warnings when doing a release
build with gcc-4.6.

llvm-svn: 142350
2011-10-18 12:44:00 +00:00
Chad Rosier c17257c4cb Removed set, but unused variable.
Patch by Joe Abbey <jabbey@arxan.com>.

llvm-svn: 142206
2011-10-17 18:01:59 +00:00
Nadav Rotem ebe13bc3f1 Move the legalization of vector loads and stores into LegalizeVectorOps. In some
cases we need the second type-legalization pass in order to support all cases.

llvm-svn: 142060
2011-10-15 07:41:10 +00:00
Nadav Rotem 7aaa0aa7a7 white space cleanups
llvm-svn: 139994
2011-09-18 10:29:29 +00:00
Nadav Rotem 66dc9ae08d Fix the assertion which checks the size of the input operand.
llvm-svn: 139633
2011-09-13 20:03:38 +00:00
Nadav Rotem 52202fbf2d Add vselect target support for targets that do not support blend but do support
xor/and/or (For example SSE2).

llvm-svn: 139623
2011-09-13 19:17:42 +00:00
Duncan Sands f2641e1bc1 Add codegen support for vector select (in the IR this means a select
with a vector condition); such selects become VSELECT codegen nodes.
This patch also removes VSETCC codegen nodes, unifying them with SETCC
nodes (codegen was actually often using SETCC for vector SETCC already).
This ensures that various DAG combiner optimizations kick in for vector
comparisons.  Passes dragonegg bootstrap with no testsuite regressions
(nightly testsuite as well as "make check-all").  Patch mostly by
Nadav Rotem.

llvm-svn: 139159
2011-09-06 19:07:46 +00:00
Nadav Rotem 771f29677f [VECTOR-SELECT]
During type legalization we often use the SIGN_EXTEND_INREG SDNode.
When this SDNode is legalized during the LegalizeVector phase, it is
scalarized because non-simple types are automatically marked to be expanded.
In this patch we add support for lowering SIGN_EXTEND_INREG manually.
This fixes CodeGen/X86/vec_sext.ll when running with the '-promote-elements'
flag.

llvm-svn: 135144
2011-07-14 11:11:14 +00:00
Nadav Rotem e7a101ccab Add support for legalizing UINT_TO_FP of vectors on platforms which do
not have native support for this operation (such as X86).
The legalized code uses two vector INT_TO_FP operations and is faster
than scalarizing.

llvm-svn: 127951
2011-03-19 13:09:10 +00:00
Wesley Peck 527da1b6e2 Renaming ISD::BIT_CONVERT to ISD::BITCAST to better reflect the LLVM IR concept.
llvm-svn: 119990
2010-11-23 03:31:01 +00:00
Dan Gohman 92c11acdb8 Change UpdateNodeOperands' operand and return value from SDValue to
SDNode *, since it doesn't care about the ResNo value.

llvm-svn: 106282
2010-06-18 15:30:29 +00:00
Dan Gohman 21cea8ac2e Use const qualifiers with TargetLowering. This eliminates several
const_casts, and it reinforces the design of the Target classes being
immutable.

SelectionDAGISel::IsLegalToFold is now a static member function, because
PIC16 uses it in an unconventional way. There is more room for API
cleanup here.

And PIC16's AsmPrinter no longer uses TargetLowering.

llvm-svn: 101635
2010-04-17 15:26:15 +00:00
Dan Gohman 6bd3ef82ff Revert an earlier change to SIGN_EXTEND_INREG for vectors. The VTSDNode
really does need to be a vector type, because
TargetLowering::getOperationAction for SIGN_EXTEND_INREG uses that type,
and it needs to be able to distinguish between vectors and scalars.

Also, fix some more issues with legalization of vector casts.

llvm-svn: 93043
2010-01-09 02:13:55 +00:00
Dan Gohman 1d459e4937 Implement vector widening, splitting, and scalarizing for SIGN_EXTEND_INREG.
llvm-svn: 91158
2009-12-11 21:31:27 +00:00
Chris Lattner a48f44d9ee improve portability to avoid conflicting with std::next in c++'0x.
Patch by Howard Hinnant!

llvm-svn: 90365
2009-12-03 00:50:42 +00:00
Mon P Wang 32f8bb9ed4 Added support to allow clients to custom widen. For X86, custom widen vectors for
divide/remainder since these operations can trap by unroll them and adding undefs
for the resulting vector.

llvm-svn: 90108
2009-11-30 02:42:02 +00:00
Owen Anderson 9f94459d24 Split EVT into MVT and EVT, the former representing _just_ a primitive type, while
the latter is capable of representing either a primitive or an extended type.

llvm-svn: 78713
2009-08-11 20:47:22 +00:00
Owen Anderson 53aa7a960c Rename MVT to EVT, in preparation for splitting SimpleValueType out into its own struct type.
llvm-svn: 78610
2009-08-10 22:56:29 +00:00
Dan Gohman f9bbcd1afd Major calling convention code refactoring.
Instead of awkwardly encoding calling-convention information with ISD::CALL,
ISD::FORMAL_ARGUMENTS, ISD::RET, and ISD::ARG_FLAGS nodes, TargetLowering
provides three virtual functions for targets to override:
LowerFormalArguments, LowerCall, and LowerRet, which replace the custom
lowering done on the special nodes. They provide the same information, but
in a more immediately usable format.

This also reworks much of the target-independent tail call logic. The
decision of whether or not to perform a tail call is now cleanly split
between target-independent portions, and the target dependent portion
in IsEligibleForTailCallOptimization.

This also synchronizes all in-tree targets, to help enable future
refactoring and feature work.

llvm-svn: 78142
2009-08-05 01:29:28 +00:00
Eli Friedman aea9b65668 Make SINT_TO_FP/UINT_TO_FP vector legalization queries query on the
integer type to be consistent with normal operation legalization.  No visible
change because nothing is actually using this at the moment.

llvm-svn: 72980
2009-06-06 03:27:50 +00:00
Eli Friedman 3b251705fd Remove special cases for more opcodes.
llvm-svn: 72467
2009-05-27 07:58:35 +00:00
Eli Friedman ac149ee60a Add a comment which should hopefully make the purpose of this method a
bit clearer.

llvm-svn: 72374
2009-05-24 20:32:10 +00:00
Eli Friedman da90dd6d72 Add a new step to legalization to legalize vector math operations. This
will allow simplifying LegalizeDAG to eliminate type legalization.  (I 
have a patch to do that, but it's not quite finished; I'll commit it 
once it's finished and I've fixed any review comments for this patch.)  
See the comment at the beginning of 
lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp for more details on the
motivation for this patch.

llvm-svn: 72325
2009-05-23 12:35:30 +00:00