Previously, all children would be checked to see if any were an
explicit Register. If anywhere no commutable patterns would be
generated. This patch loosens the restriction to only check the
children that are being commuted.
Digging back through history, this code predates the existence of
commutable intrinsics and commutable SDNodes with more than 2
operands. At that time the loop would count the number of children that
weren't registers and if that was equal to 2 it would allow commuting.
I don't think this loop was re-considered when commutable
intrinsics were added or when we allowed SDNodes with more than 2
operands.
This important for RISCV were our isel patterns have a V0 mask
operand after the commutable operands on some RISCVISD opcodes.
Reviewed By: arsenm
Differential Revision: https://reviews.llvm.org/D117955
Since 65b13610a5, raw_string_ostream has
been unbuffered by default. Based on an audit of llvm/utils/, this
commit removes every call to `raw_string_ostream::flush()` and any call
to `raw_string_ostream::str()` whose result is ignored or that doesn't
help with clarity.
I left behind a few calls to `str()`. In these cases, the underlying
std::string was declared pretty far away and never used again, whereas
stream recently had its last write. The code is easier to read as-is;
the no-op call to `flush()` inside `str()` isn't harmful, and when
https://reviews.llvm.org/D115421 lands it'll be gone anyway.
When used as a non-leaf node, TableGen does not currently use the type
of a ComplexPattern for type inference, which also means it does not
check it doesn't conflict with the use. This differs from when used as a
leaf value, where the type is used for inference. This addresses that
discrepancy. The test case is not representative of most real-world uses
but is sufficient to demonstrate inference is working.
Some of these uses also make use of ValueTypeByHwMode rather than
SimpleValueType and so the existing type inference is extended to
support that alongside the new type inference.
There are also currently various cases of using ComplexPatterns with an
untyped type, but only for non-leaf nodes. For compatibility this is
permitted, and uses the old behaviour of not inferring for non-leaf
nodes, but the existing logic is still used for leaf values. This
remaining discrepancy should eventually be eliminated, either by
removing all such uses of untyped so the special case goes away (I
imagine Any, or a more specific type in certain cases, would be
perfectly sufficient), or by copying it to the leaf value case so
they're consistent with one another if this is something that does need
to keep being supported.
All non-experimental targets have been verified to produce bit-for-bit
identical TableGen output with this change applied.
Reviewed By: kparzysz
Differential Revision: https://reviews.llvm.org/D109035
The operand of the second any_of in EnforceSmallerThan should be
B not S like the FP code in the if below.
Unfortunately, fixing that causes an infinite loop in the build
of RISCV. So I've added a workaround for that as well.
Fixes PR44768.
Reviewed By: RKSimon
Differential Revision: https://reviews.llvm.org/D111502
This gives a nice message about the location of errors in a large
tablegen file, which is much more useful for users
Differential Revision: https://reviews.llvm.org/D102740
If a cmpxchg specifies acquire or seq_cst on failure, make sure we
generate code consistent with that ordering even if the success ordering
is not acquire/seq_cst.
At one point, it was ambiguous whether this sort of construct was valid,
but the C++ standad and LLVM now accept arbitrary combinations of
success/failure orderings.
This doesn't address the corresponding issue in AtomicExpand. (This was
reported as https://bugs.llvm.org/show_bug.cgi?id=33332 .)
Fixes https://bugs.llvm.org/show_bug.cgi?id=50512.
Differential Revision: https://reviews.llvm.org/D103284
getVectorNumElements() returns a value for scalable vectors
without any warning so it is effectively getVectorMinNumElements().
By renaming it and making getVectorNumElements() forward to
it, we can insert a check for scalable vectors into getVectorNumElements()
similar to EVT. I didn't do that in this patch because there are still more
fixes needed, but I was able to temporarily do it and passed the RISCV
lit tests with these changes.
The changes to isPow2VectorType and getPow2VectorType are copied from EVT.
The change to TypeInfer::EnforceSameNumElts reduces the size of AArch64's isel table.
We're now considering SameNumElts to require the scalable property to match which
removes some unneeded type checks.
This was motivated by the bug I fixed yesterday in 80b9510806
Reviewed By: frasercrmck, sdesmalen
Differential Revision: https://reviews.llvm.org/D102262
After D100691, predicates should be cheap to compare again so
we don't need to filter anymore.
This is mostly just a revert of several patches going back to 2018.
Reviewed By: kparzysz
Differential Revision: https://reviews.llvm.org/D100695
This uses to be how predicates were handled prior to HwMode being
added. When the Predicates were converted to a std::vector it
significantly increased the cost of a compare in GenerateVariants.
Since ListInit's are uniquified by tablegen, we can use a simple
pointer comparison to check for identical lists.
In order to store the HwMode, we now add a separate string to
PatternToMatch. This will be appended separately to the predicate
string in getPredicateCheck. A new getPredicateRecords is added
to allow GlobalISel and getPredicateCheck to both get the sorted
list of Records. GlobalISel was ignoring any HwMode predicates
before and still is.
There is one slight change here, ListInits with different predicate
orders aren't sorted so the filtering in GenerateVariants might
fail to detect two isomorphic patterns with different predicate
orders. This doesn't seem to be happening in tree today.
My hope is this will allow us to remove all the BitVector tracking
in GenerateVariants that was making up for predicates beeing
expensive to compare. There's a decent amount of heap allocations
there on large targets like X86, AMDGPU, and RISCV.
Differential Revision: https://reviews.llvm.org/D100691
As discussed in D100691 and based on D100889.
I removed the ModeChecks cache which provides little value. Reduced
from three loops to two. Used ArrayRef to pass the Predicate to
AppendPattern to avoid needing to construct a vector for single
mode. Used SmallVector to avoid heap allocation constructing
DefaultCheck for the in tree targets the use it.
Reviewed By: kparzysz
Differential Revision: https://reviews.llvm.org/D101240
The key here is HwMode indices. They're going to be small numbers,
contiguous, and only a few different values. I don't think we need
to go through the SmallDenseSet hashing.
A BitVector would be even better, but we don't have the upper
bound here.
A large portion of the patterns are duplicated for HwMode on RISCV.
If we expand HwMode first, we need to check nearly twice as many
patterns for variants. HwModes shouldn't affect whether a variant
is valid so we should be able to expand after.
This also reduces the RISCV isel table by 539 bytes due to factoring
working better on this pattern order. Unfortunately it increases
Hexagon table size by ~50 bytes. But I think this is a reasonable
trade.
I have seen this error quite frequently in our out-of-tree CHERI backends
and the lack of location information sometimes makes it quite difficult
to track down the actual source of the error.
This patch changes the llvm_unreachable() to a PrintFatalError() so that
tablegen prints a stack of source locations.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D99468
I've left mask registers to a future patch as we'll need
to convert them to full vectors, shuffle, and then truncate.
Reviewed By: frasercrmck
Differential Revision: https://reviews.llvm.org/D97609
This ensures that we'll match immediates consistently regardless
of whether we match them as a standalone splat or as part of
another operation.
While I was there I added complexities to the simm5/uimm5 patterns so
we didn't have to assume that the 1 on the non-immediate was lower
than what tablegen inferred.
I had to make a minor tweak to tablegen to fix one place that
didn't expect to see a ComplexPattern that wasn't a "leaf".
Reviewed By: frasercrmck
Differential Revision: https://reviews.llvm.org/D96199
This primarily occurs with isel patterns using vnot. This reduces
the number of variants in the isel tables.
We generally canonicalize build_vectors of constants to the RHS. I think
we might fail if there is a bitcast on the build_vector, but that
should be easy to fix if we can find a case. Usually the
bitcast is introduced by type legalization or lowering. It's
likely canonicalization would have already occured.
We already used emplace_back in at least one other place so be
consistent.
AddPatternToMatch already took PTM as an rvalue reference, but
we need to use std::move again to move it into the PatternToMatch
vector.
Use vector::swap instead of copying to a local vector and clearing
the original. We can just swap into the just created local vector
instead which will move the pointers and not the data.
Use std::move in another place to avoid a copy.
In CodeGenDAGPatterns.cpp we were relying upon TypeSize comparison
operators for ordering types, when we can actually just use the known
minimum size since the scalable property is already being taken into
account. Also, in TypeInfer::EnforceSameSize I fixed some implicit
TypeSize->uint64_t casts by changing the code to test the equality
of TypeSize objects instead.
In other places I have replaced calls to getSizeInBits() with
getFixedSizeInBits() because we are only ever expecting integer values.
Differential Revision: https://reviews.llvm.org/D88947
This is very similar to 243970d03cace2, but handling a slightly
different form of predicated operations. When starting with a pattern of
the form select(p, BinOp(x, y), x), Instcombine will often transform
this to BinOp(x, select(p, y, 0)), where 0 is the identity value of the
binop (0 for adds/subs, 1 for muls, -1 for ands etc). This adds the
patterns that transforms those back into predicated binary operations.
There is also a very minor adjustment to tablegen null_frag in here, to
allow it to also be recognized as a PatLeaf node, so that it can be used
in MVE_TwoOpPattern to easily exclude the cases where we do not need the
alternate transform.
Differential Revision: https://reviews.llvm.org/D84091
This was checking for default operands in the current DAG instruction,
rather than the correct result operand list. I'm not entirly sure how
this managed to work before, but was failing for me when multiple
default operands were overridden.
Summary:
In the DAG pattern backend, `SimplifyTree` simplifies a pattern by
removing bitconverts between two identical types. But that function is
also run on the fragments list in instances of `PatFrags`, in which
the types haven't been specified yet. So the input and output of the
bitconvert always evaluate to the empty set of types, which makes them
compare equal. So the test always passes, and bitconverts are
unconditionally removed from the PatFrag RHS.
Fixed by spotting the empty type set and using it to inhibit the
optimization.
Reviewers: nhaehnle, hfinkel
Reviewed By: nhaehnle
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D74627
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
This assumed a single pattern if there was a predicate. Relax this a
bit, and allow multiple patterns as long as they have the same class.
This was only broken for the DAG path. GlobalISel seems to have
handled this correctly already.
AMDGPU was the last in tree target to use this tablegen mode. I plan to
split up the global intrinsic enum similar to the way that clang
diagnostics are split up today. I don't plan to build on this mode.
Reviewers: arsenm, echristo, efriedma
Reviewed By: echristo
Differential Revision: https://reviews.llvm.org/D71318
* Implements scalable size queries for MVTs, split out from D53137.
* Contains a fix for FindMemType to avoid using scalable vector type
to contain non-scalable types.
* Explicit casts for several places where implicit integer sign
changes or promotion from 32 to 64 bits caused problems.
* CodeGenDAGPatterns will treat scalable and non-scalable vector types
as different.
Reviewers: greened, cameron.mcinally, sdesmalen, rovka
Reviewed By: rovka
Differential Revision: https://reviews.llvm.org/D66871
Craig Topper