When determining liveness via Attributor::isAssumedDead(...) we might
end up without a liveness AA or with one pointing into another function.
Neither is helpful and we will avoid both from now on.
This reverts commit 1fbdbb5595.
All known issues surfaced by this patch should have been fixed now.
The fixes included fixing issues with SCEV expansion in LV and DA's
reliance on LCSSA phis.
This patch adds partial support for tracking (i.e. modeling) the contents of an
optional value. Specifically, it supports tracking the value after it is
accessed. We leave tracking constructed/assigned contents to a future patch.
Differential Revision: https://reviews.llvm.org/D124932
Depends on D126560. `getKnownValue` has been changed by the parent patch
in a way that simplification was removed. This is not correct when the
function is called by the Checkers. Thus, a new internal function is
introduced, `getConstValue`, which simply queries the constraint manager.
This `getConstValue` is used internally in the `SimpleSValBuilder` when a
binop is evaluated, this way we avoid the recursion into the `Simplifier`.
Differential Revision: https://reviews.llvm.org/D127285
Clang-format InstructionSimplify and convert all "FunctionName"s to
"functionName". This patch does touch a lot of files but gets done with
the cleanup of InstructionSimplify in one commit.
This is the alternative to the less invasive clang-format only patch: D126783
Reviewed By: spatel, rengolin
Differential Revision: https://reviews.llvm.org/D126889
This should fix a number of shuffle regressions in D127115 where the re-ordered combines mean we fail to fold a EXTRACT_VECTOR_ELT/INSERT_VECTOR_ELT sequence into a BUILD_VECTOR if we extract from more than one vector source.
As with Linux placce the Counters array in the __libfuzzer_extra_counters
section. This fixes the test on FreeBSD.
Reviewed by: vitalybuka
Differential Revision: https://reviews.llvm.org/D125902
This commit allows for One-Shot Bufferize to be used through the transform dialect. No op handle is currently returned for the bufferized IR.
Differential Revision: https://reviews.llvm.org/D125098
The code expanded from kmp_barrier.h uses some `KMP_INTERNAL_*`s,
so the definitions have to be placed before it.
Fixes#55815
Differential Revision: https://reviews.llvm.org/D126873
The driver stripts the first argument. Without the compiler name, the
test depends on whether GCC_INSTALL_PREFIX is set or not.
See https://reviews.llvm.org/D125862
- Updates the image to use Ubuntu Jammy.
- Installs GCC-12 as preparation to migrate to that GCC version.
NOTE: This is a re-application of f2f0dba818, which was reverted
in 2b5e3ef83c due to an issue with the CI nodes. The CI nodes have
since then been updated and this appears to be fine.
Differential Revision: https://reviews.llvm.org/D126666
This relies on the existing TileAndFuse pattern for tensor-based structured
ops. It complements pure tiling, from which some utilities are generalized.
Depends On D127300
Reviewed By: springerm
Differential Revision: https://reviews.llvm.org/D127319
co_await and co_yield are represented by (classes derived from)
CoroutineSuspendExpr. That has a number of child nodes, not all of
which are used for code-generation. In particular the operand is
represented multiple times, and, like the problem with co_return
(55406) it must only be emitted in the CFG exactly once. The operand
also appears inside OpaqueValueExprs, but that's ok.
This adds a visitor for SuspendExprs to emit the required children in
the correct order. Note that this CFG is pre-coro xform. We don't
have initial or final suspend points.
Reviewed By: bruno
Differential Revision: https://reviews.llvm.org/D127236
The specification of gnu-debuglink can be found at:
https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html
The file CRC or the CRC value from the .gnu_debuglink section is now
used to calculate the module UUID as a fallback, to allow verifying that
the debug object does match the executable. Note that if a CodeView
build id exists, it still takes precedence. This works even for MinGW
builds because LLD writes a synthetic CodeView build id which does not
get stripped from the debug object.
The `Minidump/Windows/find-module` test also needs a fix by adding a
CodeView record to the exe to match the one in the minidump, otherwise
it fails due to the new UUID calculated from the file CRC.
Fixes https://github.com/llvm/llvm-project/issues/54344
Reviewed By: DavidSpickett
Differential Revision: https://reviews.llvm.org/D126367
The generic legalizer framework is still used to reduce the problem
to scalar multiplication with the bit size a multiple of 32.
Generating optimal code sequences for big integer multiplication is
somewhat tricky and has a number of target-specific intricacies:
- The target has V_MAD_U64_U32 instructions that multiply two 32-bit
factors and add a 64-bit accumulator. Most partial products should
use this instruction.
- The accumulator is mapped to consecutive 32-bit GPRs, and partial-
product multiply-adds can feed the accumulator into each other
directly. (The register allocator's support for that is somewhat
limited, but that only matters for 128-bit integers and larger.)
- OTOH, on some hardware, V_MAD_U64_U32 requires the accumulator
to be stored in an even-aligned pair of GPRs. To avoid excessive
register copies, it makes sense to compute odd partial products
separately from even partial products (where a partial product
src0[j0] * src1[j1] is "odd" if j0 + j1 is odd) and add both
halves together as a final step.
- We can combine G_MUL+G_ADD into a single cascade of multiply-adds.
- The target can keep many carry-bits in flight simultaneously, so
combining carries using G_UADDE is preferable over G_ZEXT + G_ADD.
- Not addressed by this patch: When the factors are sign-extended,
the V_MAD_I64_I32 instruction (signed version!) can be used.
It is difficult to address these points generically:
1) Finding matching pairs of G_MUL and G_UMULH to find a wide
multiply is expensive. We could add a G_UMUL_LOHI generic instruction
and conditionally use that in the generic legalizer, but by itself
this wouldn't allow us to use the accumulation capability of
V_MAD_U64_U32. One could attempt to find matching G_ADD + G_UADDE
post-legalization, but this is also expensive.
2) Similarly, making sense of the legalization outcome of a wide
pre-legalization G_MUL+G_ADD pair is extremely expensive.
3) How could the generic legalizer possibly deal with the
particular idiosyncracy of "odd" vs. "even" partial products.
All this points in the direction of directly emitting an ideal code
sequence during legalization, but the generic legalizer should not
be burdened with such overly target-specific concerns. Hence, a
custom legalization.
Note that the implemented approach is different from that used by
SelectionDAG because narrowing of scalars works differently in
general. SelectionDAG iteratively cuts wide scalars into low and
high halves until a legal size is reached. By contrast, GlobalISel
does the narrowing in a single shot, which should be better for
compile-time and for the quality of the generated code.
This patch leaves three gaps open:
1. When the factors are uniform, we should execute the multiplication on
the SALU. Register bank mapping already ensures this.
However, the resulting code sequence is not optimal because it doesn't
fully use the carry-in capabilities of S_ADDC_U32. (V_MAD_U64_U32
doesn't have a carry-in.) It is very difficult to fix this after the
fact, so we should really use a different legalization sequence in
this case. Unfortunately, we don't have a divergence analysis and so
cannot make that choice.
(This only matters for 128-bit integers and larger.)
2. Avoid unnecessary multiplies when sources are known to be zero- or
sign-extended. The challenge is that the legalizer does not currently
have access to GISelKnownBits.
3. When the G_MUL is followed by a G_ADD, we should consider combining
the two instructions into a single multiply-add sequence, to utilize
the accumulator of V_MAD_U64_U32 fully. (Unless the multiply has
multiple uses and the implied duplication of the multiply is an
overall negative). However, this is also not true when the factors
are uniform: in that case, it is generally better to *not* combine
the two operations, so that the multiply can be done on the SALU.
Again, we don't have a divergence analysis available and so cannot
make an informed choice.
Differential Revision: https://reviews.llvm.org/D124844
Its just as safe to move shuffles across X86ISD::ANDNP as any other logical bitop, they just tend to appear too late to matter.
Noticed while triaging D127115 regressions.
All information is already available in VPlan. Note that there are some
test changes, because we now can correctly look through instructions
like truncates to analyze the actual users.
Reviewed By: Ayal
Differential Revision: https://reviews.llvm.org/D123541
Copy hmaptool using the paths for CURRENT_TOOLS_DIR, so
everything goes in the right place in case llvm is included
from a top level CMakeLists.txt.
Signed-off-by: Matheus Izvekov <mizvekov@gmail.com>
Reviewed By: stephenneuendorffer
Differential Revision: https://reviews.llvm.org/D126308
Before commit b3a991df3c SystemHeaderMap used to be a vector.
Commit b3a991df3c changed it into a map, but neglected to remove
duplicate keys (e.g. "bits/typesizes.h", "include/stdint.h", etc.).
To prevent confusion, remove all duplicates, build HeaderMapping
one pair at a time and assert() that no duplicates are found.
Change by Paul Pluzhnikov (ppluzhnikov)!
Reviewed By: ilya-biryukov
Differential Revision: https://reviews.llvm.org/D125742
Simply add a source and target materialization handler that do nothing
and that override the default handlers that would add illegal
LLVM::DialectCastOp otherwise.
This is the simplest workaround, but not an actual fix, something may be
inconsistent after D82831 (most likely fir lowering to llvm happens in a
way that mlir infrastructure is not expecting in D82831).
Here is a minimal reproducer of what the issue was:
```
func @foop(%a : !fir.real<4>) -> ()
func @bar(%a : !fir.real<2>) {
%1 = fir.convert %a : (!fir.real<2>) -> !fir.real<4>
call @foop(%1) : (!fir.real<4>) -> ()
return
}
```
tco -o - output was:
```
error: 'llvm.mlir.cast' op type must be non-index integer types, float types, or vector of mentioned types.
llvm.func @foop(!llvm.float)
llvm.func @bar(%arg0: !llvm.half) {
%0 = llvm.fpext %arg0 : !llvm.half to !llvm.float
%1 = llvm.mlir.cast %0 : !llvm.float to !fir.real<4>
llvm.call @foop(%1) : (!fir.real<4>) -> ()
llvm.return
}
```
This patch disable the introduction of the llvm.mlir.cast and preserve the previous behavior.
Also fixes https://github.com/llvm/llvm-project/issues/55210.
Note: This is part of upstreaming from the fir-dev branch of
https://github.com/flang-compiler/f18-llvm-project.
Reviewed By: awarzynski
Differential Revision: https://reviews.llvm.org/D127212
Co-authored-by: Jean Perier <jperier@nvidia.com>
Co-authored-by: Eric Schweitz <eschweitz@nvidia.com>
Add annotation handling ([key=value]) in the BNF grammar parser, which
will be used in the conditional reduction, and error recovery.
Reviewed By: sammccall
Differential Revision: https://reviews.llvm.org/D126536
We can use constant to allow undef and there is no need to force
integers in the API anyway. The user can decide if a non integer
constant is fine or not.
We need to be careful replacing values as call site arguments
(IRPosition::IRP_CALL_SITE_ARGUMENT) is representing a use and not a
value. This patch replaces the interface to take a IR position instead
making it harder to misuse accidentally. It does not change our tests
right now but a follow up exposed the potential footgun.
We used to be very conservative when integer states were merged.
Instead of adding the known range (which is large due to uncertainty)
into the assumed range (which is hopefully small), we can also only
allow to merge in both at the same time into their respective
counterpart. This will ensure we keep the invariant that assumed is part
of known.
When we recreate instructions as part of simplification we need to take
care of debug metadata and replacing the value multiple times. For now,
we handle both conservatively.
The patch simplifies some of the patterns as below
(A | (B & C0)) | (B & C1) -> A | (B & C0|C1)
((B & C0) | A) | (B & C1) -> (B & C0|C1) | A
In some scenarios like byte reverse on half word, we can see this pattern multiple times and this conversion can optimize these patterns.
Additionally this commit fixes the issue reported with the test case.
int f(int a, int b) {
int c = ((unsigned char)(a >> 23) & 925);
if (a)
c = (a >> 23 & b) | ((unsigned char)(a >> 23) & 925) | (b >> 23 & 157);
return c;
}
The previous revision/commit did not check one-use of an intermediate value that this transform re-uses.
When that value has another use, an existing transform will try to invert the transform here.
By adding one-use checks, we avoid the infinite loops seen with the earlier commit.
Differential Revision: https://reviews.llvm.org/D124119
Johannes Doerfert