Summary:
Introduce a new function attribute, amdgpu-no-multigrid-sync-arg, which is default.
We use implicitarg_ptr + offset to check whether the multigrid synchronization
pointer is used. If yes, we remove this attribute and also remove
amdgpu-no-implicitarg-ptr. We generate metadata for the hidden_multigrid_sync_arg
only when the amdgpu-no-multigrid-sync-arg attribute is removed from the function.
Reviewers: arsenm, sameerds, b-sumner and foad
Differential Revision: https://reviews.llvm.org/D123548
This change generalizes the fusion of `tensor.expand_shape` ->
`linalg.generic` op by collapsing to handle cases where only a subset
of the reassociations specified in the `tensor.expand_shape` are valid
to be collapsed.
The method that does the collapsing is refactored to allow it to be a
generic utility when required.
Reviewed By: gysit
Differential Revision: https://reviews.llvm.org/D123153
We're failing to vectorize several comparison reduction patterns.
Issue #43090 was based off this, but while that simplified test case is now folding, the original still fails due to poor cost model values for vXi1 extractions
IRLinker builds a work list of functions to materialize, then moves them
from a source module to a destination module one at a time.
This is a problem for blockaddress Constants, since they need not refer
to the function they are used in; IPSCCP is quite good at sinking these
constants deep into other functions when passed as arguments.
This would lead to curious errors during LTO:
ld.lld: error: Never resolved function from blockaddress ...
based on the ordering of function definitions in IR.
The problem was that IRLinker would basically do:
for function f in worklist:
materialize f
splice f from source module to destination module
in one pass, with Functions being lazily added to the running worklist.
This confuses BitcodeReader, which cannot disambiguate whether a
blockaddress is referring to a function which has not yet been parsed
("materialized") or is simply empty because its body was spliced out.
This causes BitcodeReader to insert Functions into its BasicBlockFwdRefs
list incorrectly, as it will never re-materialize an already
materialized (but spliced out) function.
Because of the possibility that blockaddress Constants may appear in
Functions other than the ones they reference, this patch adds a new
bitcode function code FUNC_CODE_BLOCKADDR_USERS that is a simple list of
Functions that contain BlockAddress Constants that refer back to this
Function, rather then the Function they are scoped in. We then
materialize those functions when materializing `f` from the example loop
above. This might over-materialize Functions should the user of
BitcodeReader ultimately decide not to link those Functions, but we can
at least now we can avoid this ordering related issue with blockaddresses.
Fixes: https://github.com/llvm/llvm-project/issues/52787
Fixes: https://github.com/ClangBuiltLinux/linux/issues/1215
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D120781
This patch adds tasking construct according to Section 2.10.1 of OpenMP 5.0
Reviewed By: peixin, kiranchandramohan, abidmalikwaterloo
Differential Revision: https://reviews.llvm.org/D123575
ubsan_GetStackTrace (from 52b751088b) called by
~ScopeReport leaves top/bottom zeroes in the
`!WillUseFastUnwind(request_fast_unwind)` code path.
When BufferedStackTrace::Unwind falls back to UnwindFast,
`if (stack_top < 4096) return;` will return early, leaving just one frame in the stack trace.
Fix this by always initializing top/bottom like 261d6e05d5.
Reviewed By: eugenis, yln
Differential Revision: https://reviews.llvm.org/D123562
When we build the library with the stable ABI, we need to include some
functions in the dylib that were made inline in later versions of the
library (to avoid breaking code that might be relying on those symbols).
However, those methods were made non-inline whenever we'd be building
the library, which means that all translation units would end up using
the old out-of-line definition of these methods, as opposed to the new
inlined version. This patch makes it so that only the translation units
that actually define the out-of-line methods use the old definition,
opening up potential optimization opportunities in other translation
units.
This should solve some of the issues encountered in D65667.
Differential Revision: https://reviews.llvm.org/D123519
The LOH pass iterates over instructions to build its custom register
state machine, but it uses the top-level bundle iterator.
This should be okay, because when the wrapper BUNDLE MI is built,
it aggregates the register defs/uses in its instructions into MOs.
However, that doesn't apply to regmasks, and accumulating regmasks
across multiple instructions would be messy business.
There are a couple AnalyzePhysRegInBundle (/Virt) helpers that
do look at regmasks, but those don't fit in very well here.
AArch64 has started to use a few bundle instructions, specifically
as glorified pseudos for variant call instructions, which have regmasks.
So the LOH pass ends up ignoring regmasks.
Concretely, this has been wrong for a while, but, on aarch64, the
most common bundle (rv_marker call) was always followed by the
attached call instruction, a plain BL with a regmask. Which
was properly detected by the pass.
However, we recently started keeping the attached call in the bundle,
so the regmask is now ignored. And the pass happily combines ADRPs, of
say, x8, across the bundle, resulting in corrupt pointers later.
This reverts the functional changes of D103427 but keeps its tests, and
and reimplements the functionality by reusing the existing 32-bit
MASKMOVDQU and VMASKMOVDQU instructions as suggested by skan in review.
These instructions were previously predicated on Not64BitMode. This
reimplementation restores the disassembly of a class of instructions,
which will see a test added in followup patch D122449.
These instructions are in 64-bit mode special cased in
X86MCInstLower::Lower, because we use flags with one meaning for subtly
different things: we have an AdSize32 class which indicates both that
the instruction needs a 0x67 prefix and that the text form of the
instruction implies a 0x67 prefix. These instructions are special in
needing a 0x67 prefix but having a text form that does *not* imply a
0x67 prefix, so we encode this in MCInst as an instruction that has an
explicit address size override.
Note that originally VMASKMOVDQU64 was special cased to be excluded from
disassembly, as we cannot distinguish between VMASKMOVDQU and
VMASKMOVDQU64 and rely on the fact that these are indistinguishable, or
close enough to it, at the MCInst level that it does not matter which we
use. Because VMASKMOVDQU now receives special casing, even though it
does not make a difference in the current implementation, as a
precaution VMASKMOVDQU is excluded from disassembly rather than
VMASKMOVDQU64.
Reviewed By: RKSimon, skan
Differential Revision: https://reviews.llvm.org/D122540
This patch removes inheritence from PresburgerSpace in IntegerRelation and
instead makes it a member of these classes.
This is required for three reasons:
- It prevents implicit casting to PresburgerSpace.
- Not all functions of PresburgerSpace need to be exposed by the deriving classes.
- IntegerRelation and IntegerPolyhedron are defined in a PresburgerSpace. It
makes more sense for the space to be a member instead of them inheriting from
a space.
Reviewed By: arjunp, ftynse
Differential Revision: https://reviews.llvm.org/D123585
Fix path replacement in sed (properly this time) using lit
regex_replacement.
Differential Revision: https://reviews.llvm.org/D123526
Co-authored-by: Michele Scandale <michele.scandale@gmail.com>
Co-authored-by: Zixu Wang <9819235+zixu-w@users.noreply.github.com>
This patch is similar to D122557, adding an `ArrayRef` version for `setOperationAction`, `setLoadExtAction`, `setCondCodeAction`, `setLibcallName`.
Reviewed By: craig.topper
Differential Revision: https://reviews.llvm.org/D123467
This renames functions for more general usage (and current capitalization style)
before a proposed logic change in D122485.
Differential Revision: https://reviews.llvm.org/D123614
In the C++20 Standard time is no longer section under utilities, but
became its own chapter. This moves the time tests accordingly so their
location matches the current Standard.
Reviewed By: ldionne, #libc
Differential Revision: https://reviews.llvm.org/D122745
Use default member initializers in AMDGPUSubtarget and subclasses. This
is to guard against adding a new feature boolean in AMDGPUSubtarget.h
but forgetting to initialize it to false in AMDGPUSubtarget.cpp.
This was mostly autogenerated by:
clang-tidy -checks=-*,cppcoreguidelines-prefer-member-initializer,modernize-use-default-member-init -header-filter=Subtarget -fix lib/Target/AMDGPU/*Subtarget.cpp
Differential Revision: https://reviews.llvm.org/D123613
Move features/bugs checks into the single place
allowsMisalignedMemoryAccessesImpl.
This is mostly NFCI except for the order of selection in couple places.
A separate change may be needed to stop lying about Fast.
Differential Revision: https://reviews.llvm.org/D123343
This means we don't have to remember to update this code as much.
This is all tested in lldb/test/Shell/Commands/command-disassemble-aarch64-extensions.s
which I added previously.
We don't have a way to get the latest base architecture yet
so that remains manual. Having all the extensions specified
will probably be equivalent to the latest architecture version
in any case.
Reviewed By: labath
Differential Revision: https://reviews.llvm.org/D123582
The description has been updated to reflect AMDGPU MC changes:
- enabled literals for src0 of v_fmaak_f*, v_fmamk_f*, v_madak_f32, v_madmk_f32;
- enabled global_atomic_fcmpswap and global_atomic_fcmpswap_x2;
- enabled dlc with flat_atomic* and global_atomic_*.
Bug fixing and improvements:
- enabled s_wait_idle;
- enabled s_waitcnt_depctr;
- added description of s_waitcnt_depctr syntactic sugar;
- disabled SYSMSG_OP_HOST_TRAP_ACK (it is not supported on GFX10);
- corrected description of lgkmcnt (accept values from 0 to 63).
It breaks arm build, there is no free bit for the extra
UsingShadowDecl in TemplateName::StorageType.
Reverting it to build the buildbot back until we comeup with a fix.
This reverts commit 5a5be4044f.
With this change, there's going to be a clear distinction between LLVM
and MLIR pass maanger options (e.g. `-mlir-print-after-all` vs
`-print-after-all`). This change is desirable from the point of view of
projects that depend on both LLVM and MLIR, e.g. Flang.
For consistency, all pass manager options in MLIR are prefixed with
`mlir-`, even options that don't have equivalents in LLVM .
Differential Revision: https://reviews.llvm.org/D123495
Lookup iter_arg buffers using `lookupBuffer` instead of always creating a new `ToMemrefOp`. Also cast all yielded buffers (if necessary), regardless of whether they are an equivalent buffer or a new allocation.
Note: This should have been part of D123369.
Differential Revision: https://reviews.llvm.org/D123383
Daniel Grumberg