Summary:
This patch introduces an alternative calling convention for
MemRef function arguments in LLVM dialect. It converts MemRef
function arguments to LLVM bare pointers to the MemRef element
type instead of creating a MemRef descriptor. Bare pointers are
then promoted to a MemRef descriptors at the beginning of the
function. This calling convention is only enabled with a flag.
Reviewers: ftynse, bondhugula, nicolasvasilache, rriddle, mehdi_amini
Reviewed By: ftynse, rriddle, mehdi_amini
Subscribers: Joonsoo, flaub, merge_guards_bot, jholewinski, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, csigg, arpith-jacob, mgester, lucyrfox, herhut, aartbik, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D72802
This revision does the following post-commit cleanups:
1. don't use -1 magic constants,
2. drop commented out old test that does not belong here,
3. reformat and add a proper clang-format off on a CHECK directive.
Summary:
Rationale:
When lowering to LLVM for different rank insert (n vs k), the offset
arrays needs to drop one dimension (becomes n-1), but the strides
array needs to be preserved (remains k). With regression test.
Note that this example was actually in the documentation, so
extra important to do it right :-)
Reviewers: nicolasvasilache, andydavis1, ftynse
Reviewed By: nicolasvasilache, ftynse
Subscribers: Joonsoo, merge_guards_bot, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73733
Summary:
After the `subview` operation was migrated from Linalg to Standard, it changed
semantics and does not guarantee the absence of out-of-bounds accesses through
the created view anymore. Compute the size of the subview to make sure it
always fits within the view (subviews in last iterations of the loops may be
smaller than those in other iterations).
Differential Revision: https://reviews.llvm.org/D73614
This commit adds a pattern to lower linalg.generic for reduction
to spv.GroupNonUniform* ops. Right now this only supports integer
reduction on 1-D input memref. Shader entry point ABI is queried
to make sure that the input memref's shape matches the local
workgroup's invocation configuration. This makes sure that the
workload fits in one local workgroup so that we can leverage
SPIR-V group non-uniform operations.
linglg.generic is a structured op that preserves the right level
of information. It is easier to recognize reduction at this level
than performing analysis on loops.
This commit also exposes `getElementPtr` in SPIRVLowering.h given
that it's a generally useful utility function.
Differential Revision: https://reviews.llvm.org/D73437
Summary:
The current code assumes that one always maps at least one loop to block
dimensions and at least one loop to thread dimensions. If either is not
the case, a loop would get mapped twice.
Differential Revision: https://reviews.llvm.org/D73685
Summary:
This revision switches over many operations to use the declarative methods for defining the assembly specification. This updates operations in the NVVM, ROCDL, Standard, and VectorOps dialects.
Differential Revision: https://reviews.llvm.org/D73407
Summary:
This revision add support, and testing, for generating the parser and printer from the declarative operation format.
Differential Revision: https://reviews.llvm.org/D73406
Summary:
This is the first revision in a series that adds support for declaratively specifying the asm format of an operation. This revision
focuses solely on parsing the format. Future revisions will add support for generating the proper parser/printer, as well as
transitioning the syntax definition of many existing operations.
This was originally proposed here:
https://llvm.discourse.group/t/rfc-declarative-op-assembly-format/340
Differential Revision: https://reviews.llvm.org/D73405
Summary:
In the scope of the lowering phase from GPU to ROCDL, the intructions for the conversion patterns seems to be wrong.
According to https://github.com/ROCm-Developer-Tools/HIP/blob/master/include/hip/hcc_detail/math_fwd.h the instructions need two underscores in the beginning instead of one.
Reviewers: nicolasvasilache, herhut, rriddle
Reviewed By: herhut, rriddle
Subscribers: merge_guards_bot, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, csigg, arpith-jacob, mgester, lucyrfox, herhut, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73535
Summary:
The 'gpu.terminator' operation is used as the terminator for the
regions of gpu.launch. This is to disambugaute them from the
return operation on 'gpu.func' functions.
This is a breaking change and users of the gpu dialect will need
to adapt their code when producting 'gpu.launch' operations.
Reviewers: nicolasvasilache
Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, csigg, arpith-jacob, mgester, lucyrfox, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73620
Summary:
Canonicalization and folding patterns in StandardOps may interfere with the needs
of Linalg. This revision introduces specific foldings for dynamic memrefs that can
be proven to be static.
Very concretely:
Determines whether it is possible to fold it away in the parent Linalg op:
```mlir
%1 = memref_cast %0 : memref<8x16xf32> to memref<?x?xf32>
%2 = linalg.slice %1 ... : memref<?x?xf32> ...
// or
%1 = memref_cast %0 : memref<8x16xf32, affine_map<(i, j)->(16 * i + j)>>
to memref<?x?xf32>
linalg.generic(%1 ...) : memref<?x?xf32> ...
```
into
```mlir
%2 = linalg.slice %0 ... : memref<8x16xf32> ...
// or
linalg.generic(%0 ... : memref<8x16xf32, affine_map<(i, j)->(16 * i + j)>>
```
Reviewers: ftynse, aartbik, jsetoain, tetuante, asaadaldien
Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73565
Summary:
Barrier is a simple operation that takes no arguments and returns
nothing, but implies a side effect (synchronization of all threads)
Reviewers: jdoerfert
Subscribers: mgorny, guansong, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D72400
Summary:
This diff adds a transformation patter to rewrite linalg.fill as broadcasting a scaler into a vector.
It uses the same preconditioning as matmul (memory is contiguous).
Reviewers: nicolasvasilache
Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73391
Summary:
This also removes the explicit pattern for loop.terminator to ensure
that the terminator is only erased if the parent op is rewritten.
Reductions are not yet supported.
Reviewers: nicolasvasilache
Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73348
Summary:
The intrinsic operation added multiple type annotations to the llvm intrinsic operations, but only one is needed.
The related tests in llvmir-intrinsics.mlir checked the wrong number and are adjusted as well.
Reviewers: nicolasvasilache, ftynse
Reviewed By: ftynse
Subscribers: merge_guards_bot, ftynse, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73470
Summary:
The tanh lowering from Standard dialect to NVVM and ROCDL was not working.
The conversion pattern are inserted in the lowering files.
The test cases for the lowerings were added in the test files.
Reviewers: nicolasvasilache, ftynse, herhut
Reviewed By: ftynse, herhut
Subscribers: merge_guards_bot, ftynse, jholewinski, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, csigg, arpith-jacob, mgester, lucyrfox, herhut, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73471
Summary:
The dead function elimination pass in toy was a temporary stopgap until we had proper dead function elimination support in MLIR. Now that this functionality is available, this pass is no longer necessary.
Differential Revision: https://reviews.llvm.org/D72483
Summary: This pass deletes all symbols that are found to be unreachable. This is done by computing the set of operations that are known to be live, propagating that liveness to other symbols, and then deleting all symbols that are not within this live set.
Differential Revision: https://reviews.llvm.org/D72482
Summary: This revision refactors the implementation of the symbol use-list functionality to be a bit cleaner, as well as easier to reason about. Aside from code cleanup, this revision updates the user contract to never recurse into operations if they define a symbol table. The current functionality, which does recurse, makes it difficult to examine the uses held by a symbol table itself. Moving forward users may provide a specific region to examine for uses instead.
Differential Revision: https://reviews.llvm.org/D73427
Summary: The new internal representation of operation results now allows for accessing the result types to be more efficient. Changing the API to ArrayRef is more efficient and removes the need to explicitly materialize vectors in several places.
Differential Revision: https://reviews.llvm.org/D73429
Summary: This allows for providing a default "catchall" legality check that is not dependent on specific operations or dialects. For example, this can be useful to check legality based on the specific types of operation operands or results.
Differential Revision: https://reviews.llvm.org/D73379
Summary:
Affine minimum computation will be used in tiling transformation. The
implementation is mostly boilerplate as we already lower the minimum in the
upper bound of an affine loop.
Differential Revision: https://reviews.llvm.org/D73488
Summary:
Implement the handling of llvm::ConstantDataSequential and
llvm::ConstantAggregate for (nested) array and vector types when imporitng LLVM
IR to MLIR. In all cases, the result is a DenseElementsAttr that can be used in
either a `llvm.mlir.global` or a `llvm.mlir.constant`. Nested aggregates are
unpacked recursively until an element or a constant data is found. Nested
arrays with innermost scalar type are represented as DenseElementsAttr of
tensor type. Nested arrays with innermost vector type are represented as
DenseElementsAttr with (multidimensional) vector type.
Constant aggregates of struct type are not yet supported as the LLVM dialect
does not have a well-defined way of modeling struct-type constants.
Differential Revision: https://reviews.llvm.org/D72834
Thus far certain SPIR-V ops have been required to be in spv.module.
While this provides strong verification to catch unexpected errors,
it's quite rigid and makes progressive lowering difficult. Sometimes
we would like to partially lower ops from other dialects, which may
involve creating ops like global variables that should be placed in
other module-like ops. So this commit relaxes the requirement of
such SPIR-V ops' scope to module-like ops. Similarly for function-
like ops.
Differential Revision: https://reviews.llvm.org/D73415
Summary:
Rewrites the extract/insert_slices operation in terms of
strided_slice/insert_strided_slice ops with intermediate
tuple uses (that should get optimimized away with typical
usage). This is done in a separate "pass" to enable testing
this particular rewriting in isolation.
Reviewers: nicolasvasilache, andydavis1, ftynse
Reviewed By: nicolasvasilache
Subscribers: merge_guards_bot, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73295
Summary:
LLVMIRIntrinsicGen is using LLVM_Op as the base class for intrinsics.
This works for LLVM intrinsics in the LLVM Dialect, but when we are
trying to convert custom intrinsics that originate from a custom
LLVM dialect (like NVVM or ROCDL) these usually have a different
"cppNamespace" that needs to be applied to these dialect.
These dialect specific characteristics (like "cppNamespace")
are typically organized by creating a custom op (like NVVM_Op or
ROCDL_Op) that passes the correct dialect to the LLVM_OpBase class.
It seems natural to allow LLVMIRIntrinsicGen to take that into
consideration when generating the conversion code from one of these
dialect to a set of target specific intrinsics.
Reviewers: rriddle, andydavis1, antiagainst, nicolasvasilache, ftynse
Subscribers: jdoerfert, mehdi_amini, jpienaar, burmako, shauheen, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73233
Summary:
This diff extends the Linalg EDSC builders so we can easily create mixed
tensor/buffer linalg.generic ops. This is expected to be useful for
HLO -> Linalg lowering.
The StructuredIndexed struct is made to derive from ValueHandle and can
now capture a type + indexing expressions. This is used to represent return
tensors.
Pointwise unary and binary builders are extended to allow both output buffers
and return tensors. This has implications on the number of region arguments.
Reviewers: ftynse, hanchung, asaadaldien
Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73149
Summary:
llvm::to_vector() accepts a Range value and not the pair of arguments
we are currently passing. Also we probably want the lowered LLVM
values in the vector, while operand_begin()/operand_end() on MLIR ops
returns MLIR types. lookupValues() seems the correct way to collect
such values.
Reviewers: rriddle, andydavis1, antiagainst, nicolasvasilache, ftynse
Subscribers: jdoerfert, mehdi_amini, jpienaar, burmako, shauheen, arpith-jacob, mgester, lucyrfox, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D73137
Add lowering for constant operation with ranked tensor type to
spv.constant with spv.array type.
Differential Revision: https://reviews.llvm.org/D73022
Summary:
Add method in ODS to specify verification for operations implementing a
OpInterface. Use this with infer type op interface to verify that the
inferred type matches the return type and remove special case in
TestPatterns.
This could also have been achieved by using OpInterfaceMethod but verify
seems pretty common and it is not an arbitrary method that just happened
to be named verifyTrait, so having it be defined in special way seems
appropriate/better documenting.
Differential Revision: https://reviews.llvm.org/D73122
Summary:
This diff extends the Linalg EDSC builders so we can easily create mixed
tensor/buffer linalg.generic ops. This is expected to be useful for
HLO -> Linalg lowering.
The `StructuredIndexed` struct is made to derive from `ValueHandle` and can
now capture a type + indexing expressions. This is used to represent return
tensors.
Pointwise unary and binary builders are extended to allow both output buffers
and return tensors. This has implications on the number of region arguments.
Reviewers: ftynse, herhut, hanchung, asaadaldien, stellaraccident
Reviewed By: asaadaldien
Subscribers: merge_guards_bot, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D72863
Summary:
If an intrinsic has overloadable types like llvm_anyint_ty or
llvm_anyfloat_ty then to getDeclaration() we need to pass a list
of the types that are "undefined" essentially concretizing them.
This patch add support for deriving such types from the MLIR op
that has been matched.
Reviewers: andydavis1, ftynse, nicolasvasilache, antiagainst
Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, arpith-jacob, mgester, lucyrfox, liufengdb, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D72974
For the generated builder taking in unwrapped attribute values,
if the argument is a string, we should avoid wrapping it in quotes;
otherwise we are always setting the string attribute to contain
the string argument's name. The quotes come from StrinAttr's
`constBuilderCall`, which is reasonable for string literals, but
not function arguments containing strings.
Differential Revision: https://reviews.llvm.org/D72977
Summary:
This is based on the use of code constantly checking for an attribute on
a model and instead represents the distinct operaion with a different
op. Instead, this op can be used to provide better filtering.
Reverts "Revert "[mlir] Create a gpu.module operation for the GPU Dialect.""
This reverts commit ac446302ca4145cdc89f377c0c364c29ee303be5 after
fixing internal Google issues.
This additionally updates ROCDL lowering to use the new gpu.module.
Reviewers: herhut, mravishankar, antiagainst, nicolasvasilache
Subscribers: jholewinski, mgorny, mehdi_amini, jpienaar, burmako, shauheen, csigg, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits, mravishankar, rriddle, antiagainst, bkramer
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D72921
Nicolas Vasilache