Commit Graph

6446 Commits

Author SHA1 Message Date
Uday Bondhugula 0b83a35caf [MLIR][NFC] Drop unnecessary use of OpBuilder in build trip count map
NFC. Drop unnecessary use of OpBuilder in buildTripCountMapAndOperands.
Rename this to getTripCountMapAndOperands and remove stale comments.

Differential Revision: https://reviews.llvm.org/D110993
2021-10-03 09:25:18 +05:30
Mehdi Amini 2da3facd86 Fix memory leak in MLIR SPIRV ModuleCombiner 2021-10-02 23:55:25 +00:00
Mehdi Amini 9312cb6f20 Fix Undefined Behavior in MLIR Diagnostic: don't call memcpy with a nullptr source
This happens when streaming an empty Twine as part of a diagnostic.

Differential Revision: https://reviews.llvm.org/D111002
2021-10-02 21:32:20 +00:00
Mehdi Amini db79f4a2e9 Free memory leak on duplicate interface registration
I guess this is why we should use unique_ptr as much as possible.
Also fix the InterfaceAttachmentTest.cpp test.

Differential Revision: https://reviews.llvm.org/D110984
2021-10-02 16:41:28 +00:00
wren romano af7ac1d95b [mlir][sparse] Sharing calls to adaptor.getOperands()[0]
This is preliminary work towards D110790. Depends On D110883.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D110884
2021-10-01 14:20:31 -07:00
wren romano 14fffda979 [mlir][sparse] Factoring out allocaIndices()
This is preliminary work towards D110790. Depends On D110882.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D110883
2021-10-01 14:18:56 -07:00
wren romano ca01034714 [mlir][sparse] Factoring out getZero() and avoiding unnecessary Type params
This is preliminary work towards D110790

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D110882
2021-10-01 14:17:53 -07:00
Daniel Resnick 782a97a977 [mlir][capi] Add TypeID to MLIR C-API
Exposes mlir::TypeID to the C API as MlirTypeID along with various accessors
and helper functions.

Differential Revision: https://reviews.llvm.org/D110897
2021-10-01 14:21:18 -06:00
Lei Zhang a3f425946d [mlir][linalg] Include InitTensorOp in tiling canonicalization
Tiling can create dim ops and those dim ops can take `InitTensorOp`
as input. Including it in the tiling canonicalization patterns
allows us to fold those dim ops away.

Also sorted the existing ops along the way.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D110876
2021-10-01 14:13:19 -04:00
Tobias Gysi bf28849745 [mlir][linalg] Retire PoolingMaxOp/PoolingMinOp/PoolingSumOp.
The pooling ops are among the last remaining hard coded Linalg operations that have no region attached. They got obsolete due to the OpDSL pooling operations. Removing them allows us to delete specialized code and tests that are not needed for the OpDSL counterparts that rely on the standard code paths.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110909
2021-10-01 13:51:56 +00:00
Uday Bondhugula 08b63db8bb [MLIR][GPU] Add GPU launch op support for dynamic shared memory
Add support for dynamic shared memory for GPU launch ops: add an
optional operand to gpu.launch and gpu.launch_func ops to specify the
amount of "dynamic" shared memory to use. Update lowerings to connect
this operand to the GPU runtime.

Differential Revision: https://reviews.llvm.org/D110800
2021-10-01 16:46:07 +05:30
Alexander Belyaev 693c61b2e0 [mlir] Enable loop peeling for "reduction" dimensions of tiled_loop.
Differential Revision: https://reviews.llvm.org/D110919
2021-10-01 13:07:57 +02:00
Nicolas Vasilache b016bd1230 [mlir][Linalg] Refactor comprehensive bufferize for external uses - NFC
This revision exposes some minimal funcitonality to allow comprehensive
bufferization to interop with external projects.

Differential Revision: https://reviews.llvm.org/D110875
2021-09-30 20:21:08 +00:00
wren romano 218954865e [mlir][sparse] Correcting a few typos
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D110773
2021-09-30 11:42:46 -07:00
Lei Zhang cb2e651800 [mlir][linalg] Fix incorrect bound calculation for tiling conv
For convolution, the input window dimension's access affine map
is of the form `(d0 * s0 + d1)`, where `d0`/`d1` is the output/
filter window dimension, and `s0` is the stride.

When tiling, https://reviews.llvm.org/D109267 changed how the
way dimensions are acquired. Instead of directly querying using
`*.dim` ops on the original convolution op, we now get it by
applying the access affine map to the loop upper bounds. This
is fine for dimensions having single-dimension affine maps,
like matmul, but not for convolution input. It will cause
incorrect compuation and out of bound. A concrete example, say
we have 1x225x225x3 (NHWC) input, 3x3x3x32 (HWCF) filter, and
1x112x112x3 (NHWC) output with stride 2, (112 * 2 + 3) would be
227, which is different from the correct input window dimension
size 225.

Instead, we should first calculate the max indices for each loop,
and apply the affine map to them, and then plus one to get the
dimension size. Note this makes no difference for matmul-like
ops given they will have `d0 - 1 + 1` effectively.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110849
2021-09-30 13:50:57 -04:00
Stella Laurenzo 267bb194f3 [mlir] Remove old "tc" linalg ods generator.
* This could have been removed some time ago as it only had one op left in it, which is redundant with the new approach.
* `matmul_i8_i8_i32` (the remaining op) can be trivially replaced by `matmul`, which natively supports mixed precision.

Differential Revision: https://reviews.llvm.org/D110792
2021-09-30 16:30:06 +00:00
Adrian Kuegel 68e56bd320 [mlir] Remove unused namespace alias. 2021-09-30 13:50:31 +02:00
Alex Zinenko afeda4b9ed [mlir][python] provide access to function argument/result attributes
Without this change, these attributes can only be accessed through the generic
operation attribute dictionary provided the caller knows the special operation
attribute names used for this purpose. Add some Python wrapping to support this
use case.

Also provide access to function arguments usable inside the function along with
a couple of quality-of-life improvements in using block arguments (function
arguments being the arguments of its entry block).

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D110758
2021-09-30 09:38:13 +02:00
Chris Lattner d104db531e AsmParser::getContext() - there can be only one. This should unbreak the build. 2021-09-29 22:23:03 -07:00
Chris Lattner fb093c8314 [ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser.
The former is redundant because the later carries it as part of
its builder.  Add a getContext() helper method to DialectAsmParser
to make this more convenient, and stop passing the context around
explicitly.  This simplifies ODS generated parser hooks for attrs
and types.

This resolves PR51985

Recommit 4b32f8bac4 after fixing a dependency.

Differential Revision: https://reviews.llvm.org/D110796
2021-09-30 05:10:28 +00:00
Chris Lattner 33f4315324 [AsmParser] move AsmParser::getContext to IR library.
This is (perhaps unintuitively) where the other AsmParser method
implementations are, which means that dialects don't generally need
to depend on MLIRParser directly.  This should fix a build failure
building .so files on the mlir-nvidia builder.
2021-09-29 22:07:00 -07:00
Mehdi Amini 3310e0020c Revert "[ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser."
This reverts commit 4b32f8bac4.

Seems like the build is broken with -DDBUILD_SHARED_LIBS=ON
2021-09-30 05:01:17 +00:00
Chris Lattner 4b32f8bac4 [ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser.
The former is redundant because the later carries it as part of
its builder.  Add a getContext() helper method to DialectAsmParser
to make this more convenient, and stop passing the context around
explicitly.  This simplifies ODS generated parser hooks for attrs
and types.

This resolves PR51985

Differential Revision: https://reviews.llvm.org/D110796
2021-09-29 21:36:05 -07:00
Matthias Springer 27451a05ed [mlir][vector] Fold transfer ops and tensor.extract/insert_slice.
* Fold vector.transfer_read and tensor.extract_slice.
* Fold vector.transfer_write and tensor.insert_slice.

Differential Revision: https://reviews.llvm.org/D110627
2021-09-30 09:28:00 +09:00
Rob Suderman 826d3eaae7 [mlir][tosa] Ranked check for transpose was wrong.
Should have verified the perm length and input rank were the same before
inferring shape. Caused a crash with invalid IR.

Differential Revision: https://reviews.llvm.org/D110674
2021-09-29 15:14:42 -07:00
Aart Bik 7f1cb43d60 [mlir][sparse] simplify negi code generation with subi
The lack of negi details leaked from merger class into codegen part.
Also, special case for vector code was not needed, the type can be used directly!

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D110677
2021-09-29 10:00:06 -07:00
Marcel Koester 09cd4a71ed Introduced AllocationOpInterface to create deallocation operations on-the-fly that are compatible with the allocation operation implementing this interface.
Added interface implementations for AllocOp and CloneOp defined in the MemRef diallect.
Adapted the BufferDeallocation pass to be compatible with the interface introduced in this CL.

Differential Revision: https://reviews.llvm.org/D109350
2021-09-29 15:54:21 +02:00
Nicolas Vasilache 92ea624a13 [mlir][Linalg] Rewrite CodegenStrategy to populate a pass pipeline.
This revision retires a good portion of the complexity of the codegen strategy and puts the logic behind pass logic.

Differential revision: https://reviews.llvm.org/D110678
2021-09-29 13:35:45 +00:00
Sean Silva 204d301bb1 [mlir][Python] Fix lifetime of ExecutionEngine runtime functions.
We weren't retaining the ctypes closures that the ExecutionEngine was
calling back into, leading to mysterious errors.

Open to feedback about how to test this. And an extra pair of eyes to
make sure I caught all the places that need to be aware of this.

Differential Revision: https://reviews.llvm.org/D110661
2021-09-28 22:32:20 +00:00
bakhtiyar bdde959533 Remove unnecessary async group creates and awaits.
Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D110605
2021-09-28 14:52:08 -07:00
bakhtiyar 55dfab39a2 Rename target block size to min task size for clarity.
Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D110604
2021-09-28 14:51:55 -07:00
Amy Zhuang 7ab14b8886 [mlir] Unroll-and-jam loops with iter_args.
Unroll-and-jam currently doesn't work when the loop being unroll-and-jammed
or any of its inner loops has iter_args. This patch modifies the
unroll-and-jam utility to support loops with iter_args.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D110085
2021-09-28 14:13:27 -07:00
thomasraoux b12e4c17e0 [mlir] Fix bug in FoldSubview with rank reducing subview
Fix how we calculate the new permutation map of the transfer ops.

Differential Revision: https://reviews.llvm.org/D110638
2021-09-28 13:18:29 -07:00
Alexander Belyaev 9fb57c8c1d [mlir] Add min/max operations to Standard.
[RFC: Add min/max ops](https://llvm.discourse.group/t/rfc-add-min-max-operations/4353)

I was following the naming style for Arith dialect in
https://reviews.llvm.org/D110200,
i.e. similar to DivSIOp and DivUIOp I defined MaxSIOp, MaxUIOp.

When Arith PR is landed, I will migrate these ops as well.

Differential Revision: https://reviews.llvm.org/D110540
2021-09-28 09:40:22 +02:00
Tobias Gysi d20d0e145d [mlir][linalg] Finer-grained padding control.
Adapt the signature of the PaddingValueComputationFunction callback to either return the padding value or failure to signal padding is not desired.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110572
2021-09-27 19:21:37 +00:00
Aart Bik ec97a205c3 [mlir][sparse] preserve zero-initialization for materializing buffers
This revision makes sure that when the output buffer materializes locally
(in contrast with the passing in of output tensors either in-place or not
in-place), the zero initialization assumption is preserved. This also adds
a bit more documentation on our sparse kernel assumption (viz. TACO
assumptions).

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D110442
2021-09-27 11:22:05 -07:00
Sumesh Udayakumaran b2af2aeea6 [mlir] Mode for explicitly controlling the fusion kind
New mode option that allows for either running the default fusion kind that happens today or doing either of producer-consumer or sibling fusion. This will also be helpful to minimize the compile-time of the fusion tests.

Reviewed By: bondhugula, dcaballe

Differential Revision: https://reviews.llvm.org/D110102
2021-09-27 20:37:42 +03:00
William S. Moses 6dd5b1e33e [MLIR][LLVM] Add error if using incorrect attribute type for specifying LLVM linkage
Address post-commit review in https://reviews.llvm.org/D108524 to add appropriate diagnostics.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D110566
2021-09-27 13:24:05 -04:00
Bixia Zheng fbd5821c6f Implement the conversion from sparse constant to sparse tensors.
The sparse constant provides a constant tensor in coordinate format. We first split the sparse constant into a constant tensor for indices and a constant tensor for values. We then generate a loop to fill a sparse tensor in coordinate format using the tensors for the indices and the values. Finally, we convert the sparse tensor in coordinate format to the destination sparse tensor format.

Add tests.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D110373
2021-09-27 09:47:29 -07:00
Eugene Zhulenev 92db09cde0 [mlir] AsyncRuntime: use int64_t for ref counting operations
Workaround for SystemZ ABI problem: https://bugs.llvm.org/show_bug.cgi?id=51898

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D110550
2021-09-27 07:55:01 -07:00
Tobias Gysi e158b5634a [mlir][linalg] Make fusion on tensor rewriter friendly (NFC).
Let the calling pass or pattern replace the uses of the original root operation. Internally, the tileAndFuse still replaces uses and updates operands but only of newly created operations.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110169
2021-09-27 11:28:25 +00:00
Nicolas Vasilache 1b49a72de9 [mlir] Factor out constraint set creation from hoist padding.
This revision adds a

```
FlatAffineValueConstraints(ValueRange ivs, ValueRange lbs, ValueRange ubs)
```

method and use it in hoist padding.

Differential Revision: https://reviews.llvm.org/D110427
2021-09-27 10:11:35 +00:00
Nicolas Vasilache b74493ecea [mlir][Linalg] Refactor padding hoisting - NFC
This revision extracts padding hoisting in a new file and cleans it up in prevision of future improvements and extensions.

Differential Revision: https://reviews.llvm.org/D110414
2021-09-27 09:50:31 +00:00
Matthias Springer ffdf0a370d [mlir][vector] Fix bug in vector-transfer-full-partial-split
When splitting with linalg.copy, cannot write into the destination alloc directly. Instead, write into a subview of the alloc.

Differential Revision: https://reviews.llvm.org/D110512
2021-09-27 18:12:17 +09:00
Kunwar Shaanjeet Singh Grover 0f78ece169 [MLIR] Add functionality to remove redundant local variables
This patch adds functionality to FlatAffineConstraints to remove local
variables using equalities. This helps in keeping output representation of
FlatAffineConstraints smaller.

This patch is part of a series of patches aimed at generalizing affine
dependence analysis.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D110056
2021-09-25 16:10:43 +05:30
Lei Zhang b45476c94c [mlir][tosa] Do not fold transpose with quantized types
For such cases, the type of the constant DenseElementsAttr is
different from the transpose op return type.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D110446
2021-09-24 16:57:55 -04:00
Diego Caballero 2a876a711d [mlir] Create a generic reduction detection utility
This patch introduces a generic reduction detection utility that works
across different dialecs. It is mostly a generalization of the reduction
detection algorithm in Affine. The reduction detection logic in Affine,
Linalg and SCFToOpenMP have been replaced with this new generic utility.

The utility takes some basic components of the potential reduction and
returns: 1) the reduced value, and 2) a list with the combiner operations.
The logic to match reductions involving multiple combiner operations disabled
until we can properly test it.

Reviewed By: ftynse, bondhugula, nicolasvasilache, pifon2a

Differential Revision: https://reviews.llvm.org/D110303
2021-09-24 20:45:59 +00:00
River Riddle 531206310a [mlir:OpAsm] Factor out the common bits of (Op/Dialect)Asm(Parser/Printer)
This has a few benefits:
* It allows for defining parsers/printer code blocks that
  can be shared between operations and attribute/types.
* It removes the weird duplication of generic parser/printer hooks,
  which means that newly added hooks only require touching one class.

Differential Revision: https://reviews.llvm.org/D110375
2021-09-24 20:12:19 +00:00
River Riddle aca9bea199 [mlir:MemRef] Move DmaStartOp/DmaWaitOp to ODS
These are among the last operations still defined explicitly in C++. I've
tried to keep this commit as NFC as possible, but these ops
definitely need a non-NFC cleanup at some point.

Differential Revision: https://reviews.llvm.org/D110440
2021-09-24 19:35:28 +00:00
Lei Zhang e325ebb9c7 [mlir][tosa] Add some transpose folders
* If the input is a constant splat value, we just
  need to reshape it.
* If the input is a general constant with one user,
  we can also constant fold it, without bloating
  the IR.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D110439
2021-09-24 15:25:14 -04:00