Commit Graph

5248 Commits

Author SHA1 Message Date
Matthias Springer 95f8135043 [mlir] Change vector.transfer_read/write "masked" attribute to "in_bounds".
This is in preparation for adding a new "mask" operand. The existing "masked" attribute was used to specify dimensions that may be out-of-bounds. Such transfers can be lowered to masked load/stores. The new "in_bounds" attribute is used to specify dimensions that are guaranteed to be within bounds. (Semantics is inverted.)

Differential Revision: https://reviews.llvm.org/D99639
2021-03-31 18:04:22 +09:00
Nicolas Vasilache 43b9fa3ce0 [mlir][Linalg][Python] Create the body of builtin named Linalg ops
This revision adds support to properly add the body of registered
builtin named linalg ops.
At this time, indexing_map and iterator_type support is still
missing so the op is not executable yet.

Differential Revision: https://reviews.llvm.org/D99578
2021-03-31 07:58:32 +00:00
Alexander Belyaev 465b9a4a33 Revert "Revert "[mlir] Introduce CloneOp and adapt test cases in BufferDeallocation.""
This reverts commit 883912abe6.
2021-03-31 09:49:09 +02:00
Stella Laurenzo 9a9214fa25 [mlir] Add C and python API for is_registered_operation.
* Suggested to be broken out of D99578

Differential Revision: https://reviews.llvm.org/D99638
2021-03-30 22:56:02 -07:00
Mehdi Amini a360a9786f Fix deletion of operations through the rewriter in a pattern matching a consumer operation
This allows for the conversion to match `A(B()) -> C()` with a pattern matching
`A` and marking `B` for deletion.

Also add better assertions when an operation is erased while still having uses.

Differential Revision: https://reviews.llvm.org/D99442
2021-03-30 22:02:14 +00:00
Mehdi Amini 7a4d630764 Add a "register_runtime" method to the mlir.execution_engine and show calling back from MLIR into Python
This exposes the ability to register Python functions with the JIT and
exposes them to the MLIR jitted code. The provided test case illustrates
the mechanism.

Differential Revision: https://reviews.llvm.org/D99562
2021-03-30 17:04:38 +00:00
Inho Seo f584633454 Added static verification for Linalg Ops.
This verification is to check if the indices for static shaped operands
on linalgOps access out of bound memory or not. For dynamic shaped
operands, we would be able to check it on runtime stage.

Found several invalid Linalg ops testcases, and fixed them.

Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D98390
2021-03-30 07:10:17 -07:00
MaheshRavishankar c4d5b95617 Fix broken build for commit 9b0517035f
Differential Revision: https://reviews.llvm.org/D99533
2021-03-29 12:48:45 -07:00
MaheshRavishankar 9b0517035f [mlir] Enhance InferShapedTypeOpInterface and move LinalgOps to use them.
A new `InterfaceMethod` is added to `InferShapedTypeOpInterface` that
allows an operation to return the `Value`s for each dim of its
results. It is intended for the case where the `Value` returned for
each dim is computed using the operands and operation attributes. This
interface method is for cases where the result dim of an operation can
be computed independently, and it avoids the need to aggregate all
dims of a result into a single shape value. This also implies that
this is not suitable for cases where the result type is unranked (for
which the existing interface methods is to be used).

Also added is a canonicalization pattern that uses this interface and
resolves the shapes of the output in terms of the shapes of the
inputs. Moving Linalg ops to use this interface, so that many
canonicalization patterns implemented for individual linalg ops to
achieve the same result can be removed in favor of the added
canonicalization pattern.

Differential Revision: https://reviews.llvm.org/D97887
2021-03-29 11:39:48 -07:00
Stella Laurenzo 4ca39dad52 NFC: Update MLIR python bindings docs to install deps via requirements.txt.
* Also adds some verbiage about upgrading `pip` itself, since this is a
  common source of issues.

Differential Revision: https://reviews.llvm.org/D99522
2021-03-29 18:32:51 +00:00
MaheshRavishankar f0a2fe7f79 [mlir][Linalg] Rewrite SubTensors that take a slice out of a unit-extend dimension.
Subtensor operations that are taking a slice out of a tensor that is
unit-extent along a dimension can be rewritten to drop that dimension.

Differential Revision: https://reviews.llvm.org/D99226
2021-03-29 09:19:36 -07:00
MaheshRavishankar 7d8b478ce1 [mlir][Linalg] Drop spurious error message
Drop usage of `emitRemark` and use `notifyMatchFailure` instead to
avoid unnecessary spew during compilation.

Differential Revision: https://reviews.llvm.org/D99485
2021-03-29 09:17:25 -07:00
thomasraoux 5288c25c70 [mlir][vector] Add lowering of Transfer_read with broadcast and permutation map
Convert transfer_read ops with permutation maps into simpler
transfer_read with minority map + vector.braodcast and vector.transpose.
And transfer_read with leading dimensions broacast into transfer_read of
lower rank.

Differential Revision: https://reviews.llvm.org/D99019
2021-03-29 08:38:43 -07:00
Nicolas Vasilache 2f367f34fd [mlir][Linalg] Allow calling named ops when available and make it the default.
Differential Revision: https://reviews.llvm.org/D99419
2021-03-29 13:23:11 +00:00
Frederik Gossen 630afc61a8 [MLIR][Shape] Canonicalize casted dynamic extent tensor
Differential Revision: https://reviews.llvm.org/D99161
2021-03-29 13:59:19 +02:00
Alexander Belyaev 883912abe6 Revert "[mlir] Introduce CloneOp and adapt test cases in BufferDeallocation."
This reverts commit 06b03800f3.
Until some kind of support for region args is added.
2021-03-29 12:47:59 +02:00
Julian Gross 06b03800f3 [mlir] Introduce CloneOp and adapt test cases in BufferDeallocation.
Add a new clone operation to the memref dialect. This operation implicitly
copies data from a source buffer to a new buffer. In contrast to the linalg.copy
operation, this operation does not accept a target buffer as an argument.
Instead, this operation performs a conceptual allocation which does not need to
be performed manually.

Furthermore, this operation resolves the dependency from the linalg-dialect
in the BufferDeallocation pass. In addition, we also extended the canonicalization
patterns to fold clone operations. The copy removal pass has been removed.

Differential Revision: https://reviews.llvm.org/D99172
2021-03-29 10:19:10 +02:00
KareemErgawy-TomTom c52a5f2aa7 MLIR][STD] Fold trunci (sexti).
This patch folds the following pattern:

```
%arg0 = ...
%0 = sexti %arg0 : i1 to i8
%1 = trunci %0 : i8 to i1
```

into just `%arg0`.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D99464
2021-03-29 08:34:08 +02:00
KareemErgawy-TomTom e5f2898bc7 [MLIR][STD] Fold trunci (zexti).
This patch folds the following pattern:

```
  %arg0 = ...
  %0 = zexti %arg0 : i1 to i8
  %1 = trunci %0 : i8 to i1
```

into just `%arg0`.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D99453
2021-03-27 19:40:10 +01:00
Alex Zinenko d68ba1fe50 [mlir] Register Linalg passes in C API and Python Bindings
Provide a registration mechanism for Linalg dialect-specific passes in C
API and Python bindings. These are being built into the dialect library
but exposed in separate headers (C) or modules (Python).

Differential Revision: https://reviews.llvm.org/D99431
2021-03-27 09:57:56 +01:00
Jacques Pienaar 7ce07c6494 [mlir] Remove unneeded ShapeFunctionLibraryTerminatorOp
Now that NoTerminator is possible this op can be removed/it was only
needed structurally before. NFC.
2021-03-26 16:03:51 -07:00
Nicolas Vasilache 69d01e0e40 [mlir][python] NFC - Fix stale path in doc
Differential Revision: https://reviews.llvm.org/D99345
2021-03-26 15:27:12 +00:00
Stella Laurenzo ec294eb87b [mlir][linalg] Add an InitTensorOp python builder.
* This has the API I want but I am not thrilled with the implementation. There are various things that could be improved both about the way that Python builders are mapped and the way the Linalg ops are factored to increase code sharing between C++/Python.
* Landing this as-is since it at least makes the InitTensorOp usable with the right API. Will refactor underneath in follow-ons.

Differential Revision: https://reviews.llvm.org/D99000
2021-03-25 15:17:48 -07:00
Mehdi Amini fcdf142ed5 Remove unused function, fix warning (NFC)
The `mayNotHaveTerminator` was initially on Block but moved to the
verifier before landing and wasn't removed from its original place
where it is unused.
2021-03-25 18:37:57 +00:00
Alexander Belyaev 7f2236cf58 [mlir][linalg] Add output tensor args folding for linalg.tiled_loop.
Folds away TiledLoopOp output tensors when the following conditions are met:
* result of `linalg.tiled_loop` has no uses
* output tensor is the argument of `linalg.yield`

Example:

```
%0 = linalg.tiled_loop ...  outs (%out, %out_buf:tensor<...>, memref<...>) {
  ...
  linalg.yield %out : tensor ...
}
```

Becomes

```
linalg.tiled_loop ...  outs (%out_buf:memref<...>) {
  ...
  linalg.yield
}
```

Differential Revision: https://reviews.llvm.org/D99333
2021-03-25 18:11:05 +01:00
Uday Bondhugula 0b20413ef6 Revert "[Canonicalizer] Process regions top-down instead of bottom up & reuse existing constants."
This reverts commit 361b7d125b by Chris
Lattner <clattner@nondot.org> dated Fri Mar 19 21:22:15 2021 -0700.

The change to the greedy rewriter driver picking a different order was
made without adequate analysis of the trade-offs and experimentation. A
change like this has far reaching consequences on transformation
pipelines, and a major impact upstream and downstream. For eg., one
can’t be sure that it doesn’t slow down a large number of cases by small
amounts or create other issues. More discussion here:
https://llvm.discourse.group/t/speeding-up-canonicalize/3015/25

Reverting this so that improvements to the traversal order can be made
on a clean slate, in bigger steps, and higher bar.

Differential Revision: https://reviews.llvm.org/D99329
2021-03-25 22:17:26 +05:30
Vladislav Vinogradov 70b6f16e07 [mlir] Support MemRefType with multiple AffineMaps in getStridesAndOffset
Compose multiple AffineMaps into single map before strides extraction.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D99166
2021-03-25 12:18:49 +03:00
Jean Perier ffa455d4d4 [mlir] Translate global initializers after creating all LLVM IR globals
In case an operation in a global initializer region refers to another
global variable defined afterwards in the module of itself, translation
to LLVM IR was currently crashing because it could not find the LLVM IR global
when going through the initializer block.

To solve this problem, split global conversion to LLVM IR into two passes. A
first pass that creates LLVM IR global variables, and a second one that converts
the initializer, if any, and adds it to the llvm global.

Differential Revision: https://reviews.llvm.org/D99246
2021-03-25 09:53:58 +01:00
Mehdi Amini 973ddb7d6e Define a `NoTerminator` traits that allows operations with a single block region to not provide a terminator
In particular for Graph Regions, the terminator needs is just a
historical artifact of the generalization of MLIR from CFG region.
Operations like Module don't need a terminator, and before Module
migrated to be an operation with region there wasn't any needed.

To validate the feature, the ModuleOp is migrated to use this trait and
the ModuleTerminator operation is deleted.

This patch is likely to break clients, if you're in this case:

- you may iterate on a ModuleOp with `getBody()->without_terminator()`,
  the solution is simple: just remove the ->without_terminator!
- you created a builder with `Builder::atBlockTerminator(module_body)`,
  just use `Builder::atBlockEnd(module_body)` instead.
- you were handling ModuleTerminator: it isn't needed anymore.
- for generic code, a `Block::mayNotHaveTerminator()` may be used.

Differential Revision: https://reviews.llvm.org/D98468
2021-03-25 03:59:03 +00:00
Rob Suderman f5ba3eea67 [mlir][tosa] Add tosa.bitwise_not lowering to constant and xor
Lowering of bitwise_not to linalg dialect using a xor operation with a constant
of all-bits-one.

Differential Revision: https://reviews.llvm.org/D99221
2021-03-24 17:27:27 -07:00
Lei Zhang 19435d3863 [mlir][linalg] Fold fill -> tensor_reshape chain
For such op chains, we can create new linalg.fill ops
with the result type of the linalg.tensor_reshape op.

Differential Revision: https://reviews.llvm.org/D99116
2021-03-24 18:17:58 -04:00
Lei Zhang c241e1c2f5 [mlir][linalg] Support dropping unit dimensions for init tensors
init tensor operands also has indexing map and generally follow
the same constraints we expect for non-init-tensor operands.

Differential Revision: https://reviews.llvm.org/D99115
2021-03-24 18:17:58 -04:00
Lei Zhang 7f28d27cb6 [mlir][linalg] Allow controlling folding unit dim reshapes
This commit exposes an option to the pattern
FoldWithProducerReshapeOpByExpansion to allow
folding unit dim reshapes. This gives callers
more fine-grained controls.

Differential Revision: https://reviews.llvm.org/D99114
2021-03-24 18:17:57 -04:00
Lei Zhang f66120a357 [mlir][affine] Add canonicalization to merge affine min/max ops
This identifies a pattern where the producer affine min/max op
is bound to a dimension/symbol that is used as a standalone
expression in the consumer affine op's map. In that case the
producer affine min/max op can be merged into its consumer.

For example, a pattern like the following:

```
  %0 = affine.min affine_map<()[s0] -> (s0 + 16, s0 * 8)> ()[%sym1]
  %1 = affine.min affine_map<(d0)[s0] -> (s0 + 4, d0)> (%0)[%sym2]
```

Can be turned into:

```
  %1 = affine.min affine_map<
         ()[s0, s1] -> (s0 + 4, s1 + 16, s1 * 8)> ()[%sym2, %sym1]
```

Differential Revision: https://reviews.llvm.org/D99016
2021-03-24 18:17:57 -04:00
Lei Zhang 23fd26608c [mlir][affine] Deduplicate affine min/max op expressions
If there are multiple identical expressions in an affine
min/max op's map, we can just keep one.

Differential Revision: https://reviews.llvm.org/D99015
2021-03-24 18:17:57 -04:00
Lei Zhang e58597ee1c [mlir][linalg] Fuse producers with non-permutation indexing maps
Until now Linalg fusion only allow fusing producers whose operands
are all permutation indexing maps. It's easier to deduce the
subtensor/subview but it is an unnecessary constraint, as in tiling
we have more advanced logic to deduce the subranges even when the
operand is not of permutation indexing maps, e.g., the input operand
for convolution ops.

This patch uses the logic on tiling side to deduce subranges for
fusion. This enables fusing convolution with its consumer ops
when possible.

Along the way, we are now generating proper affine.min ops to guard
against size boundaries, if we cannot be certain they won't be
out of bounds.

Differential Revision: https://reviews.llvm.org/D99014
2021-03-24 18:17:57 -04:00
Lei Zhang ddf93abf49 [mlir][linalg] NFC: Move makeTiledShapes into Utils.{h|cpp}
This is a preparation step to reuse makeTiledShapes in tensor
fusion. Along the way, did some lightweight cleanups.

Differential Revision: https://reviews.llvm.org/D99013
2021-03-24 18:17:57 -04:00
Jacques Pienaar 5d6b4aa80d [mlir] Compare elements directly rather than creating pair first
This avoided some conversion overhead on a model in TypeUniquer when
converting from ArrayRef -> TypeRange.

Differential Revision: https://reviews.llvm.org/D99300
2021-03-24 14:39:11 -07:00
Tobias Gysi 880822255e [mlir][linalg] Do not call region builder during vectorization.
All linalg operations having a region builder shall call it during op creation. Calling it during vectorization is obsolete.

Differential Revision: https://reviews.llvm.org/D99168
2021-03-24 14:55:11 +00:00
Alex Zinenko b3386a734e [mlir] introduce data layout entry for index type
Index type is an integer type of target-specific bitwidth present in many MLIR
operations (loops, memory accesses). Converting values of this type to
fixed-size integers has always been problematic. Introduce a data layout entry
to specify the bitwidth of `index` in a given layout scope, defaulting to 64
bits, which is a commonly used assumption, e.g., in constants.

Port builtin-to-LLVM type conversion to use this data layout entry when
converting `index` type and untie it from pointer size. This is particularly
relevant for GPU targets. Keep a possibility to forcibly override the index
type in lowerings.

Depends On D98525

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D98937
2021-03-24 15:13:42 +01:00
Alex Zinenko 842d243508 [mlir] forward data layout query to scoping op in absence of specification
Even if the layout specification is missing from an op that supports it, the op
is still expected to provide meaningful responses to data layout queries.
Forward them to the op instead of directly calling the default implementation.

Depends On D98524

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D98525
2021-03-24 15:13:41 +01:00
Alex Zinenko f9cdc61d11 [mlir] provide a version of data layout size hooks in bits
This is useful for bit-packing types such as vectors and tuples as well as for
exotic architectures that have non-8-bit bytes.

Depends On D98500

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D98524
2021-03-24 15:13:40 +01:00
Alex Zinenko 1916b0e098 [mlir] support data layout specs on ModuleOp
ModuleOp is a natural place to provide scoped data layout information. However,
it is undesirable for ModuleOp to implement the entirety of
DataLayoutOpInterface because that would require either pushing the interface
inside the IR library instead of a separate library, or putting the default
implementation of the interface as inline functions in headers leading to
binary bloat. Instead, ModuleOp accepts an arbitrary data layout spec attribute
and has a dedicated hook to extract it, and DataLayout is modified to know
about ModuleOp particularities.

Reviewed By: herhut, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D98500
2021-03-24 15:13:38 +01:00
Nicolas Vasilache 7716e5535c [mlir] Fixes to hoist padding
Fix the BlockAndValueMapping update that was missing entries for scf.for op's blockIterArgs.
Skip cloning subtensors of the padded tensor as the logic for these is separate.
Add a filter to drop side-effecting ops.

Tests are beefed up to verify the IR is sound in all hoisting configurations for 2-level 3-D tiled matmul.

Differential Revision: https://reviews.llvm.org/D99255
2021-03-24 11:51:28 +00:00
Vladislav Vinogradov 18a2f479bf [mlir][NFC] Replace `getMemorySpaceAsInt` with `getMemorySpace` where possible
Use new `MemRefType::getMemorySpace` method with generic Attribute
in cases, where there is no specific logic around the memory space.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D99154
2021-03-24 13:23:59 +03:00
Mehdi Amini d905c10353 Add a mechanism for Dialects to provide a fallback for OpInterface
This mechanism makes it possible for a dialect to not register all
operations but still answer interface-based queries.
This can useful for dialects that are "open" or connected to an external
system and still interoperate with the compiler. It can also open up the
possibility to have a more extensible compiler at runtime: the compiler
does not need a pre-registration for each operation and the dialect can
inject behavior dynamically.

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D93085
2021-03-24 08:41:40 +00:00
Rob Suderman 28e6420744 [mlir][tosa] Add tosa.argmax to linalg lowering
Tosa's argmax lowering is representable as a linalg.indexed_generic
operation. Include the lowering to this type for both integer and
floating point types.

Differential Revision: https://reviews.llvm.org/D99137
2021-03-23 16:06:55 -07:00
Rob Suderman 4157a079af [mlir][tosa] Add tosa.pad to linalg.pad operation
Lowers from tosa's pad op to the linalg equivalent for floating,
integer, and quantized values.

Differential Revision: https://reviews.llvm.org/D98990
2021-03-23 14:15:48 -07:00
River Riddle 76f3c2f3f3 [mlir][Pattern] Add better support for using interfaces/traits to match root operations in rewrite patterns
To match an interface or trait, users currently have to use the `MatchAny` tag. This tag can be quite problematic for compile time for things like the canonicalizer, as the `MatchAny` patterns may get applied to  *every* operation. This revision adds better support by bucketing interface/trait patterns based on which registered operations have them registered. This means that moving forward we will only attempt to match these patterns to operations that have this interface registered. Two simplify defining patterns that match traits and interfaces, two new utility classes have been added: OpTraitRewritePattern and OpInterfaceRewritePattern.

Differential Revision: https://reviews.llvm.org/D98986
2021-03-23 14:05:33 -07:00
Chris Lattner 782c534117 [ODS] Implement a new 'hasCanonicalizeMethod' bit for cann patterns.
This provides a simplified way to implement 'matchAndRewrite' style
canonicalization patterns for ops that don't need the full power of
RewritePatterns.  Using this style, you can implement a static method
with a signature like:

```
LogicalResult AssertOp::canonicalize(AssertOp op, PatternRewriter &rewriter) {
  return success();
}
```

instead of dealing with defining RewritePattern subclasses.  This also
adopts this for a few canonicalization patterns in the std dialect to
show how it works.

Differential Revision: https://reviews.llvm.org/D99143
2021-03-23 13:45:45 -07:00