Commit Graph

4651 Commits

Author SHA1 Message Date
Mehdi Amini c8a3f561eb Decouple registring passes from specifying argument/description
This patch changes the (not recommended) static registration API from:

 static PassRegistration<MyPass> reg("my-pass", "My Pass Description.");

to:

 static PassRegistration<MyPass> reg;

And the explicit registration from:

  void registerPass("my-pass", "My Pass Description.",
                    [] { return createMyPass(); });

To:

  void registerPass([] { return createMyPass(); });

It is expected that Pass implementations overrides the getArgument() method
instead. This will ensure that pipeline description can be printed and parsed
back.

Differential Revision: https://reviews.llvm.org/D104421
2021-06-16 23:41:50 +00:00
Robert David 7cfb7a67c5 [mlir] Make Type::print and Type::dump const 2021-06-16 15:31:20 -07:00
Jacques Pienaar 0e760a0870 Add hook for dialect specializing processing blocks post inlining calls
This allows for dialects to do different post-processing depending on operations with the inliner (my use case requires different attribute propagation rules depending on call op). This hook runs before the regular processInlinedBlocks method.

Differential Revision: https://reviews.llvm.org/D104399
2021-06-16 12:53:21 -07:00
Aart Bik 727a63e0d9 [mlir][sparse] allow all-dense annotated "sparse" tensor output
This is a very careful start with alllowing sparse tensors at the
left-hand-side of tensor index expressions (viz. sparse output).
Note that there is a subtle difference between non-annotated tensors
(dense, remain n-dim, handled by classic bufferization) and all-dense
annotated "sparse" tensors (linearized to 1-dim without overhead
storage, bufferized by sparse compiler, backed by runtime support library).
This revision gently introduces some new IR to facilitate annotated outputs,
to be generalized to truly sparse tensors in the future.

Reviewed By: gussmith23, bixia

Differential Revision: https://reviews.llvm.org/D104074
2021-06-15 14:55:07 -07:00
Arpith C. Jacob dd1992efd3 Support lowering of index-cast on vector types.
The index cast operation accepts vector types. Implement its lowering in this patch.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D104280
2021-06-15 12:51:30 -07:00
Tobias Gysi ff2ef4d684 [mlir][linalg] Adapt yaml codegen to support scalar parameters.
The patch updates the C++ yaml code generation to support scalar operands as added in https://reviews.llvm.org/D104220.

Differential Revision: https://reviews.llvm.org/D104224
2021-06-15 15:20:48 +00:00
Tobias Gysi e893708aac [mlir] Avoid GCC naming conflict.
Rename AbstractType to AbstractTy to avoid a GCC naming conflict after https://reviews.llvm.org/D104234.

Differential Revision: https://reviews.llvm.org/D104297
2021-06-15 14:48:30 +00:00
Adrian Kuegel f112bd61eb [mlir] Add SignOp to complex dialect.
Also add a conversion pattern from Complex Dialect to Standard/Math Dialect.

Differential Revision: https://reviews.llvm.org/D104292
2021-06-15 15:22:31 +02:00
Alex Zinenko 9b2a1bcf6f [mlir] separable registration of attribute and type interfaces
It may be desirable to provide an interface implementation for an attribute or
a type without modifying the definition of said attribute or type. Notably,
this allows to implement interfaces for attributes and types outside of the
dialect that defines them and, in particular, provide interfaces for built-in
types. Provide the mechanism to do so.

Currently, separable registration requires the attribute or type to have been
registered with the context, i.e. for the dialect containing the attribute or
type to be loaded. This can be relaxed in the future using a mechanism similar
to delayed dialect interface registration.

See https://llvm.discourse.group/t/rfc-separable-attribute-type-interfaces/3637

Depends On D104233

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D104234
2021-06-15 15:20:27 +02:00
Adrian Kuegel 662e074d90 [mlir] Add NegOp to complex dialect.
Also add a lowering pattern from complex dialect to standard dialect.

Differential Revision: https://reviews.llvm.org/D104284
2021-06-15 12:16:22 +02:00
Matthias Springer b6ab4f1a8b [mlir][linalg] Fold linalg.pad_tensor if src type == result type
Fold PadTensorOp to source if source type and result type have static shape and are equal.

Differential Revision: https://reviews.llvm.org/D103778
2021-06-15 17:25:12 +09:00
River Riddle 66e2708205 [mlir:Linalg] Populate LinalgOp patterns on LinalgDialect as opposed to each op
Interface patterns are unique in that they get added to every operation that also implements that interface, given that they aren't tied to individual operations. When the same interface pattern gets added to multiple operations (such as the current behavior with Linalg), an reference to each of these patterns is added to every op (meaning that an operation will now have N references to effectively the same pattern). This revision fixes this problematic behavior in Linalg, and can bring upwards of a 25% reduction in compile time in Linalg based workloads.

Differential Revision: https://reviews.llvm.org/D104160
2021-06-14 11:20:15 -07:00
Denys Shabalin c83e696732 Add AutomaticAllocationScope to memref.alloca_scope
This change adds `AutomaticAllocationScope` to the
memref.alloca_scope op. Additionally, it also clarifies
that alloca_scope is is conceptually a passthrough operation.

Reviewed By: ftynse, bondhugula

Differential Revision: https://reviews.llvm.org/D104227
2021-06-14 16:09:06 +02:00
Uday Bondhugula 88e4aae57d [MLIR][NFC] Rename MemRefDataFlow -> AffineScalarReplacement
NFC. Rename MemRefDataFlow -> AffineScalarReplacement and move to
AffineTransforms library. Pass command line rename: -memref-dataflow-opt
-> affine-scalrep. Update outdated pass documentation.

Rationale:
https://llvm.discourse.group/t/move-and-rename-memref-dataflow-opt-lib-transforms-lib-affine-dialect-transforms/3640

Differential Revision: https://reviews.llvm.org/D104190
2021-06-14 17:52:53 +05:30
Adrian Kuegel 2116eadf09 [mlir] Remove traits that require vector type support from ops in Complex dialect.
Actually, no vector types are supported so far. We should add the traits once
the vector types are supported (e.g. ElementwiseMappable.traits).
Instead add Elementwise trait to each op.

Differential Revision: https://reviews.llvm.org/D104103
2021-06-14 08:30:22 +02:00
Tobias Gysi 046922e100 [mlir][linalg] Add support for scalar input operands.
Up to now all structured op operands are assumed to be shaped. The patch relaxes this assumption and allows scalar input operands. In contrast to shaped operands scalar operands are not indexed and directly forwarded to the body of the operation. As all other operands, scalar operands are associated to an indexing map that in case of a scalar or a 0D-operand has an empty range.

We will use scalar operands as a replacement for the capture mechanism. In contrast to captures, the approach ensures we can generate the function signature from the operand list and it prevents outdated capture values in case a transformation updates only the capture operand but not the hidden body of a named operation.

Removing captures and updating existing operations such as linalg.fill is left for a later patch.

The patch depends on https://reviews.llvm.org/D103891 and https://reviews.llvm.org/D103890.

Differential Revision: https://reviews.llvm.org/D104109
2021-06-14 06:27:16 +00:00
Adrian Kuegel 73cbc91c93 [mlir] Add ExpOp to Complex dialect.
Also add a conversion pattern from Complex to Standard/Math dialect.

Differential Revision: https://reviews.llvm.org/D104108
2021-06-14 08:08:53 +02:00
Chris Lattner 4fa8677860 [DominanceInfo] Make the ctor take a defaulted value for the operand. NFC.
This allows it to be default constructible, which makes sense given it
ignores the operand.
2021-06-13 18:25:47 -07:00
Matthias Springer bf5d3092f8 [mlir][linalg] Add constant padding helper to PadTensorOp
* Add a helper function that returns the constant padding value (if applicable).
* Remove existing getConstantYieldValueFromBlock function, which does almost the same.
* Adapted from D103243.

Differential Revision: https://reviews.llvm.org/D104004
2021-06-14 09:44:39 +09:00
Hanhan Wang b4baccc2a7 Introduce tensor.insert op to Tensor dialect.
Add `tensor.insert` op to make `tensor.extract`/`tensor.insert` work in pairs
for `scalar` domain. Like `subtensor`/`subtensor_insert` work in pairs in
`tensor` domain, and `vector.transfer_read`/`vector.transfer_write` work in
pairs in `vector` domain.

Reviewed By: silvas

Differential Revision: https://reviews.llvm.org/D104139
2021-06-13 13:45:40 -07:00
Mehdi Amini 8bc1ce0f61 Use dyn_cast_or_null instead of dyn_cast in FunctionLike::verifyTrait (NFC)
This is making the verifier more tolerant to cases where a "null"
Attribute would be inserted in the array of func arguments/results
attributes.
2021-06-12 20:08:37 +00:00
Stephen Neuendorffer 984e270a9a [mlir] make normalizeAffineFor public
Previously this was just a static method.
2021-06-11 20:12:37 -07:00
Uday Bondhugula c8b8e8e022 [MLIR] Execution engine python binding support for shared libraries
Add support to Python bindings for the MLIR execution engine to load a
specified list of shared libraries - for eg. to use MLIR runtime
utility libraries.

Differential Revision: https://reviews.llvm.org/D104009
2021-06-12 05:46:38 +05:30
Denys Shabalin fdc0d4360b Introduce alloca_scope op
## Introduction

This proposal describes the new op to be added to the `std` (and later moved `memref`)
dialect called `alloca_scope`.

## Motivation

Alloca operations are easy to misuse, especially if one relies on it while doing
rewriting/conversion passes. For example let's consider a simple example of two
independent dialects, one defines an op that wants to allocate on-stack and
another defines a construct that corresponds to some form of looping:

```
dialect1.looping_op {
  %x = dialect2.stack_allocating_op
}
```

Since the dialects might not know about each other they are going to define a
lowering to std/scf/etc independently:

```
scf.for … {
   %x_temp = std.alloca …
   … // do some domain-specific work using %x_temp buffer
   … // and store the result into %result
   %x = %result
}
```

Later on the scf and `std.alloca` is going to be lowered to llvm using a
combination of `llvm.alloca` and unstructured control flow.

At this point the use of `%x_temp` is bound to either be either optimized by
llvm (for example using mem2reg) or in the worst case: perform an independent
stack allocation on each iteration of the loop. While the llvm optimizations are
likely to succeed they are not guaranteed to do so, and they provide
opportunities for surprising issues with unexpected use of stack size.

## Proposal

We propose a new operation that defines a finer-grain allocation scope for the
alloca-allocated memory called `alloca_scope`:

```
alloca_scope {
   %x_temp = alloca …
   ...
}
```

Here the lifetime of `%x_temp` is going to be bound to the narrow annotated
region within `alloca_scope`. Moreover, one can also return values out of the
alloca_scope with an accompanying `alloca_scope.return` op (that behaves
similarly to `scf.yield`):

```
%result = alloca_scope {
   %x_temp = alloca …
   …
   alloca_scope.return %myvalue
}
```

Under the hood the `alloca_scope` is going to lowered to a combination of
`llvm.intr.stacksave` and `llvm.intr.strackrestore` that are going to be invoked
automatically as control-flow enters and leaves the body of the `alloca_scope`.

The key value of the new op is to allow deterministic guaranteed stack use
through an explicit annotation in the code which is finer-grain than the
function-level scope of `AutomaticAllocationScope` interface. `alloca_scope`
can be inserted at arbitrary locations and doesn’t require non-trivial
transformations such as outlining.

## Which dialect

Before memref dialect is split, `alloca_scope` can temporarily reside in `std`
dialect, and later on be moved to `memref` together with the rest of
memory-related operations.

## Implementation

An implementation of the op is available [here](https://reviews.llvm.org/D97768).

Original commits:

* Add initial scaffolding for alloca_scope op
* Add alloca_scope.return op
* Add no region arguments and variadic results
* Add op descriptions
* Add failing test case
* Add another failing test
* Initial implementation of lowering for std.alloca_scope
* Fix backticks
* Fix getSuccessorRegions implementation

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D97768
2021-06-11 19:28:41 +02:00
thomasraoux edd9515bd1 [mlir][VectorToGPU] First step to convert vector ops to GPU MMA ops
This is the first step to convert vector ops to MMA operations in order to
target GPUs tensor core ops. This currently only support simple cases,
transpose and element-wise operation will be added later.

Differential Revision: https://reviews.llvm.org/D102962
2021-06-11 07:52:32 -07:00
Tobias Gysi 6b4b98d98c [mlir][[linalg] Remove deprecated structured op interface methods.
Cleanup the refactoring started by https://reviews.llvm.org/D103394.

Differential Revision: https://reviews.llvm.org/D104025
2021-06-11 12:45:20 +00:00
Adrian Kuegel f98b779614 [mlir] Refactor ComplexOps.td [NFC]
Create a ComplexUnaryOp base class and use it for AbsOp, ReOp and ImOp.
Sort all ops in lexicographic order.

Differential Revision: https://reviews.llvm.org/D104095
2021-06-11 10:53:29 +02:00
River Riddle c42dd5dbb0 [mlir] Add new SubElementAttr/SubElementType Interfaces
These interfaces allow for a composite attribute or type to opaquely provide access to any held attributes or types. There are several intended use cases for this interface. The first of which is to allow the printer to create aliases for non-builtin dialect attributes and types. In the future, this interface will also be extended to allow for SymbolRefAttr to be placed on other entities aside from just DictionaryAttr and ArrayAttr.

To limit potential test breakages, this revision only adds the new interfaces to the builtin attributes/types that are currently hardcoded during AsmPrinter alias generation. In a followup the remaining builtin attributes/types, and non-builtin attributes/types can be extended to support it.

Differential Revision: https://reviews.llvm.org/D102945
2021-06-10 17:23:07 -07:00
River Riddle f8a1d652da [mlir][IR] Move MemRefElementTypeInterface to a new BuiltinTypeInterfaces file
This allows for using other type interfaces in the builtin dialect, which currently results in a compile time failure (as it generates duplicate interface declarations).
2021-06-10 17:23:06 -07:00
Benoit Jacob 20daedacca 2d Arm Neon sdot op, and lowering to the intrinsic.
This adds Sdot2d op, which is similar to the usual Neon
intrinsic except that it takes 2d vector operands, reflecting the
structure of the arithmetic that it's performing: 4 separate
4-dimensional dot products, whence the vector<4x4xi8> shape.

This also adds a new pass, arm-neon-2d-to-intr, lowering
this new 2d op to the 1d intrinsic.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D102504
2021-06-10 14:36:39 -07:00
River Riddle ff81a2c95d [mlir-lsp-server] Add support for textDocument/documentSymbols
This allows for building an outline of the symbols and symbol tables within the IR. This allows for easy navigations to functions/modules and other symbol/symbol table operations within the IR.

Differential Revision: https://reviews.llvm.org/D103729
2021-06-10 10:58:39 -07:00
thomasraoux 428a62f65f [mlir][gpu] Add op to create MMA constant matrix
This allow creating a matrix with all elements set to a given value. This is
needed to be able to implement a simple dot op.

Differential Revision: https://reviews.llvm.org/D103870
2021-06-10 08:34:04 -07:00
Guillaume Chatelet e0569033e2 [llvm] Make Sequence reverse-iterable
This is a roll forward of D102679.
This patch simplifies the implementation of Sequence and makes it compatible with llvm::reverse.
It exposes the reverse iterators through rbegin/rend which prevents a dangling reference in std::reverse_iterator::operator++().

Note: Compared to D102679, this patch introduces a `asSmallVector()` member function and fixes compilation issue with GCC 5.

Differential Revision: https://reviews.llvm.org/D103948
2021-06-10 11:15:28 +00:00
Alex Zinenko 7325aaefa5 [mlir] make LLVMPointerType implement the data layout type interface
This brings us closer to replacing the LLVM data layout string with a
first-class layout modeling in MLIR.

Depends On D103945

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D103946
2021-06-10 11:24:16 +02:00
Alexander Kornienko ad6a84f82c Revert "[Verifier] Speed up and parallelize dominance checking. NFC"
This reverts commit 08664d005c, which according to
https://reviews.llvm.org/D103373 was pushed accidentally, and I believe it
causes timeouts in some internal mlir tests.
2021-06-10 09:58:05 +02:00
Ahmed Taei b9d7ffd9cf Folds linalg.pad_tensor with zero padding
Differential Revision: https://reviews.llvm.org/D103984
2021-06-09 15:39:40 -07:00
Mehdi Amini 1b21e9c1fa Add a static assertions for custom Op<> to not defined data members (NFC)
A common mistake for newcomers to MLIR is to try to store extra member
on the Op class. However these are intended to be thing wrapper around
an Operation*, all the storage is meant to be encoded in attribute on
the underlying Operation. This can be confusing to debug, so better
catch it at build time.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D103869
2021-06-09 21:45:06 +00:00
Lei Zhang 56f60a1ce7 [mlir][spirv] Use SingleBlock + NoTerminator for spv.module
This allows us to remove the `spv.mlir.endmodule` op and
all the code associated with it.

Along the way, tightened the APIs for `spv.module` a bit
by removing some aliases. Now we use `getRegion` to get
the only region, and `getBody` to get the region's only
block.

Reviewed By: mravishankar, hanchung

Differential Revision: https://reviews.llvm.org/D103265
2021-06-09 14:00:06 -04:00
Javier Setoain 96ca2d92b5 [mlir][ArmSVE] Add basic load/store operations
ArmSVE-specific memory operations are needed to generate end-to-end
code for as long as MLIR core doesn't support scalable vectors. This
instructions will be eventually unnecessary, for now they're required
for more complex testing.

Differential Revision: https://reviews.llvm.org/D103535
2021-06-09 15:53:40 +01:00
Benjamin Kramer c0db8d50ca [mlir] Expose a function to populate tensor constant bufferization patterns
This makes it easier to use it from other bufferization passes.

Differential Revision: https://reviews.llvm.org/D103838
2021-06-09 13:47:33 +02:00
Javier Setoain f880bd261f [mlir][ArmSVE] Add basic mask generation operations
These `arm_sve.cmp` functions are needed to generate scalable vector
masks as long as scalable vectors are not part of the standard types.
Once in standard, these can be removed and `std.cmp` can be used
instead.

Differential Revision: https://reviews.llvm.org/D103473
2021-06-09 09:56:53 +01:00
Fabian Schuiki 41135a4367
[MLIR] Make DictionaryAttr::getAs take name as && reference
As a follow-up to the discussion in https://reviews.llvm.org/D103822,
make the templated `DictionaryAttr::getAs` take the name by `&&`
reference and properly forward the argument to the underlying `get`.
2021-06-09 10:04:40 +02:00
Mehdi Amini a0ac514676 Revert "Add a static assertions for custom Op<> to not defined data members (NFC)"
This reverts commit c0edcec630.

The windows bot was broken by this change.
2021-06-09 00:46:02 +00:00
Suraj Sudhir 05cadc6f71 [mlir][tosa] Temporarily support 2D and 3D tensor types in matmul
Temporarily support 2D and 3D while the TOSA Matmul op is updated to support batched operations.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D103854
2021-06-08 16:03:59 -07:00
Mehdi Amini c0edcec630 Add a static assertions for custom Op<> to not defined data members (NFC)
A common mistake for newcomers to MLIR is to try to store extra member
on the Op class. However these are intended to be thing wrapper around
an Operation*, all the storage is meant to be encoded in attribute on
the underlying Operation. This can be confusing to debug, so better
catch it at build time.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D103869
2021-06-08 18:38:18 +00:00
Mehdi Amini a4e2cf712a Revert "[llvm] Make Sequence reverse-iterable"
This reverts commit e772216e70
(and fixup 7f6c878a2c).

The build is broken with gcc5 host compiler:

In file included from
                 from mlir/lib/Dialect/Utils/StructuredOpsUtils.cpp:9:
tools/mlir/include/mlir/IR/BuiltinAttributes.h.inc:424:57: error: type/value mismatch at argument 1 in template parameter list for 'template<class ItTy, class FuncTy, class FuncReturnTy> class llvm::mapped_iterator'
                               std::function<T(ptrdiff_t)>>;
                                                         ^
tools/mlir/include/mlir/IR/BuiltinAttributes.h.inc:424:57: note:   expected a type, got 'decltype (seq<ptrdiff_t>(0, 0))::const_iterator'
2021-06-08 17:03:10 +00:00
Chris Lattner 92a79dbe91 [Core] Add Twine support for StringAttr and Identifier. NFC.
This is both more efficient and more ergonomic than going
through an std::string, e.g. when using llvm::utostr and
in string concat cases.

Unfortunately we can't just overload ::get().  This causes an
ambiguity because both twine and stringref implicitly convert
from std::string.

Differential Revision: https://reviews.llvm.org/D103754
2021-06-08 09:47:07 -07:00
Chris Lattner 08664d005c [Verifier] Speed up and parallelize dominance checking. NFC
One of the key algorithms used in the "mlir::verify(op)" method is the
dominance checker, which ensures that operand values properly dominate
the operations that use them.

The MLIR dominance implementation has a number of algorithmic problems,
and is not really set up in general to answer dense queries: it's constant
factors are really slow with multiple map lookups and scans, even in the
easy cases.  Furthermore, when calling mlir::verify(module) or some other
high level operation, it makes sense to parallelize the dominator
verification of all the functions within the module.

This patch has a few changes to enact this:
 1) It splits dominance checking into "IsolatedFromAbove" units.  Instead
    of building a monolithic DominanceInfo for everything in a module,
    for example, it checks dominance for the module to all the functions
    within it (noop, since there are no operands at this level) then each
    function gets their own DominanceInfo for each of their scope.
 2) It adds the ability for mlir::DominanceInfo (and post dom) to be
    constrained to an IsolatedFromAbove region.  There is no reason to
    recurse into IsolatedFromAbove regions since use/def relationships
    can't span this region anyway.  This is already checked by the time
    the verifier gets here.
 3) It avoids querying DominanceInfo for trivial checks (e.g. intra Block
    references) to eliminate constant factor issues).
 4) It switches to lazily constructing DominanceInfo because the trivial
    check case handles the vast majority of the cases and avoids
    constructing DominanceInfo entirely in some cases (e.g. at the module
    level or for many Regions's that contain a single Block).
 5) It parallelizes analysis of collections IsolatedFromAbove operations,
    e.g. each of the functions within a Module.

All together this is more than a 10% speedup on `firtool` in circt on a
large design when run in -verify-each mode (our default) since the verifier
is invoked after each pass.

Still todo is to parallelize the main verifier pass.  I decided to split
this out to its own thing since this patch is already large-ish.

Differential Revision: https://reviews.llvm.org/D103373
2021-06-08 09:47:07 -07:00
Kiran Chandramohan cd73af9231 [MLIR] Remove LLVM_AnyInteger type constraint
LLVM Dialect uses builtin-integer types. The existing LLVM_AnyInteger
type constraint is a dupe of AnyInteger. This patch removes LLVM_AnyInteger
and replaces all usage with AnyInteger.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D103839
2021-06-08 17:21:00 +01:00
Guillaume Chatelet e772216e70 [llvm] Make Sequence reverse-iterable
This patch simplifies the implementation of Sequence and makes it compatible with llvm::reverse.
It exposes the reverse iterators through rbegin/rend which prevents a dangling reference in std::reverse_iterator::operator++().

Differential Revision: https://reviews.llvm.org/D102679
2021-06-08 13:18:57 +00:00
Fabian Schuiki 41eb2cec47
[MLIR] Mark additional builtin attr methods const
* Mark the following methods const:
  * `ArrayAttr::getAsRange`
  * `ArrayAttr::getAsValueRange`
  * `DictionaryAttr::getAs`
* Make `DictionarAttr::getAs` generic over the name class, such that
  `Identifier` and `StringRef` arguments get forwarded to the underlying
  call to `get`. (Made generic to avoid introducing a dependency on
  `include/mlir/IR/Identifier.h` as per the diff discussion.)

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D103822
2021-06-08 14:45:19 +02:00
Alex Zinenko c59ce1f625 [mlir] support memref of memref in standard-to-llvm conversion
Now that memref supports arbitrary element types, add support for memref of
memref and make sure it is properly converted to the LLVM dialect. The type
support itself avoids adding the interface to the memref type itself similarly
to other built-in types. This allows the shape, and therefore byte size, of the
memref descriptor to remain a lowering aspect that is easier to customize and
evolve as opposed to sanctifying it in the data layout specification for the
memref type itself.

Factor out the code previously in a testing pass to live in a dedicated data
layout analysis and use that analysis in the conversion to compute the
allocation size for memref of memref. Other conversions will be ported
separately.

Depends On D103827

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D103828
2021-06-08 11:11:31 +02:00
Alex Zinenko ada9aa5a22 [mlir] Make MemRef element type extensible
Historically, MemRef only supported a restricted list of element types that
were known to be storable in memory. This is unnecessarily restrictive given
the open nature of MLIR's type system. Allow types to opt into being used as
MemRef elements by implementing a type interface. For now, the interface is
merely a declaration with no methods. Later, methods to query, e.g., the type
size or whether a type can alias elements of another type may be added.

Harden the "standard"-to-LLVM conversion against memrefs with non-builtin
types.

See https://llvm.discourse.group/t/rfc-memref-of-custom-types/3558.

Depends On D103826

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D103827
2021-06-08 11:11:30 +02:00
Javier Setoain 57546f5b22 Revert "[mlir][ArmSVE] Add basic mask generation operations"
This reverts commit 392af6a78b
2021-06-08 10:02:19 +01:00
Javier Setoain 392af6a78b [mlir][ArmSVE] Add basic mask generation operations
These `arm_sve.cmp` functions are needed to generate scalable vector
masks as long as scalable vectors are not part of the standard types.
Once in standard, these can be removed and `std.cmp` can be used
instead.

Differential Revision: https://reviews.llvm.org/D103473
2021-06-08 08:56:31 +01:00
River Riddle f492c35965 [mlir-lsp-server] Add support for hover on region operations
This revision adds support for hover on region operations, by temporarily removing the regions during printing. This revision also tweaks the hover format for operations to include symbol information, now that FuncOp can be shown in the hover.

Differential Revision: https://reviews.llvm.org/D103727
2021-06-07 14:07:41 -07:00
William S. Moses 00b6463b26 [MLIR][GPU] Simplify memcpy of cast
Introduce a simplification that allows memcpy of a cast to simply use the underlying op

Differential Revision: https://reviews.llvm.org/D103830
2021-06-07 14:00:13 -04:00
Valentin Clement fb5b590b5e [mlir][openacc] Add conversion for if operand to scf.if for standalone data operation
This patch convert the if condition on standalone data operation such as acc.update,
acc.enter_data and acc.exit_data to a scf.if with the operation in the if region.
It removes the operation when the if condition is constant and false. It removes the
the condition if it is contant and true.

Conversion to scf.if is done in order to use the translation to LLVM IR dialect out of the box.
Not sure this is the best approach or we should perform this during the translation from OpenACC
to LLVM IR dialect. Any thoughts welcome.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D103325
2021-06-07 12:10:03 -04:00
Valentin Clement aa4e6a609a [mlir][openacc] Add canonicalization for standalone data operations for if condition
This patch add canonicalization for the standalone data operation with constant if condition.
It is extracted from this patch D103325.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D103712
2021-06-07 11:40:59 -04:00
Valentin Clement cfcdebaf32 [mlir][openacc] Conversion of data operands in acc.parallel to LLVM IR dialect
Convert data operands from the acc.parallel operation using the same conversion pattern than D102170.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D103337
2021-06-07 11:22:20 -04:00
KareemErgawy 2def12ebc6 [MLIR][SPIRV] Use getAsmResultName(...) hook for AddressOfOp.
Implements better naming for results of spv.mlir.addressof ops by making it
inherit from OpAsmOpInterface and implementing the associated
getAsmResultName(...) hook.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D103594
2021-06-07 13:58:26 +02:00
Matthias Springer fe0befb123 [mlir][linalg] Add padding helper functions to PadTensorOp
Add helper functions to quickly check for zero low/high padding.

Differential Revision: https://reviews.llvm.org/D103781
2021-06-07 20:18:06 +09:00
Matthias Springer 6e7bbdd6e7 [mlir] Add offset/stride helper functions to OffsetSizeAndStrideOpInterface
* Add hasUnitStride and hasZeroOffset to OffsetSizeAndStrideOpInterface. These functions are useful for various patterns. E.g., some vectorization patterns apply only for tensor ops with zero offsets and/or unit stride.
* Add getConstantIntValue and isEqualConstantInt helper functions, which are useful for implementing the two above functions, as well as various patterns.

Differential Revision: https://reviews.llvm.org/D103763
2021-06-07 20:11:41 +09:00
Aart Bik 86e9bc1a34 [mlir][sparse] add option for 32-bit indices in scatter/gather
Controlled by a compiler option, if 32-bit indices can be handled
with zero/sign-extention alike (viz. no worries on non-negative
indices), scatter/gather operations can use the more efficient
32-bit SIMD version.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D103632
2021-06-04 16:57:12 -07:00
Rob Suderman d86ef4364f [mlir][tosa] Update tosa.rescale for i48 input type
i48 integers require slightly tweaked behavior, specifically supporting zero
point offsetting with slightly higher bitdepth. Updated results lowering
appropriately.

Reviewed By: NatashaKnk

Differential Revision: https://reviews.llvm.org/D102659
2021-06-04 16:36:48 -07:00
Matthias Springer e789efc92a [mlir][linalg] Refactor PadTensorOpVectorizationPattern (NFC)
* Rename PadTensorOpVectorizationPattern to GenericPadTensorOpVectorizationPattern.
* Make GenericPadTensorOpVectorizationPattern a private pattern, to be instantiated via populatePadTensorOpVectorizationPatterns.
* Factor out parts of PadTensorOpVectorizationPattern into helper functions.

This commit prepares PadTensorOpVectorizationPattern for a series of subsequent commits that add more specialized PadTensorOp vectorization patterns.

Differential Revision: https://reviews.llvm.org/D103681
2021-06-04 23:45:08 +09:00
Valentin Clement fcb1547229 [mlir][openacc] Conversion of data operands in acc.data to LLVM IR dialect
Convert data operands from the acc.data operation using the same conversion pattern than D102170.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D103332
2021-06-04 10:26:22 -04:00
Tobias Gysi 67b1c37d9f [mlir][linalg] Cleanup left over uses of deprecated LinalgOp methods.
Replace all remaining uses of deprecated Structured Op Interface methods. This patch is based on https://reviews.llvm.org/D103394.

Differential Revision: https://reviews.llvm.org/D103673
2021-06-04 08:48:02 +00:00
Stella Laurenzo 8a96993da5 [NFC] Add ArrayRef includes to two files.
These started failing on one of our buildbots. I didn't completely root cause the situation and just added the explicit includes that correct the issue.

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D103657
2021-06-03 16:38:58 -07:00
River Riddle d6af89beb2 [mlir-lsp-server] Add support for tracking the use/def chains of symbols
This revision adds assembly state tracking for uses of symbols, allowing for go-to-definition and references support for SymbolRefAttrs.

Differential Revision: https://reviews.llvm.org/D103585
2021-06-03 16:12:27 -07:00
Nikita Popov 983565a6fe [ADT] Move DenseMapInfo for ArrayRef/StringRef into respective headers (NFC)
This is a followup to D103422. The DenseMapInfo implementations for
ArrayRef and StringRef are moved into the ArrayRef.h and StringRef.h
headers, which means that these two headers no longer need to be
included by DenseMapInfo.h.

This required adding a few additional includes, as many files were
relying on various things pulled in by ArrayRef.h.

Differential Revision: https://reviews.llvm.org/D103491
2021-06-03 18:34:36 +02:00
Alexander Belyaev 89df483d30 [mlir] Fix warnings. 2021-06-03 17:09:09 +02:00
Nicolas Agostini 0804a88e48 [mlir][linalg] Transform PadTensorOp into InitOp, FillOp, GenericOp
Introduces a test pass that rewrites PadTensorOps with static shapes as a sequence of:

```
linalg.init_tensor // to create output
linalg.fill        // to initialize with padding value
linalg.generic     // to copy the original contents to the padded tensor
```

The pass can be triggered with:

- `--test-linalg-transform-patterns="test-transform-pad-tensor"`

Differential Revision: https://reviews.llvm.org/D102804
2021-06-03 22:09:09 +09:00
Alexander Belyaev 485c21be8a [mlir] Split linalg reshape ops into expand/collapse.
Differential Revision: https://reviews.llvm.org/D103548
2021-06-03 11:40:22 +02:00
Chia-hung Duan 2f98dfe5b6 [mlir-reduce] Create MlirReduceLib
Move the core reducer algorithm into a library so that it'll be easier
for porting to different projects.

Depends On D101046

Reviewed By: jpienaar, rriddle

Differential Revision: https://reviews.llvm.org/D101607
2021-06-03 15:58:26 +08:00
River Riddle c89dff5855 [mlir][NFC] Split the non-templated bits out of IROperand into a base class
This removes the need to define the derived Operand class before the derived
Value class. The major benefit of this refactoring is that we no longer need
the OpaqueValue class, as OpOperand can now be defined after Value. As part of
this refactoring the BlockOperand and OpOperand classes are moved out of
UseDefLists.h and to more suitable locations in BlockSupport and Value. After
this change, UseDefLists.h is almost entirely composed of generic use def utilities.

Differential Revision: https://reviews.llvm.org/D103353
2021-06-02 12:48:37 -07:00
River Riddle fa51c5af5d [mlir] Resolve TODO and use the pass argument instead of the TypeID for registration
This simplifies various pieces of code that interact with the pass registry, e.g. this removes the need to register passes to get accurate pass pipelines descriptions when generating crash reproducers.

Differential Revision: https://reviews.llvm.org/D101880
2021-06-02 12:17:36 -07:00
River Riddle 0289a2692e [mlir] Add support for filtering patterns based on debug names and labels
This revision allows for attaching "debug labels" to patterns, and provides to FrozenRewritePatternSet for  filtering patterns based on these labels (in addition to the debug name of the pattern). This will greatly simplify the ability to write tests targeted towards specific patterns (in cases where many patterns may interact),  will also simplify debugging pattern application by observing how application changes when enabling/disabling specific patterns.

To enable better reuse of pattern rewrite options between passes, this revision also adds a new PassUtil.td file to the Rewrite/ library that will allow for passes to easily hook into a common interface for pattern debugging. Two options are used to seed this utility, `disable-patterns` and `enable-patterns`, which are used to enable the filtering behavior indicated above.

Differential Revision: https://reviews.llvm.org/D102441
2021-06-02 12:05:25 -07:00
Jacques Pienaar 644f722b36 [mlir-lsp] Report range of potential identifier starting at location of diagnostic
Currently the diagnostics reports the file:line:col, but some LSP
frontends require a non-empty range. Report either the range of an
identifier that starts at location, or a range of 1. Expose the id
location to range helper and reuse here.

Differential Revision: https://reviews.llvm.org/D103482
2021-06-02 10:49:53 -07:00
Tobias Gysi e1a150846d [mlir][linalg] Update result position calculation in the Structured Op Interface (NFC).
Remove two unused methods and replace the implementation of getResultsPositionInLoopsToShapeMap. The patch is based on https://reviews.llvm.org/D103394.

Differential Revision: https://reviews.llvm.org/D103397
2021-06-02 12:31:02 +00:00
Matthias Springer 558e740170 [mlir] Support tensor types in non-unrolled VectorToSCF
Support for tensor types in the unrolled version will follow in a separate commit.

Add a new pass option to activate lowering of transfer ops with tensor types (default: deactivated).

Differential Revision: https://reviews.llvm.org/D102666
2021-06-02 10:37:58 +09:00
Chia-hung Duan c484c7dd9d [mlir-reduce] Reducer refactor.
* A Reducer is a kind of RewritePattern, so it's just the same as
writing graph rewrite.
* ReductionTreePass operates on Operation rather than ModuleOp, so that
* we are able to reduce a nested structure(e.g., module in module) by
* self-nesting.

Reviewed By: jpienaar, rriddle

Differential Revision: https://reviews.llvm.org/D101046
2021-06-02 07:45:00 +08:00
Chris Lattner 412ae15de4 [Dominators] Rewrite the dominator implementation for efficiency. NFC.
The previous impl densely scanned the entire region starting with an op
when dominators were created, creating a DominatorTree for every region.

This is extremely expensive up front -- particularly for clients like
Linalg/Transforms/Fusion.cpp that construct DominanceInfo for a single
query.  It is also extremely memory wasteful for IRs that use single
block regions commonly (e.g. affine.for) because it's making a
dominator tree for a region that has trivial dominance.  The
implementation also had numerous unnecessary minor efficiencies, e.g.
doing multiple walks of the region tree or tryGetBlocksInSameRegion
building a DenseMap that it didn't need.

This patch switches to an approach where [Post]DominanceInfo is free
to construct, and which lazily constructs DominatorTree's for any
multiblock regions that it needs.  This avoids the up-front cost
entirely, making its runtime proportional to the complexity of the
region tree instead of # ops in a region.  This also avoids the memory
and time cost of creating DominatorTree's for single block regions.

Finally this rewrites the implementation for simplicity and to avoids
the constant factor problems the old implementation had.

Differential Revision: https://reviews.llvm.org/D103384
2021-06-01 14:46:37 -07:00
Rob Suderman 422c7036d5 [mlir] Updated depthwise conv to support kernel dilation
Depthwise convolution should support kernel dilation and non-dilation should
not be a special case. Updated op definition to include a dilation attribute.

This also adds a tosa.depthwise_conv2d lowering to linalg to support the new
linalg behavior.

Differential Revision: https://reviews.llvm.org/D103219
2021-06-01 13:25:19 -07:00
Tobias Gysi 94643fda13 [mlir][linalg] Cleanup LinalgOp usage in dependence analysis (NFC).
Replace the uses of deprecated Structured Op Interface methods in DependenceAnalysis.cpp and DependenceAnalysis.h. This patch is based on https://reviews.llvm.org/D103394.

Differential Revision: https://reviews.llvm.org/D103411
2021-06-01 08:44:15 +00:00
Andi Drebes 73b759a33a [MLIR] Add dialect prefix to all operations from Math
Prefix all operations from the ODS of the `Math` dialect with `Math_`
in order to avoid name clashes when including `MathOps.td` in other
TableGen files (e.g., for `FloatUnaryOp`, which also exists in
`Standard`).

Reviewed By: jpienaar, mehdi_amini

Differential Revision: https://reviews.llvm.org/D103248
2021-05-31 22:59:26 +05:30
Andi Drebes e4034881c3 [MLIR] Specify namespace for standard dialect using cppNamespace field
The `::mlir` namespace for operations from standard is currently
defined by enclosing the header file generated from the ODS in
`Ops.td` in a namespace in `Ops.h`. However, when referencing
operations from `Ops.td` in other TableGen files, this causes the
generated C++ code to refer to classes from the global namespace
instead of `::mlir`.

By defining the namespace through the `cppNamespace` field for
`StandardOps_Dialect` directly in `Ops.td` instead, the ODS
becomes reusable in other TableGen files through simple
inclusion.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D103234
2021-05-31 22:43:25 +05:30
Tobias Gysi 0a52d9006c [mlir][linalg] Update Structured Op Interface (NFC).
Adding methods to access operand properties via OpOperands and mark outdated methods as deprecated.

Differential Revision: https://reviews.llvm.org/D103394
2021-05-31 13:20:48 +00:00
Chris Lattner 1e344ce4f3 [CSE] Make domInfo a stored property, cut use of DominanceInfo::hasDominanceInfo. NFC.
CSE is the only client of this API, refactor it a bit to pull the query
internally to make changes to DominanceInfo a bit easier.  This commit
also improves comments a bit.
2021-05-30 12:23:39 -07:00
Butygin bb542f2a76 [mlir] StandardToLLVM: option to disable AllocOp lowering
Differential Revision: https://reviews.llvm.org/D103237
2021-05-30 17:50:25 +03:00
Chris Lattner bde21b6245 [Verifier] Significantly speed up IsolatedFromAbove checking. NFC.
The implementation had a couple of problems, including checking
"isProperAncestor" in an inefficient way.  It also recursed into
other "isolated from above" ops.  In the case of CIRCT, we get
three levels of isolated ops:

  mlir::ModuleOp
    firrtl::CircuitOp
       firrtl::FModuleOp

The verification for module would recurse into the circuits and
fmodules checking them.  The verifier hook for circuit would
recurse into all the modules reverifying them, fmoduleop would
then reverify them.  The same happens for mlir::ModuleOp and Func.

While here, fix an old design problem: IsolatedFromAbove checking
was implemented by a method on the Region class, which isn't
actually general and isn't used by anything else.  Move it over
to be a trait impl verifier method like the others and specialize
it for its task.

Differential Revision: https://reviews.llvm.org/D103345
2021-05-28 16:13:45 -07:00
KareemErgawy e493abcf55 [MLIR][SPIRV] Use getAsmResultName(...) hook for ConstantOp.
Implements better naming for results of `spv.Constant` ops by making it
inherit from OpAsmOpInterface and implementing the associated
getAsmResultName(...) hook.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D103152
2021-05-28 09:28:02 +02:00
River Riddle d47dd11071 [mlir] Add support for querying the ModRef behavior from the AliasAnalysis class
This allows for checking if a given operation may modify/reference/or both a given value. Right now this API is limited to Value based memory locations, but we should expand this to include attribute based values at some point. This is left for future work because the rest of the AliasAnalysis API also has this restriction.

Differential Revision: https://reviews.llvm.org/D101673
2021-05-27 13:57:29 -07:00
Eugene Zhulenev d8c84d2a4e [mlir] Async: Add error propagation support to async groups
Depends On D103109

If any of the tokens/values added to the `!async.group` switches to the error state, than the group itself switches to the error state.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D103203
2021-05-27 09:35:11 -07:00
Eugene Zhulenev 39957aa424 [mlir] Add error state and error propagation to async runtime values
Depends On D103102

Not yet implemented:
1. Error handling after synchronous await
2. Error handling for async groups

Will be addressed in the followup PRs

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D103109
2021-05-27 09:28:47 -07:00
thomasraoux b44007bec2 [mlir][gpu] Relax restriction on MMA store op to allow chain of mma ops.
In order to allow large matmul operations using the MMA ops we need to chain
operations this is not possible unless "DOp" and "COp" type have matching
layout so remove the "DOp" layout and force accumulator and result type to
match.
Added a test for the case where the MMA value is accumulated.

Differential Revision: https://reviews.llvm.org/D103023
2021-05-27 09:13:51 -07:00
Nicolas Vasilache ce4f99e7f2 [mlir][Linalg] Add comprehensive bufferization support for subtensor (5/n)
This revision refactors and simplifies the pattern detection logic: thanks to SSA value properties, we can actually look at all the uses of a given value and avoid having to pattern-match specific chains of operations.

A bufferization pattern for subtensor is added and specific inplaceability analysis is implemented for the simple case of subtensor. More advanced use cases will follow.

Differential revision: https://reviews.llvm.org/D102512
2021-05-27 12:48:08 +00:00
Matthias Springer 108ca7a7e7 [mlir] Support dialect-wide canonicalization pattern registration
* Add `hasCanonicalizer` option to Dialect.
* Initialize canonicalizer with dialect-wide canonicalization patterns.
* Add test case to TestDialect.

Dialect-wide canonicalization patterns are useful if a canonicalization pattern does not conceptually associate with any single operation, i.e., it should not be registered as part of an operation's `getCanonicalizationPatterns` function. E.g., this is the case for canonicalization patterns that match an op interface.

Differential Revision: https://reviews.llvm.org/D103226
2021-05-27 17:35:21 +09:00
Alexander Belyaev 281ee42911 [mlir] Add a pass to distribute linalg::TiledLoopOp.
Differential Revision: https://reviews.llvm.org/D103194
2021-05-27 08:45:20 +02:00
Frank Laub b5c3f17e70 [MLIR] Add support for empty IVs to affine.parallel
Allow support for specifying empty IVs in an `affine.parallel`.

For example:

```
affine.parallel () = () to () {
  affine.yield
}
```

Reviewed By: bondhugula, jbruestle

Differential Revision: https://reviews.llvm.org/D102895
2021-05-26 23:45:11 +00:00
Aart Bik 97f15eda4f [mlir][python] Provide "all passes" registration module in Python
Currently, passes are registered on a per-dialect basis, which
provides the smallest footprint obviously. But for prototyping
and experimentation, a convenience "all passes" module is provided,
which registers all known MLIR passes in one run.

Usage in Python:

import mlir.all_passes_registration

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D103130
2021-05-26 15:14:57 -07:00
Alexander Belyaev 74a89cba8c [mlir] Add `distributionTypes` to LinalgTilingOptions.
Differential Revision: https://reviews.llvm.org/D103161
2021-05-26 17:51:38 +02:00
Adrian Kuegel dee46d0829 [mlir] Fold complex.create(complex.re(op), complex.im(op))
Differential Revision: https://reviews.llvm.org/D103148
2021-05-26 14:02:53 +02:00
Chris Lattner aaa2982d71 [MLIR Core] Cache the empty StringAttr like we do for empty dictionaries. NFC.
MLIRContext holds a few special case values that occur frequently like empty
dictionary and NoneType, which allow us to avoid taking locks to get an instance
of them.  Give the empty StringAttr this treatment as well.  This cuts several
percent off compile time for CIRCT.

Differential Revision: https://reviews.llvm.org/D103117
2021-05-25 14:58:17 -07:00
Chris Lattner a004da0d77 [Canonicalize] Switch the default setting to "top down".
This provides a sizable compile time improvement by seeding
the worklist in an order that leads to less iterations of the
worklist.

This patch only changes the behavior of the Canonicalize pass
itself, it does not affect other passes that use the
GreedyPatternRewrite driver

Differential Revision: https://reviews.llvm.org/D103053
2021-05-25 13:42:11 -07:00
Chris Lattner 2f23f9e641 [Canonicalize] Fully parameterize the pass based on config options. NFC.
This allows C++ clients of the Canonicalize pass to specify their own
Config option struct to control how Canonicalize works, increasing reusability.

This also allows controlling these settings for the default Canonicalize pass
using command line options.  This is useful for testing and for playing with
things on the command line.

Differential Revision: https://reviews.llvm.org/D103069
2021-05-25 13:09:51 -07:00
Mathieu Fehr dc2aa47676 [mlir] Add a copy constructor to FailureOr
The copy constructor was missing from FailureOr.

Note that I do not have commit access.

Differential Revision: https://reviews.llvm.org/D98955
2021-05-25 12:10:57 -07:00
Mathieu Fehr 1bf3fd9bb5 [mlir] Use unique_function in AbstractOperation fields
Currently, AbstractOperation fields are function pointers.
Modifying them to unique_function allow them to contain
runtime information.

For instance, this allows operations to be defined at runtime.

Differential Revision: https://reviews.llvm.org/D103031
2021-05-25 11:36:12 -07:00
Alexander Belyaev 2ea6e13bf8 [mlir] Add an optional distributionTypes attribute to TiledLoopOp.
Differential Revision: https://reviews.llvm.org/D103104
2021-05-25 20:04:41 +02:00
Aart Bik ca446e58c8 [sparse][mlir] simplify sparse runtime support library
Removed some of the older raw "MLIRized" versions that are
no longer needed now that the sparse runtime support library
can focus on the proper sparse tensor types rather than the
opague pointer approach of the past. This avoids legacy...

Reviewed By: penpornk

Differential Revision: https://reviews.llvm.org/D102960
2021-05-25 09:39:14 -07:00
Markus Böck 9b99336d5d [mlir][doc] Fix links and references in documentation of Dialects
This patch is the first in a series of patches fixing markdown links and references inside the mlir documentation. I chose to split it in a few reviews to be able to iterate quicker and to ease review.

This patch addresses all broken references to other markdown files and sections inside the Dialects folder.

One change that was also done was to insert '/' between the markdown files and section:
Example:
Builtin.md#integertype
was changed to:
Builtin.md/#integertype

After compilation, hugo then translates the later to jump directly to the integer type section, but not the former. Not inserting the slash would simply jump to just the Builtin page, instead of the integertype section. I therefore changed occurrences of the former version to the later as well.

Differential Revision: https://reviews.llvm.org/D103011
2021-05-25 14:51:15 +02:00
Chris Lattner 64716b2c39 [GreedyPatternRewriter] Introduce a config object that allows controlling internal parameters. NFC.
This exposes the iterations and top-down processing as flags, and also
allows controlling whether region simplification is desirable for a client.
This allows deleting some duplicated entrypoints to
applyPatternsAndFoldGreedily.

This also deletes the Constant Preprocessing pass, which isn't worth it
on balance.

All defaults are all kept the same, so no one should see a behavior change.

Differential Revision: https://reviews.llvm.org/D102988
2021-05-24 12:40:40 -07:00
George d3e6c2ddc3 Surface clone APIs in CAPI
Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D102987
2021-05-24 11:53:00 -07:00
Markus Böck d35bd98651 [mlir][doc] Fix links and references in top level docs directory
This is the fourth and final patch in a series of patches fixing markdown links and references inside the mlir documentation. This patch combined with the other three should fix almost every broken link on mlir.llvm.org as far as I can tell.

This patch in particular addresses all Markdown files in the top level docs directory.

Differential Revision: https://reviews.llvm.org/D103032
2021-05-24 18:43:00 +02:00
Suraj Sudhir 1ceff40df0 [mlir][tosa] Align tensor rank specifications with current spec
Deconstrains several TOSA operators to align with the current TOSA spec, including all the elementwise ops.
Note: some more ops are under consideration for further cleanup; they will follow once the spec has been updated.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D102958
2021-05-24 16:03:42 +00:00
Haruki Imai 000a05fd1a [mlir] Normalize dynamic memrefs with a map of tiled-layout.
Steps for normalizing dynamic memrefs for tiled layout map
1. Check if original map is tiled layout. Only tiled layout is supported.
2. Create normalized memrefType. Dimensions that include dynamic dimensions
   in the map output will be dynamic dimensions.
3. Create new maps to calculate each dimension size of new memref.
   In tiled layout, the dimension size can be calculated by replacing
    "floordiv <tile size>" with "ceildiv <tile size>" and
    "mod <tile size>" with "<tile size>".
4. Create AffineApplyOp to apply the new maps. The output of AffineApplyOp is
   dynamicSizes for new AllocOp.
5. Add the new dynamic sizes in new AllocOp.

This patch also set MemRefsNormalizable trant in CastOp and DimOp since
they used with dynamic memrefs.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D97655
2021-05-24 08:39:36 +05:30
Chris Lattner 81467f500f [IR] Add a Location to BlockArgument
This adds the ability to specify a location when creating BlockArguments.
Notably Value::getLoc() will return this correctly, which makes diagnostics
more precise (e.g. the example in test-legalize-type-conversion.mlir).

This is currently optional to avoid breaking any existing code - if
absent, the BlockArgument defaults to using the location of its enclosing
operation (preserving existing behavior).

The bulk of this change is plumbing location tracking through the parser
and printer to make sure it can round trip (in -mlir-print-debuginfo
mode).  This is complete for generic operations, but requires manual
adoption for custom ops.

I added support for function-like ops to round trip their argument
locations - they print correctly, but when parsing the locations are
dropped on the floor.  I intend to fix this, but it will require more
invasive plumbing through "function_like_impl" stuff so I think it
best to split it out to its own patch.

This is a reapply of the patch here: https://reviews.llvm.org/D102567
with an additional change: we now never defer block argument locations,
guaranteeing that we can round trip correctly.

This isn't required in all cases, but allows us to hill climb here and
works around unrelated bugs like https://bugs.llvm.org/show_bug.cgi?id=50451

Differential Revision: https://reviews.llvm.org/D102991
2021-05-23 14:10:00 -07:00
Markus Böck 9a6eb4968e [mlir][doc] Fix links and indentation of mlir::ModuleOp description
All lines after the first are currently indented by one char further to the left than the first line. This leads to the first character of each sentence being cut from the resulting Markdown file after compilation. The text also contains 3 references to sections of other markdown files. One was missing the file, while the other two had outdated files, leading to 404 errors in the documentation.

Differential Revision: https://reviews.llvm.org/D102983
2021-05-23 20:01:54 +02:00
Butygin 0dd36f81b9 [mlir][linalg][nfc] Fix signed/unsigned comparison warning in header
Differential Revision: https://reviews.llvm.org/D102968
2021-05-22 17:18:01 +03:00
Butygin 9afbca746b [mlir] ConvertStandardToLLVM: make AllocLikeOpLowering public
It is useful for someone who wants to implement custom AllocOp LLVM lowering

Differential Revision: https://reviews.llvm.org/D102932
2021-05-22 12:57:45 +03:00
Aart Bik c194b49c9c [mlir][sparse] add full dimension ordering support
This revision completes the "dimension ordering" feature
of sparse tensor types that enables the programmer to
define a preferred order on dimension access (other than
the default left-to-right order). This enables e.g. selection
of column-major over row-major storage for sparse matrices,
but generalized to any rank, as in:

dimOrdering = affine_map<(i,j,k,l,m,n,o,p) -> (p,o,j,k,i,l,m,n)>

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D102856
2021-05-21 12:35:13 -07:00
Alexander Belyaev 335fa18028 [mlir] NFC: Expose tiled_loop->scf pattern.
Differential Revision: https://reviews.llvm.org/D102921
2021-05-21 18:19:00 +02:00
Alexander Belyaev 9ecc8178d7 [mlir] Add support for fusion into TiledLoopOp.
Differential Revision: https://reviews.llvm.org/D102722
2021-05-21 18:13:45 +02:00
Navdeep Kumar eaaf7a6a09 [MLIR][GPU][NVVM] Add conversion of warp synchronous matrix-multiply accumulate GPU ops
Add conversion of warp synchronous matrix-multiply
accumulate GPU ops
Add conversion of warp synchronous matrix-multiply accumulate GPU ops to
NVVM ops. The following conversions are added :-
  1.) subgroup_mma_load_matrix -> wmma.m16n16k16.load.[a,b,c]..row.stride
  2.) subgroup_mma_store_matrix -> wmma.m16n16k16.store.d.[f16,f32].row.stride
  3.) subgroup_mma_compute -> wmma.m16n16k16.mma.row.row.[f16,f32].[f16,f32]

Reviewed By: bondhugula, ftynse

Differential Revision: https://reviews.llvm.org/D95331
2021-05-21 21:20:33 +05:30
Stephan Herhut 884a6291f0 [mlir][linalg] Add scalar operands inlining pattern
This pattern inlines operands to a linalg.generic operation that use a constant
index and hence are loop-invariant scalars. This reduces the number of
linalg.generic operands and unlocks some canonicalizations that rely on seeing
an explicit tensor.extract.

Differential Revision: https://reviews.llvm.org/D102682
2021-05-21 15:23:28 +02:00
Adrian Kuegel 28844212fe [mlir] Add additional traits to EqualOp and NotEqualOp.
This matches the traits of the other binary ops.

Differential Revision: https://reviews.llvm.org/D102916
2021-05-21 15:04:26 +02:00
Nicolas Vasilache 8eb18a0f3e [mlir][Standard] NFC - Drop remaining EDSC usage
Drop the remaining EDSC subdirectories and update all uses.

Differential Revision: https://reviews.llvm.org/D102911
2021-05-21 10:40:39 +00:00
Adrian Kuegel a1a429dc1d [mlir] NFC - Use builder.getI1Type() 2021-05-21 11:44:57 +02:00
Adrian Kuegel 7eb85cdb10 [mlir] Add convenience builders for EqualOp and NotEqualOp.
The result type is always I1. This additional builder allows to omit the result type.

Differential Revision: https://reviews.llvm.org/D102905
2021-05-21 11:32:25 +02:00
Nicolas Vasilache e84a9b9bb3 [mlir][Affine] NFC - Drop Affine EDSC usage
Drop the Affine dialect EDSC subdirectory and update all uses.

Differential Revision: https://reviews.llvm.org/D102878
2021-05-20 21:45:45 +00:00
Nicolas Vasilache e3cf7c88c4 [mlir][MemRef] NFC - Drop MemRef EDSC usage
Drop the MemRef dialect EDSC subdirectory and update all uses.

Differential Revision: https://reviews.llvm.org/D102868
2021-05-20 20:13:58 +00:00
Nicolas Vasilache 4519ca3d2e [mlir][Linalg] NFC - Drop Linalg EDSC usage
Drop the Linalg dialect EDSC subdirectory and update all uses.

Differential Revision: https://reviews.llvm.org/D102848
2021-05-20 15:33:56 +00:00
Adrian Kuegel a28fe17d73 [mlir] Add EqualOp and NotEqualOp to complex dialect. 2021-05-20 13:25:07 +02:00
Nicolas Vasilache ef33c6e3ce [mlir][Linalg] Drop spurious usage of OperationFolder
Instead, use createOrFold builders which result in more static information available.

Differential Revision: https://reviews.llvm.org/D102832
2021-05-20 09:17:58 +00:00
River Riddle 92469ca027 [mlir] Refactor the implementation of pass crash reproducers
The current implementation has several key limitations and weirdness, e.g local reproducers don't support dynamic pass pipelines, error messages don't include the passes that failed, etc. This revision refactors the implementation to support more use cases, and also be much cleaner.

The main change in this revision, aside from moving the implementation out of Pass.cpp and into its own file, is the addition of a crash recovery pass instrumentation. For local reproducers, this instrumentation handles setting up the recovery context before executing each pass. For global reproducers, the instrumentation is used to provide a more detailed error message, containing information about which passes are running and on which operations.

Example of new message:

```
error: Failures have been detected while processing an MLIR pass pipeline
note: Pipeline failed while executing [`TestCrashRecoveryPass` on 'module' operation: @foo]: reproducer generated at `crash-recovery.mlir.tmp`
```

Differential Revision: https://reviews.llvm.org/D101854
2021-05-19 16:59:53 -07:00
River Riddle 64ce90e1af [mlir] Add a new `print-ir-after-failure` IR pass printing flag
This flag will print the IR after a pass only in the case where the pass failed. This can be useful to more easily view the invalid IR, without needing to print after every pass in the pipeline.

Differential Revision: https://reviews.llvm.org/D101853
2021-05-19 16:54:20 -07:00
River Riddle 3b7f8daed4 [mlir] Properly align StorageUniquer::BaseStorage to fix 32 bit build
We allow stealing up to 3 bits of pointers to BaseStorage, so we need to make sure that we align by at least 8.
2021-05-19 13:46:43 -07:00
Marius Brehler f878e1af9f [mlir] Harmonize TOSA include guards
Reviewed By: sjarus

Differential Revision: https://reviews.llvm.org/D102802
2021-05-19 22:45:06 +02:00
Sean Silva 35454268cf [mlir][CAPI] Expose [u]int8 DenseElementsAttr.
Also, fix a small typo where the "unsigned" splat variants were not
being created with an unsigned type.

Differential Revision: https://reviews.llvm.org/D102797
2021-05-19 13:41:44 -07:00
Marius Brehler 745ddd27ea [mlir] Add include guard to TOSA tblgen passes
Reviewed By: sjarus, stellaraccident

Differential Revision: https://reviews.llvm.org/D102800
2021-05-19 22:02:31 +02:00
Nicolas Vasilache 84a880e1e2 [mlir][SCF] NFC - Drop SCF EDSC usage
Drop the SCF dialect EDSC subdirectory and update all uses.

Differential Revision: https://reviews.llvm.org/D102780
2021-05-19 15:52:14 +00:00
Tobias Gysi 9a2769db80 [mir][Python][linalg] Support OpDSL extensions in C++.
The patch extends the yaml code generation to support the following new OpDSL constructs:
- captures
- constants
- iteration index accesses
- predefined types
These changes have been introduced by revision
https://reviews.llvm.org/D101364.

Differential Revision: https://reviews.llvm.org/D102075
2021-05-19 13:36:56 +00:00
Nicolas Vasilache 6825bfe23e [mlir][Vector] NFC - Drop vector EDSC usage
Drop the vector dialect EDSC subdirectory and update all uses.
2021-05-19 12:44:38 +00:00
Matthias Springer fb7ec1f187 [mlir] Use VectorTransferPermutationMapLoweringPatterns in VectorToSCF
VectorTransferPermutationMapLoweringPatterns can be enabled via a pass option. These additional patterns lower permutation maps to minor identity maps with broadcasting, if possible, allowing for more efficient vector load/stores. The option is deactivated by default.

Differential Revision: https://reviews.llvm.org/D102593
2021-05-19 14:46:19 +09:00
Wenyi Zhao 851d02f61e Enhance InferShapedTypeOpInterface to make it accessible during dialect conversion
Original interfaces are not safe to be called during dialect conversion.
This is because some ops (e.g. `dynamic_reshape(input, target_shape)`)
depend on the values of their operands to calculate the output shape.
However the operands may be out of reach during dialect conversion (e.g.
converting from tensor world to buffer world). This patch provides a new
kind of interface which accpets user-provided operands to solve this
problem.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D102317
2021-05-19 02:51:14 +00:00
Richard Smith 80d981eda6 Revert "[IR] Add a Location to BlockArgument." and follow-on commit
"[mlir] Speed up Lexer::getEncodedSourceLocation"

This reverts commit 3043be9d2d and commit
861d69a525.

This change resulted in printing textual MLIR that can't be parsed; see
review thread https://reviews.llvm.org/D102567 for details.
2021-05-18 19:26:00 -07:00
Thomas Köppe 58369fce30 Add a helper function to convert LogicalResult to int for return from main
At present, a lot of code contains main function bodies like "return failed(mlir::MlirOptMain(...);". This is unfortunate for two reasons: a) it uses ADL, which is maybe not what the free "failed" function was designed for; and b) it is a bit awkward to read, requring the reader to both understand the boolean nature of the value and the semantics of main's return value. (And it's also not portable, since 1 is not a portable success value.)

The replacement code, `return mlir::AsMainReturnCode(mlir::MlirOptMain(...))` is a bit more self-explanatory.

The change applies the new function to a few internal uses of MlirOptMain, too.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D102641
2021-05-19 00:12:39 +00:00
River Riddle 2257e4a70e [mlir] Allow derived rewrite patterns to define a non-virtual `initialize` hook
This is a hook that allows for providing custom initialization of the pattern, e.g. if it has bounded recursion, setting the debug name, etc., without needing to define a custom constructor. A non-virtual hook was chosen to avoid polluting the vtable with code that we really just want to be inlined when constructing the pattern. The alternative to this would be to just define a constructor for each pattern, this unfortunately creates a lot of otherwise unnecessary boiler plate for a lot of patterns and a hook provides a much simpler/cleaner interface for the very common case.

Differential Revision: https://reviews.llvm.org/D102440
2021-05-18 14:40:32 -07:00
Chris Lattner 855b42ddd0 [IntegerAttr] Add helpers for working with LLVM's APSInt type.
The FIRRTL dialect in CIRCT uses inherently signful types, and APSInt
is the best way to model that.  Add a couple of helpers that make it
easier to work with an IntegerAttr that carries a sign.

This follows the example of getZExt() and getSExt() which assert when
the underlying type of the attribute is unexpected.  In this case
we assert fail when the underlying type of the attribute is signless.

This is strictly additive, so it is NFC.  It is tested in the CIRCT
repo.

Differential Revision: https://reviews.llvm.org/D102701
2021-05-18 10:51:52 -07:00
Chris Lattner 3043be9d2d [IR] Add a Location to BlockArgument.
This adds the ability to specify a location when creating BlockArguments.
Notably Value::getLoc() will return this correctly, which makes diagnostics
more precise (e.g. the example in test-legalize-type-conversion.mlir).

This is currently optional to avoid breaking any existing code - if
absent, the BlockArgument defaults to using the location of its enclosing
operation (preserving existing behavior).

The bulk of this change is plumbing location tracking through the parser
and printer to make sure it can round trip (in -mlir-print-debuginfo
mode).  This is complete for generic operations, but requires manual
adoption for custom ops.

I added support for function-like ops to round trip their argument
locations - they print correctly, but when parsing the locations are
dropped on the floor.  I intend to fix this, but it will require more
invasive plumbing through "function_like_impl" stuff so I think it
best to split it out to its own patch.

Differential Revision: https://reviews.llvm.org/D102567
2021-05-18 10:18:04 -07:00
Adrian Kuegel fa765a0944 [mlir] Add folder for complex.ReOp and complex.ImOp.
Now that complex constants are supported, we can also fold.

Differential Revision: https://reviews.llvm.org/D102616
2021-05-18 11:27:23 +02:00
River Riddle e2e1a78abc [mlir][NFC] Remove stale `createLowerAffinePass` declaration
This pass isn't defined in the Transforms/ library anymore.
2021-05-17 13:25:31 -07:00
Jacques Pienaar 24bf554b10 Add type function for ConstShape op.
- Enables inferring return type for ConstShape, takes into account valid return types;
- The compatible return type function could be reused, leaving that for next use refactoring;

Differential Revision: https://reviews.llvm.org/D102182
2021-05-17 11:47:19 -07:00
Aart Bik 5879da496c [mlir][sparse] replace experimental flag with inplace attribute
The experimental flag for "inplace" bufferization in the sparse
compiler can be replaced with the new inplace attribute. This gives
a uniform way of expressing the more efficient way of bufferization.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D102538
2021-05-17 11:43:44 -07:00
Chris Lattner 648f34a284 Merge with mainline.
Differential Revision: https://reviews.llvm.org/D102636
2021-05-17 11:15:10 -07:00
Mehdi Amini 43f6e04258 Make `mlir::OpState::operator bool` explicit
This change makes the conversion of an mlir::OpState to bool `explicit`. Idiomatic boolean uses continue to work as before, but questionable implicit uses (e.g. accumulating over a range of OpStates to count "true" states) become ill-formed. This makes the class interface a lilttle less error-prone.

I tested this change on our internal (fairly large) codebase, and only one fix was needed, which was ultimately an improvement of the affected code.

Reviewed By: rriddle, mehdi_amini

Differential Revision: https://reviews.llvm.org/D101989
2021-05-17 17:29:25 +00:00
Matthias Springer 2c9688d201 [mlir] Improve TransferOp verifier: broadcasts are in_bounds
Broadcast dimensions of vector transfer ops are always in-bounds. This is consistent with the fact that the starting position of a transfer is always in-bounds.

Differential Revision: https://reviews.llvm.org/D102566
2021-05-17 22:35:44 +09:00
Stephan Herhut db81e88f25 [mlir][memref] Mark memref.buffer_cast as NoSideEffect
This brings it in line with the bultin unrealized_conversion_cast,
which memref.buffer_cast is a specialized version of.

Differential Revision: https://reviews.llvm.org/D102608
2021-05-17 14:20:00 +02:00
Adrian Kuegel 967f07f547 Revert "[mlir] Add folder for complex.ReOp and complex.ImOp."
This reverts commit 6b49834d65.

Some tests fail.
2021-05-17 13:49:42 +02:00
Adrian Kuegel 6b49834d65 [mlir] Add folder for complex.ReOp and complex.ImOp.
Now that complex constants are supported, we can also fold.

Differential Revision: https://reviews.llvm.org/D102609
2021-05-17 13:35:51 +02:00
Adam Paszke d89602ed62 Add `mlirModuleFromOperation` to C API
At the moment `MlirModule`s can be converted to `MlirOperation`s, but not
the other way around (at least not without going around the C API). This
makes it impossible to e.g. run passes over a `ModuleOp` created through
`mlirOperationCreate`.

Reviewed By: nicolasvasilache, mehdi_amini

Differential Revision: https://reviews.llvm.org/D102497
2021-05-17 10:14:16 +00:00
Julian Gross 1fbb484ea4 [WIP][mlir] Resolve memref dependency in canonicalize pass.
Splitting the memref dialect lead to an introduction of several dependencies
to avoid compilation issues. The canonicalize pass also depends on the
memref dialect, but it shouldn't. This patch resolves the dependencies
and the unintuitive includes are removed. However, the dependency moves
to the constructor of the std dialect.

Differential Revision: https://reviews.llvm.org/D102060
2021-05-17 11:33:38 +02:00
Tobias Gysi 7c16f93c44 [mlir][linalg] Remove template parameter from loop lowering.
Replace the templated linalgLowerOpToLoops method by three specialized methods linalgOpToLoops, LinalgOpToParallelLoops, and linalgOpToAffineLoops.

Differential Revision: https://reviews.llvm.org/D102324
2021-05-17 09:31:53 +00:00
Adrian Kuegel 5ef21506b9 Add support for complex constants to MLIR core.
BEGIN_PUBLIC
Add support for complex constants to MLIR core.
END_PUBLIC

Differential Revision: https://reviews.llvm.org/D101908
2021-05-17 09:12:39 +02:00
Uday Bondhugula 185ce8cdfc [MLIR][PYTHON] Provide opt level for ExecutionEngine Python binding
Provide an option to specify optimization level when creating an
ExecutionEngine via the MLIR JIT Python binding. Not only is the
specified optimization level used for code generation, but all LLVM
optimization passes at the optimization level are also run prior to
machine code generation (akin to the mlir-cpu-runner tool).

Default opt level continues to remain at level two (-O2).

Contributions in part from Prashant Kumar <prashantk@polymagelabs.com>
as well.

Differential Revision: https://reviews.llvm.org/D102551
2021-05-16 13:58:49 +05:30
Uday Bondhugula 1d2ce7d6d6 [MLIR][NFC] Fix clang-tidy warnings in IntegerSet.h
Fix clang-tidy warnings and some comments in IntegerSet.h. NFC.

Differential Revision: https://reviews.llvm.org/D102387
2021-05-16 12:26:12 +05:30
Ian Bearman 0816b96a10 Allow same memory space for SRC and DST of dma_start operations
This change allows the SRC and DST of dma_start operations to be located in the
    same memory space. This applies to both the Affine dialect and Memref dialect
    versions of these Ops. The documention has been updated to reflect this by
    explicitly stating overlapping memory locations are not supported (undefined
    behavior).

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D102274
2021-05-14 10:40:15 -07:00
Benoit Jacob e0a88db545 Fix some typos.
Fix some typos

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D102503
2021-05-14 21:34:09 +05:30
Matthias Springer 2ca887de6e [mlir] VectorToSCF target rank is a pass option
Make "target rank" a pass option of VectorToSCF.

Depends On D102101

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D102123
2021-05-14 10:30:43 +09:00
Nicolas Vasilache 1e01a8919f [mlir][Linalg] Add ComprehensiveBufferize for functions(step 1/n)
This is the first step towards upstreaming comprehensive bufferization following the
discourse post: https://llvm.discourse.group/t/rfc-linalg-on-tensors-update-and-comprehensive-bufferization-rfc/3373/6.

This first commit introduces a basic pass for bufferizing within function boundaries,
assuming that the inplaceable function boundaries have been marked as such.

Differential revision: https://reviews.llvm.org/D101693
2021-05-13 22:24:40 +00:00
Sean Silva 12874e93a1 [mlir][NFC] Add helper for common pattern of replaceAllUsesExcept
This covers the extremely common case of replacing all uses of a Value
with a new op that is itself a user of the original Value.

This should also be a little bit more efficient than the
`SmallPtrSet<Operation *, 1>{op}` idiom that was being used before.

Differential Revision: https://reviews.llvm.org/D102373
2021-05-13 12:42:10 -07:00
Weiwei Li cd0eeb52ad [mlir][spirv] Define spv.ImageQuerySize operation
Support OpImageQuerySize in spirv dialect

co-authored-by: Alan Liu <alanliu.yf@gmail.com>

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D102029
2021-05-13 13:17:08 -04:00
Tobias Gysi cf194da1bb [mlir][linalg] Remove IndexedGenericOp support from FusionOnTensors...
after introducing the IndexedGenericOp to GenericOp canonicalization (https://reviews.llvm.org/D101612).

Differential Revision: https://reviews.llvm.org/D102163
2021-05-13 14:57:16 +00:00
Matthias Springer 0f24163870 [mlir] Replace vector-to-scf with progressive-vector-to-scf
Depends On D102388

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D102101
2021-05-13 23:27:31 +09:00
Matthias Springer 9b77be5583 [mlir] Unrolled progressive-vector-to-scf.
Instead of an SCF for loop, these pattern generate fully unrolled loops with no temporary buffer allocations.

Differential Revision: https://reviews.llvm.org/D101981
2021-05-13 13:08:48 +09:00
Matthias Springer 864adf399e [mlir] Allow empty position in vector.insert and vector.extract
Such ops are no-ops and are folded to their respective `source`/`vector` operand.

Differential Revision: https://reviews.llvm.org/D101879
2021-05-13 12:54:18 +09:00
Matthias Springer c52cbe63e4 [mlir] Fix masked vector transfer ops with broadcasts
Broadcast dimensions of a vector transfer op have no corresponding dimension in the mask vector. E.g., a 2-D TransferReadOp, where one dimension is a broadcast, can have a 1-D `mask` attribute.

This commit also adds a few additional transfer op integration tests for various combinations of broadcasts, masking, dim transposes, etc.

Differential Revision: https://reviews.llvm.org/D101745
2021-05-13 12:46:03 +09:00
Matthias Springer 6555e53ab0 Revert "[mlir] Fix masked vector transfer ops with broadcasts"
This reverts commit c9087788f7.

Accidentally pushed old version of the commit.
2021-05-13 11:55:00 +09:00
Matthias Springer c9087788f7 [mlir] Fix masked vector transfer ops with broadcasts
Broadcast dimensions of a vector transfer op have no corresponding dimension in the mask vector. E.g., a 2-D TransferReadOp, where one dimension is a broadcast, can have a 1-D `mask` attribute.

This commit also adds a few additional transfer op integration tests for various combinations of broadcasts, masking, dim transposes, etc.

Differential Revision: https://reviews.llvm.org/D101745
2021-05-13 11:37:36 +09:00
Aart Bik 58d12332a4 [mlir][sparse][capi][python] add sparse tensor passes
First set of "boilerplate" to get sparse tensor
passes available through CAPI and Python.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D102362
2021-05-12 16:40:50 -07:00
MaheshRavishankar b7911e80d6 [mlir][Linalg] Add interface methods to get lhs and rhs of contraction
Differential Revision: https://reviews.llvm.org/D102301
2021-05-12 16:07:10 -07:00
River Riddle b3911cdfc8 [mlir-lsp-server] Add support for sending diagnostics to the client
This allows for diagnostics emitted during parsing/verification to be surfaced to the user by the language client, as opposed to just being emitted to the logs like they are now.

Differential Revision: https://reviews.llvm.org/D102293
2021-05-12 13:02:25 -07:00
Suraj Sudhir 4b01435230 [mlir][tosa] Remove tosa.identityn operator
Removes the identityn operator from TOSA MLIR definition.
Removes TosaToLinAlg mappings

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D102329
2021-05-12 12:46:22 -07:00
Valentin Clement 113b807017 [mlir][openacc] Add OpenACC translation to LLVM IR (enter_data op create/copyin)
This patch begins to translate acc.enter_data operation to call to tgt runtime call.
It currently only translate create/copyin operands of memref type. This acts as a basis to add support
for FIR types in the Flang/OpenACC support. It follows more or less a similar path than clang
with `omp target enter data map` directives.
This patch is taking a different approach than D100678 and perform a translation to LLVM IR
and make use of the OpenMPIRBuilder instead of doing a conversion to the LLVMIR dialect.

OpenACC support in Flang will rely on the current OpenMP runtime where 1:1 lowering can be
applied. Some extension will be added where features are not available yet.

Big part of this code will be shared for other standalone data operations in the OpenACC
dialect such as acc.exit_data and acc.update.

It is likely that parts of the lowering can also be shared later with the ops for
standalone data directives in the OpenMP dialect when they are introduced.

This is an initial translation and it probably needs more work.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D101504
2021-05-12 13:41:14 -04:00
Fabian Schuiki 33f908c428
[MLIR] Factor pass timing out into a dedicated timing manager
This factors out the pass timing code into a separate `TimingManager`
that can be plugged into the `PassManager` from the outside. Users are
able to provide their own implementation of this manager, and use it to
time additional code paths outside of the pass manager. Also allows for
multiple `PassManager`s to run and contribute to a single timing report.

More specifically, moves most of the existing infrastructure in
`Pass/PassTiming.cpp` into a new `Support/Timing.cpp` file and adds a
public interface in `Support/Timing.h`. The `PassTiming` instrumentation
becomes a wrapper around the new timing infrastructure which adapts the
instrumentation callbacks to the new timers.

Reviewed By: rriddle, lattner

Differential Revision: https://reviews.llvm.org/D100647
2021-05-12 18:14:51 +02:00
Valentin Clement 6110b667b0 [mlir][openacc] Conversion of data operand to LLVM IR dialect
Add a conversion pass to convert higher-level type before translation.
This conversion extract meangingful information and pack it into a struct that
the translation (D101504) will be able to understand.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D102170
2021-05-12 11:34:15 -04:00
Tobias Gysi 06bb9cf30d [mlir][linalg] Remove IndexedGenericOp support from LinalgInterchangePattern...
after introducing the IndexedGenericOp to GenericOp canonicalization (https://reviews.llvm.org/D101612).

Differential Revision: https://reviews.llvm.org/D102245
2021-05-12 13:01:37 +00:00
Tobias Gysi 0fb364a97e [mlir][linalg] Remove IndexedGenericOp support from LinalgToStandard...
after introducing the IndexedGenericOp to GenericOp canonicalization (https://reviews.llvm.org/D101612).

Differential Revision: https://reviews.llvm.org/D102236
2021-05-12 11:56:07 +00:00
Ulysse Beaugnon 27b2bd7601 [MLIR] Enable conversion from llvm::SMLoc to mlir::Location with OpAsmParser.
DialectAsmParser already allows converting an llvm::SMLoc location to a
mlir::Location location. This commit adds the same functionality to OpAsmParser.
Implementation is copied from DialectAsmParser.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D102165
2021-05-12 09:08:32 +02:00
Dumitru Potop 9a0ea5994b [mlir] Support alignment in LLVM dialect GlobalOp
First step in adding alignment as an attribute to MLIR global definitions. Alignment can be specified for global objects in LLVM IR. It can also be specified as a named attribute in the LLVMIR dialect of MLIR. However, this attribute has no standing and is discarded during translation from MLIR to LLVM IR. This patch does two things: First, it adds the attribute to the syntax of the llvm.mlir.global operation, and by doing this it also adds accessors and verifications. The syntax is "align=XX" (with XX being an integer), placed right after the value of the operation. Second, it allows transforming this operation to and from LLVM IR. It is checked whether the value is an integer power of 2.

Reviewed By: ftynse, mehdi_amini

Differential Revision: https://reviews.llvm.org/D101492
2021-05-12 09:07:20 +02:00
River Riddle 731206f368 [mlir] Move move capture in SparseElementsAttr::getValues
This was a TODO for the move to C++14. Now that the move has been completed, we can resolve it.
2021-05-11 12:09:42 -07:00
Sean Silva 49755871ad [mlir][ODS]: Add per-op cppNamespace.
This is useful for dialects that have logical subparts.

Differential Revision: https://reviews.llvm.org/D102200
2021-05-11 10:48:05 -07:00
Chris Lattner 2b09a89daf [OpAsmParser] Refactor parseOptionalInteger to support wide integers, NFC.
OpAsmParser (and DialectAsmParser) supports a pair of
parseInteger/parseOptionalInteger methods, which allow parsing a bare
integer into a C type of your choice (e.g. int8_t) using templates.  It
was implemented in terms of a virtual method call that is hard coded to
int64_t because "that should be big enough".

Change the virtual method hook to return an APInt instead.  This allows
asmparsers for custom ops to parse large integers if they want to, without
changing any of the clients of the fixed size C API.

Differential Revision: https://reviews.llvm.org/D102120
2021-05-10 22:35:42 -07:00
Aart Bik bf812ea484 [mlir][linalg] remove the -now- obsolete sparse support in linalg
All glue and clutter in the linalg ops has been replaced by proper
sparse tensor type encoding. This code is no longer needed. Thanks
to ntv@ for giving us a temporary home in linalg.

So long, and thanks for all the fish.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D102098
2021-05-10 16:49:33 -07:00
Benjamin Kramer 7b52aeadfa [mlir][Tensor] Add folding for tensor.from_elements
This trivially folds into a constant when all operands are constant.

Differential Revision: https://reviews.llvm.org/D102199
2021-05-11 00:42:45 +02:00
Stella Laurenzo a2c8aebd8f [mlir][Python] Finish adding RankedTensorType support for encoding.
Differential Revision: https://reviews.llvm.org/D102184
2021-05-10 20:39:16 +00:00
Aart Bik 96a23911f6 [mlir][sparse] complete migration to sparse tensor type
A very elaborate, but also very fun revision because all
puzzle pieces are finally "falling in place".

1. replaces lingalg annotations + flags with proper sparse tensor types
2. add rigorous verification on sparse tensor type and sparse primitives
3. removes glue and clutter on opaque pointers in favor of sparse tensor types
4. migrates all tests to use sparse tensor types

NOTE: next CL will remove *all* obsoleted sparse code in Linalg

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D102095
2021-05-10 12:55:22 -07:00
Stella Laurenzo f13893f66a [mlir][Python] Upstream the PybindAdaptors.h helpers and use it to implement sparse_tensor.encoding.
* The PybindAdaptors.h file has been evolving across different sub-projects (npcomp, circt) and has been successfully used for out of tree python API interop/extensions and defining custom types.
* Since sparse_tensor.encoding is the first in-tree custom attribute we are supporting, it seemed like the right time to upstream this header and use it to define the attribute in a way that we can support for both in-tree and out-of-tree use (prior, I had not wanted to upstream dead code which was not used in-tree).
* Adapted the circt version of `mlir_type_subclass`, also providing an `mlir_attribute_subclass`. As we get a bit of mileage on this, I would like to transition the builtin types/attributes to this mechanism and delete the old in-tree only `PyConcreteType` and `PyConcreteAttribute` template helpers (which cannot work reliably out of tree as they depend on internals).
* Added support for defaulting the MlirContext if none is passed so that we can support the same idioms as in-tree versions.

There is quite a bit going on here and I can split it up if needed, but would prefer to keep the first use and the header together so sending out in one patch.

Differential Revision: https://reviews.llvm.org/D102144
2021-05-10 17:15:43 +00:00
Stella Laurenzo bcfa7baec8 [mlir][CAPI] Add CAPI bindings for the sparse_tensor dialect.
* Adds dialect registration, hand coded 'encoding' attribute and test.
* An MLIR CAPI tablegen backend for attributes does not exist, and this is a relatively complicated case. I opted to hand code it in a canonical way for now, which will provide a reasonable blueprint for building out the tablegen version in the future.
* Also added a (local) CMake function for declaring new CAPI tests, since it was getting repetitive/buggy.

Differential Revision: https://reviews.llvm.org/D102141
2021-05-10 16:54:56 +00:00
Julian Gross fc253e69f9 Fixed bug in buffer deallocation pass using unranked memref types.
In the buffer deallocation pass, unranked memref types are not properly supported.
After investigating this issue, it turns out that the Clone and Dealloc operation
does not support unranked memref types in the current implementation.
This patch adds the missing feature and enables the transformation of any memref
type.

This patch solves this bug: https://bugs.llvm.org/show_bug.cgi?id=48385

Differential Revision: https://reviews.llvm.org/D101760
2021-05-10 10:50:29 +02:00
Frederik Gossen a81e45b8bc [MLIR][Shape] Concretize broadcast result type if possible
As a canonicalization, infer the resulting shape rank if possible.

Differential Revision: https://reviews.llvm.org/D102068
2021-05-10 10:24:08 +02:00
Alex Zinenko 72d013dd73 [mlir] OpenMP-to-LLVM: properly set outer alloca insertion point
Previously, the OpenMP to LLVM IR conversion was setting the alloca insertion
point to the same position as the main compuation when converting OpenMP
`parallel` operations. This is problematic if, for example, the `parallel`
operation is placed inside a loop and would keep allocating on stack on each
iteration leading to stack overflow.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D101307
2021-05-10 10:04:52 +02:00
Adrian Kuegel 9ba661f912 [mlir] Fix compile error.
Inside a templated function, other class members need to be called with
this->.
Otherwise we get: explicit qualification required to use member
'setDebugName' from dependent base class.
2021-05-10 07:48:45 +02:00
Chia-hung Duan 34b5482b33 Support NativeCodeCall binding in rewrite pattern.
We are able to bind the result from native function while rewriting
pattern. In matching pattern, if we want to get some values back, we can
do that by passing parameter as return value placeholder. Besides, add
the semantic of '$_self' in NativeCodeCall while matching, it'll be the
operation that defines certain operand.

Differential Revision: https://reviews.llvm.org/D100746
2021-05-10 09:29:27 +08:00
Vinayaka Bandishti 9610a2d753 [MLIR] Add memref dialect dependency for affine fusion pass
For `AffineLoopFusion` pass, add `memref` dialect as a dependent
dialect. Since the fusion pass can create `memref::AllocOp`s, the
dialect must be registered in its dependent dialects.

The missing dependency was not discovered until now because the above
said op creation happes only when the input already has
`memref::AllocOp`s in it, and all dialects in the input are
automatically added to the context.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D102104
2021-05-08 20:12:33 +05:30
Butygin e2a7764481 [mlir] Debug print pattern before and after matchAndRewrite call
Motivation: we have passes with lot of rewrites and when one one them segfaults or asserts, it is very hard to find waht exactly pattern failed without debug info.

Differential Revision: https://reviews.llvm.org/D101443
2021-05-08 12:00:36 +03:00
River Riddle 53b946aa63 [mlir] Refactor the representation of function-like argument/result attributes.
The current design uses a unique entry for each argument/result attribute, with the name of the entry being something like "arg0". This provides for a somewhat sparse design, but ends up being much more expensive (from a runtime perspective) in-practice. The design requires building a string every time we lookup the dictionary for a specific arg/result, and also requires N attribute lookups when collecting all of the arg/result attribute dictionaries.

This revision restructures the design to instead have an ArrayAttr that contains all of the attribute dictionaries for arguments and another for results. This design reduces the number of attribute name lookups to 1, and allows for O(1) lookup for individual element dictionaries. The major downside is that we can end up with larger memory usage, as the ArrayAttr contains an entry for each element even if that element has no attributes. If the memory usage becomes too problematic, we can experiment with a more sparse structure that still provides a lot of the wins in this revision.

This dropped the compilation time of a somewhat large TensorFlow model from ~650 seconds to ~400 seconds.

Differential Revision: https://reviews.llvm.org/D102035
2021-05-07 19:32:31 -07:00
River Riddle 5c84195b8c [mlir] Add hover support to mlir-lsp-server
This provides information when the user hovers over a part of the source .mlir file. This revision adds the following hover behavior:
* Operation:
  - Shows the generic form.
* Operation Result:
  - Shows the parent operation name, result number(s), and type(s).
* Block:
  - Shows the parent operation name, block number, predecessors, and successors.
* Block Argument:
  - Shows the parent operation name, parent block, argument number, and type.

Differential Revision: https://reviews.llvm.org/D101113
2021-05-07 18:09:01 -07:00
Emilio Cota 21db1e3b01 [mlir][docs] remove stale statement about index type in vectors
b614ada0e8 ("[mlir] add support for index type in vectors.") removed
this limitation.

Differential Revision: https://reviews.llvm.org/D102081
2021-05-07 19:25:17 +00:00
Tobias Gysi f31531a30b [mlir][linalg] Remove redundant indexOp builder.
Remove the builder signature taking a signed dimension identifier.

Reviewed By: ergawy

Differential Revision: https://reviews.llvm.org/D102055
2021-05-07 14:22:12 +00:00
Tres Popp faab8c140a [mlir] Rename BufferAliasAnalysis to BufferViewFlowAnalysis
This it to make more clear the difference between this and
an AliasAnalysis.

For example, given a sequence of subviews that create values
A -> B -> C -> d:
BufferViewFlowAnalysis::resolve(B) => {B, C, D}
AliasAnalysis::resolve(B) => {A, B, C, D}

Differential Revision: https://reviews.llvm.org/D100838
2021-05-07 16:12:54 +02:00
Tobias Gysi 26e916334e [mlir][linalg] Add IndexedGenericOp to GenericOp canonicalization.
Replace all `linalg.indexed_generic` ops by `linalg.generic` ops that access the iteration indices using the `linalg.index` op.

Differential Revision: https://reviews.llvm.org/D101612
2021-05-07 06:00:16 +00:00
Rob Suderman d3e987c389 [mlir][tosa] Added div op, variadic concat. Removed placeholder. Spec v0.22 alignment.
Nearly complete alignment to spec v0.22
- Adds Div op
- Concat inputs now variadic
- Removes Placeholder op

Note: TF side PR https://github.com/tensorflow/tensorflow/pull/48921 deletes Concat legalizations to avoid breaking TensorFlow CI. This must be merged only after the TF PR has merged.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D101958
2021-05-06 15:55:58 -07:00
Lei Zhang 41bc54cc56 [mlir][spirv] NFC: Replace OwningSPIRVModuleRef with OwningOpRef
Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D102009
2021-05-06 17:17:44 -04:00
River Riddle 6304c0836a [mlir] Store the flag for dynamic operand storage in the low bits
It is currently stored in the high bits, which is disallowed on certain
platforms (e.g. android). This revision switches the representation to use
the low bits instead, fixing crashes/breakages on those platforms.

Differential Revision: https://reviews.llvm.org/D101969
2021-05-06 12:45:35 -07:00
thomasraoux 52525cb20f [mlir][linalg][NFC] Make reshape folding control more fine grain
This expose a lambda control instead of just a boolean to control unit
dimension folding.
This however gives more control to user to pick a good heuristic.
Folding reshapes helps fusion opportunities but may generate sub-optimal
generic ops.

Differential Revision: https://reviews.llvm.org/D101917
2021-05-06 10:11:39 -07:00
Navdeep Kumar 875eb523c1 [MLIR][GPU][NVVM] Add warp synchronous matrix-multiply accumulate ops
Add warp synchronous matrix-multiply accumulate ops in GPU and NVVM
dialect. Add following three ops to GPU dialect :-
  1.) subgroup_mma_load_matrix
  2.) subgroup_mma_store_matrix
  3.) subgroup_mma_compute
Add following three ops to NVVM dialect :-
  1.) wmma.m16n16k16.load.[a,b,c].[f16,f32].row.stride
  2.) wmma.m16n16k16.store.d.[f16,f32].row.stride
  3.) wmma.m16n16k16.mma.row.row.[f16,f32].[f16,f32]

Reviewed By: bondhugula, ftynse, ThomasRaoux

Differential Revision: https://reviews.llvm.org/D95330
2021-05-06 12:06:25 +05:30
Javier Setoain 95861216ac [mlir][ArmSVE] Add masked arithmetic operations
These instructions map to SVE-specific instrinsics that accept a
predicate operand to support control flow in vector code.

Differential Revision: https://reviews.llvm.org/D100982
2021-05-05 17:41:58 +01:00
Sergei Grechanik d80b04ab00 [mlir][Affine][Vector] Support vectorizing reduction loops
This patch adds support for vectorizing loops with 'iter_args'
implementing known reductions along the vector dimension. Comparing to
the non-vector-dimension case, two additional things are done during
vectorization of such loops:
- The resulting vector returned from the loop is reduced to a scalar
  using `vector.reduce`.
- In some cases a mask is applied to the vector yielded at the end of
  the loop to prevent garbage values from being written to the
  accumulator.

Vectorization of reduction loops is disabled by default. To enable it, a
map from loops to array of reduction descriptors should be explicitly passed to
`vectorizeAffineLoops`, or `vectorize-reductions=true` should be passed
to the SuperVectorize pass.

Current limitations:
- Loops with a non-unit step size are not supported.
- n-D vectorization with n > 1 is not supported.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D100694
2021-05-05 09:03:59 -07:00
Uday Bondhugula 62851ea7ea [MLIR] Rename free function `verify` on OffsetSizeAndStrideOpInterface
Using a free function verify(<Op>) is error prone. Rename it.

Differential Revision: https://reviews.llvm.org/D101886
2021-05-05 17:44:15 +05:30
Alexander Belyaev 2865d114f9 [mlir] Use ReassociationIndices instead of affine maps in linalg.reshape.
Differential Revision: https://reviews.llvm.org/D101861
2021-05-05 12:59:57 +02:00
Javier Setoain 001d601ac4 [mlir][ArmSVE] Add basic arithmetic operations
While we figure out how to best add Standard support for scalable
vectors, these instructions provide a workaround for basic arithmetic
between scalable vectors.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D100837
2021-05-05 09:50:18 +02:00
William S. Moses f4a2dbfe29 [MLIR][SCF] Combine adjacent scf.if with same condition
Differential Revision: https://reviews.llvm.org/D101798
2021-05-05 00:39:58 -04:00
Aart Bik a2c9d4bb04 [mlir][sparse] Introduce proper sparsification passes
This revision migrates more code from Linalg into the new permanent home of
SparseTensor. It replaces the test passes with proper compiler passes.

NOTE: the actual removal of the last glue and clutter in Linalg will follow

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D101811
2021-05-04 17:10:09 -07:00
William S. Moses 8e211bf1c8 [MLIR][SCF] Assume uses of condition in the body of scf.while is true
Differential Revision: https://reviews.llvm.org/D101801
2021-05-04 11:39:07 -04:00
William S. Moses 93297e4bac [MLIR] Replace a not of a comparison with appropriate comparison
Differential Revision: https://reviews.llvm.org/D101710
2021-05-04 11:23:29 -04:00
Tobias Gysi 05d2297b86 [mlir][linalg] Always lower index operations during loop lowering.
Ensure the index operations are lowered on all linalg loop lowering paths.

Differential Revision: https://reviews.llvm.org/D101827
2021-05-04 14:30:59 +00:00
MaheshRavishankar a6e09391bb [mlir][Linalg] Add a utility method to get reassociations maps for reshape.
Given the source and destination shapes, if they are static, or if the
expanded/collapsed dimensions are unit-extent, it is possible to
compute the reassociation maps that can be used to reshape one type
into another. Add a utility method to return the reassociation maps
when possible.

This utility function can be used to fuse a sequence of reshape ops,
given the type of the source of the producer and the final result
type. This pattern supercedes a more constrained folding pattern added
to DropUnitDims pass.

Differential Revision: https://reviews.llvm.org/D101343
2021-05-03 14:40:15 -07:00
MaheshRavishankar fd15e2b825 [mlir][Linalg] Use rank-reduced versions of subtensor and subtensor insert when possible.
Convert subtensor and subtensor_insert operations to use their
rank-reduced versions to drop unit dimensions.

Differential Revision: https://reviews.llvm.org/D101495
2021-05-03 12:51:24 -07:00
thomasraoux 9621c1ef56 [mlir][linalg] Fix vectorization bug in vector transfer indexing map calculation
The current implementation had a bug as it was relying on the target vector
dimension sizes to calculate where to insert broadcast. If several dimensions
have the same size we may insert the broadcast on the wrong dimension. The
correct broadcast cannot be inferred from the type of the source and
destination vector.

Instead when we want to extend transfer ops we calculate an "inverse" map to the
projected permutation and insert broadcast in place of the projected dimensions.

Differential Revision: https://reviews.llvm.org/D101738
2021-05-03 12:16:38 -07:00
Frederik Gossen ec339163a7 [MLIR][Linalg] Lower `linalg.tiled_loop` in a separate pass
Add dedicated pass `convert-linalg-tiled-loops-to-scf` to lower
`linalg.tiled_loop`s.

Differential Revision: https://reviews.llvm.org/D101768
2021-05-03 21:02:02 +02:00
thomasraoux 7417541fd8 [mlir][vector] Add canonicalization for extract/insert -> shapecast
Differential Revision: https://reviews.llvm.org/D101643
2021-05-03 10:41:15 -07:00
thomasraoux be8e2801a4 [mlir][vector][NFC] split TransposeOp lowerning out of contractLowering
Move TransposeOp lowering in its own populate function as in some cases
it is better to keep it during ContractOp lowering to better
canonicalize it rather than emiting scalar insert/extract.

Differential Revision: https://reviews.llvm.org/D101647
2021-05-03 10:23:45 -07:00
William S. Moses 039bdcc0a8 [MLIR] Canonicalize sub/add of a constant and another sub/add of a constant
Differential Revision: https://reviews.llvm.org/D101705
2021-05-03 11:49:23 -04:00
William S. Moses 78720296f3 [MLIR] Canonicalization of Integer Cast Operations
1) Canonicalize IndexCast(SExt(x)) => IndexCast(x)
2) Provide constant folds of sign_extend and truncate

Differential Revision: https://reviews.llvm.org/D101714
2021-05-02 11:22:18 -04:00
William S. Moses a2b5314cbc [MLIR] Handle llvm.icmp of pointers
Differential Revision: https://reviews.llvm.org/D101712
2021-05-02 01:17:50 -04:00
eopXD 0c1ff26bd3 [mlir] [affine] add canonicalization for affine.vector_load, vector_store
Added canonicalization for vector_load and vector_store. An existing
pattern SimplifyAffineOp can be reused to compose maps that supplies
result into them. Added AffineVectorStoreOp and AffineVectorLoadOp
into static_assert of SimplifyAffineOp to allow operation to use it.

This fixes the bug filed: https://bugs.llvm.org/show_bug.cgi?id=50058

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D101691
2021-05-02 09:06:46 +05:30
Aart Bik 0a29219931 [mlir][sparse] sparse tensor type encoding migration (new home, new builders)
(1) migrates the encoding from TensorDialect into the new SparseTensorDialect
(2) replaces dictionary-based storage and builders with struct-like data

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D101669
2021-04-30 19:30:38 -07:00
Ahmed Taei 499e89fc91 Add patterns to lower vector.multi_reduction into a sequence of vector.reduction
Three patterns are added to convert into vector.multi_reduction into a
sequence of vector.reduction as the following:

- Transpose the inputs so inner most dimensions are always reduction.
- Reduce rank of vector.multi_reduction into 2d with inner most
reduction dim (get the 2d canical form)
- 2D canonical form is converted into a sequence of vector.reduction.

There are two things we might worth in a follow up diff:

- An scf.for (maybe optionally) around vector.reduction instead of unrolling it.
- Breakdown the vector.reduction into a sequence of vector.reduction
(e.g tree-based reduction) instead of relying on how downstream dialects
handle it.
  Note: this will requires passing target-vector-length

Differential Revision: https://reviews.llvm.org/D101570
2021-04-30 10:52:21 -07:00
Aart Bik 319072f4e3 [mlir][sparse] migrate sparse operations into new sparse tensor dialect
This is the very first step toward removing the glue and clutter from linalg and
replace it with proper sparse tensor types. This revision migrates the LinalgSparseOps
into SparseTensorOps of a sparse tensor dialect. This also provides a new home for
sparse tensor related transformation.

NOTE: the actual replacement with sparse tensor types (and removal of linalg glue/clutter)
will follow but I am trying to keep the amount of changes per revision manageable.

Differential Revision: https://reviews.llvm.org/D101573
2021-04-29 15:52:35 -07:00
Mehdi Amini 086e0f05bf Revert "[mlir][sparse] migrate sparse operations into new sparse tensor dialect"
This reverts commit a6d92a9711.

The build with -DBUILD_SHARED_LIBS=ON is broken.
2021-04-29 20:59:41 +00:00
Aart Bik a6d92a9711 [mlir][sparse] migrate sparse operations into new sparse tensor dialect
This is the very first step toward removing the glue and clutter from linalg and
replace it with proper sparse tensor types. This revision migrates the LinalgSparseOps
into SparseTensorOps of a sparse tensor dialect. This also provides a new home for
sparse tensor related transformation.

NOTE: the actual replacement with sparse tensor types (and removal of linalg glue/clutter)
will follow but I am trying to keep the amount of changes per revision manageable.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D101488
2021-04-29 12:09:10 -07:00
Alex Zinenko 6841e6afba [mlir] support max/min lower/upper bounds in affine.parallel
This enables to express more complex parallel loops in the affine framework,
for example, in cases of tiling by sizes not dividing loop trip counts perfectly
or inner wavefront parallelism, among others. One can't use affine.max/min
and supply values to the nested loop bounds since the results of such
affine.max/min operations aren't valid symbols. Making them valid symbols
isn't an option since they would introduce selection trees into memref
subscript arithmetic as an unintended and undesired consequence. Also
add support for converting such loops to SCF. Drop some API that isn't used in
the core repo from AffineParallelOp since its semantics becomes ambiguous in
presence of max/min bounds. Loop normalization is currently unavailable for
such loops.

Depends On D101171

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D101172
2021-04-29 13:16:25 +02:00
Alex Zinenko 545fa37834 [mlir] Affine: parallelize affine loops with reductions
Introduce a basic support for parallelizing affine loops with reductions
expressed using iteration arguments. Affine parallelism detector now has a flag
to assume such reductions are parallel. The transformation handles a subset of
parallel reductions that are can be expressed using affine.parallel:
integer/float addition and multiplication. This requires to detect the
reduction operation since affine.parallel only supports a fixed set of
reduction operators.

Reviewed By: chelini, kumasento, bondhugula

Differential Revision: https://reviews.llvm.org/D101171
2021-04-29 13:16:24 +02:00
Tres Popp 42e5f42215 [mlir] Support complex numbers in Linalg promotion
FillOp allows complex ops, and filling a properly sized buffer with
a default zero complex number is implemented.

Differential Revision: https://reviews.llvm.org/D99939
2021-04-29 11:58:57 +02:00
Nicolas Vasilache b6113db955 [mlir][Linalg] Generalize linalg vectorization
This revision adds support for vectorizing more general linalg operations with projected permutation maps.

This is achieved by eagerly broadcasting the intermediate vector to the common size
of the iteration domain of the linalg op. This allows a much more natural expression of
generalized vectorization but may introduce additional computations until all the
proper canonicalizations are implemented.

This generalization modifies the vector.transfer_read/write permutation logic and
exposes the fact that the logic employed in vector.contract was too ad-hoc.

As a consequence, changes occur in the permutation / transposition logic for contraction. In turn this prompts supporting more cases in the lowering of contract
to matrix intrinsics, which is required to make the corresponding tests pass.

Differential revision: https://reviews.llvm.org/D101165
2021-04-29 07:44:01 +00:00
MaheshRavishankar 41849a9195 [mlir][Linalg] Avoid changing the rank of the result in canonicalizations of subtensor.
Canonicalizations for subtensor operations defaulted to use the
rank-reduced version of the operation, but the cast inserted to get
back the original type would be illegal if the rank was actually
reduced. Instead make the canonicalization not reduce the rank of the
operation.

Differential Revision: https://reviews.llvm.org/D101258
2021-04-28 11:33:26 -07:00
Frederik Gossen 511ffe17ed Revert "[MLIR][Shape] Concretize broadcast result type if possible"
This reverts commit dca5361035.
2021-04-28 17:16:02 +02:00
Nicolas Vasilache b87219f77e [mlir][python] Add basic python support for GPU dialect and passes
Differential Revision: https://reviews.llvm.org/D101449
2021-04-28 14:52:28 +00:00
Nicolas Vasilache e7db8408d0 [mlir][python] Add python support for async dialect and passes.
since the `async` keyword is reserved in python, the dialect is called async_dialect.

Differential Revision: https://reviews.llvm.org/D101447
2021-04-28 14:52:27 +00:00