Commit Graph

5032 Commits

Author SHA1 Message Date
Stella Laurenzo 5d6d30edf8 [mlir] Extend C and Python API to support bulk loading of DenseElementsAttr.
* This already half existed in terms of reading the raw buffer backing a DenseElementsAttr.
* Documented the precise expectations of the buffer layout.
* Extended the Python API to support construction from bitcasted buffers, allowing construction of all primitive element types (even those that lack a compatible representation in Python).
* Specifically, the Python API can now load all integer types at all bit widths and all floating point types (f16, f32, f64, bf16).

Differential Revision: https://reviews.llvm.org/D111284
2021-10-07 08:42:12 -07:00
Lei Zhang 3964c1db91 [mlir][vector] Split populateVectorContractLoweringPatterns
It was bundling quite a lot of patterns that convert high-D
vector ops into low-D elementary ops. It might not be good
for all of the patterns to happen for a particular downstream
user. For example, `ShapeCastOpRewritePattern` rewrites
`vector.shape_cast` into data movement extract/insert ops.

Instead, split the entry point into multiple ones so users
can pull in patterns on demand.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D111225
2021-10-07 09:39:26 -04:00
Eugene Zhulenev 8276ac13e9 [mlir] Add alignment attribute to memref.global
Revived https://reviews.llvm.org/D102435

Add alignment attribute to `memref.global` and propagate it to llvm global in memref->llvm lowering

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D111309
2021-10-07 06:21:57 -07:00
Tobias Gysi 3fe7fe4424 [mlir][linalg] Add unsigned min/max/cast function to OpDSL.
Update OpDSL to support unsigned integers by adding unsigned min/max/cast signatures. Add tests in OpDSL and on the C++ side to verify the proper signed and unsigned operations are emitted.

The patch addresses an issue brought up in https://reviews.llvm.org/D111170.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D111230
2021-10-07 06:27:20 +00:00
Mogball 8c08f21b60 [MLIR] Split arith dialect from the std dialect
Create the Arithmetic dialect that contains basic integer and floating
point arithmetic operations. Ops that did not meet this criterion were
moved to the Math dialect.

First of two atomic patches to remove integer and floating point
operations from the standard dialect. Ops will be removed from the
standard dialect in a subsequent patch.

Reviewed By: ftynse, silvas

Differential Revision: https://reviews.llvm.org/D110200
2021-10-06 19:25:51 +00:00
Alexandre Rames fd9613324d [MLIR] Rename Shape dialect's `join` to `meet`.
For the type lattice, we (now) use the "less specialized or equal" partial
order, leading to the bottom representing the empty set, and the top
representing any type.

This naming is more in line with the generally used conventions, where the top
of the lattice is the full set, and the bottom of the lattice is the empty set.
A typical example is the powerset of a finite set: generally, meet would be the
intersection, and join would be the union.

```
top:  {a,b,c}
     /   |   \
 {a,b} {a,c} {b,c}
   |  X     X  |
   {a} { b } {c}
      \  |  /
bottom: { }
```

This is in line with the examined lattice representations in LLVM:
* lattice for `BitTracker::BitValue` in `Hexagon/BitTracker.h`
* lattice for constant propagation in `HexagonConstPropagation.cpp`
* lattice in `VarLocBasedImpl.cpp`
* lattice for address space inference code in `InferAddressSpaces.cpp`

Reviewed By: silvas, jpienaar

Differential Revision: https://reviews.llvm.org/D110766
2021-10-06 09:41:33 -07:00
Simon Pilgrim e244a6fec7 [mlir] Replace report_fatal_error(std::string) uses with report_fatal_error(Twine)
As described on D111049, we're trying to remove the <string> dependency from error handling and replace uses of report_fatal_error(const std::string&) with the Twine() variant which can be forward declared.
2021-10-06 12:57:20 +01:00
Diego Caballero eaf2588a51 [mlir][Linalg] Add support for min/max reduction vectorization in linalg.generic
This patch extends Linalg core vectorization with support for min/max reductions
in linalg.generic ops. It enables the reduction detection for min/max combiner ops.
It also renames MIN/MAX combining kinds to MINS/MAXS to make the sign explicit for
floating point and signed integer types. MINU/MAXU should be introduce din the future
for unsigned integer types.

Reviewed By: pifon2a, ThomasRaoux

Differential Revision: https://reviews.llvm.org/D110854
2021-10-05 22:47:20 +00:00
Lei Zhang 7a89444cd9 [mlir][spirv] Add ops and patterns for lowering standard max/min ops
Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D111143
2021-10-05 14:27:32 -04:00
Aart Bik 16b8f4ddae [mlir][sparse] add a "release" operation to sparse tensor dialect
We have several ways to materialize sparse tensors (new and convert) but no explicit operation to release the underlying sparse storage scheme at runtime (other than making an explicit delSparseTensor() library call). To simplify memory management, a sparse_tensor.release operation has been introduced that lowers to the runtime library call while keeping tensors, opague pointers, and memrefs transparent in the initial IR.

*Note* There is obviously some tension between the concept of immutable tensors and memory management methods. This tension is addressed by simply stating that after the "release" call, no further memref related operations are allowed on the tensor value. We expect the design to evolve over time, however, and arrive at a more satisfactory view of tensors and buffers eventually.

Bug:
http://llvm.org/pr52046

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D111099
2021-10-05 09:35:59 -07:00
Lei Zhang 83e074a0c6 [mlir] Add an 'cppNamespace' field to availability
This allows us to generate interfaces in a namespace,
following other TableGen'erated code.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D108311
2021-10-05 09:38:09 -04:00
Tobias Gysi e826db6240 [mlir][linalg] Move generalization pattern to Transforms (NFC).
Move the generalization pattern to the other Linalg transforms to make it available to the codegen strategy.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110728
2021-10-05 12:49:42 +00:00
Frederik Gossen 519663beba [MLIR] Add an option to disable `maxIterations` in greedy pattern rewrites
This option is needed for passes that are known to reach a fix point, but may
need many iterations depending on the size of the input IR.

Differential Revision: https://reviews.llvm.org/D111058
2021-10-05 11:49:01 +02:00
Nicolas Vasilache 8096759519 [mlir][Linalg] NFC - Add support to specify that a tensor value is known to bufferize to writeable memory
This change allows better interop with external clients of comprehensive bufferization functions
but is otherwise NFC for the MLIR pass itself.

Differential Revision: https://reviews.llvm.org/D111121
2021-10-05 08:37:34 +00:00
Alex Zinenko 01d696e563 [mlir] rename the "packing" flag of linalg.pad_tensor to "nofold"
The discussion in https://reviews.llvm.org/D110425 demonstrated that "packing"
may be a confusing term to define the behavior of this op in presence of the
attribute. Instead, indicate the intended effect of preventing the folder from
being applied.

Reviewed By: nicolasvasilache, silvas

Differential Revision: https://reviews.llvm.org/D111046
2021-10-04 21:28:11 +02:00
Weiwei Li 1e4cfe5e4f [mlir][SPIRVToLLVM] Propagate location attribute from spv.GlobalVariable to llvm.mlir.global
This patch is mainly to propogate location attribute from spv.GlobalVariable to llvm.mlir.global.

It also contains three small changes.

1. Remove the restriction on UniformConstant In SPIRVToLLVM.cpp;
2. Remove the errorCheck on relaxedPrecision when deserializering SPIR-V in Deserializer.cpp
3. In SPIRVOps.cpp, let ConstantOp take signedInteger too.

Co-authered: Alan Liu <alanliu.yf@gmail.com> and Xinyi Liu <xyliuhelen@gmail.com>

Reviewed by:antiagainst

Differential revision: https://reviews.llvm.org/D110207
2021-10-05 00:09:09 +08:00
Nicolas Vasilache fab634b4e2 [mlir] Tighten strided layout specification.
Clarify that the strided layout specification is represented by a single semi-affine map.

Differential Revision: https://reviews.llvm.org/D110921
2021-10-04 10:37:05 +00:00
Cullen Rhodes d023298b61 [MLIR] Fix unused tablegen template arg warnings
Identified in D109359.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D110805
2021-10-04 10:22:39 +00:00
Uday Bondhugula 0b83a35caf [MLIR][NFC] Drop unnecessary use of OpBuilder in build trip count map
NFC. Drop unnecessary use of OpBuilder in buildTripCountMapAndOperands.
Rename this to getTripCountMapAndOperands and remove stale comments.

Differential Revision: https://reviews.llvm.org/D110993
2021-10-03 09:25:18 +05:30
Daniel Resnick 782a97a977 [mlir][capi] Add TypeID to MLIR C-API
Exposes mlir::TypeID to the C API as MlirTypeID along with various accessors
and helper functions.

Differential Revision: https://reviews.llvm.org/D110897
2021-10-01 14:21:18 -06:00
Tobias Gysi bf28849745 [mlir][linalg] Retire PoolingMaxOp/PoolingMinOp/PoolingSumOp.
The pooling ops are among the last remaining hard coded Linalg operations that have no region attached. They got obsolete due to the OpDSL pooling operations. Removing them allows us to delete specialized code and tests that are not needed for the OpDSL counterparts that rely on the standard code paths.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110909
2021-10-01 13:51:56 +00:00
Uday Bondhugula 08b63db8bb [MLIR][GPU] Add GPU launch op support for dynamic shared memory
Add support for dynamic shared memory for GPU launch ops: add an
optional operand to gpu.launch and gpu.launch_func ops to specify the
amount of "dynamic" shared memory to use. Update lowerings to connect
this operand to the GPU runtime.

Differential Revision: https://reviews.llvm.org/D110800
2021-10-01 16:46:07 +05:30
Nicolas Vasilache b016bd1230 [mlir][Linalg] Refactor comprehensive bufferize for external uses - NFC
This revision exposes some minimal funcitonality to allow comprehensive
bufferization to interop with external projects.

Differential Revision: https://reviews.llvm.org/D110875
2021-09-30 20:21:08 +00:00
Stella Laurenzo 267bb194f3 [mlir] Remove old "tc" linalg ods generator.
* This could have been removed some time ago as it only had one op left in it, which is redundant with the new approach.
* `matmul_i8_i8_i32` (the remaining op) can be trivially replaced by `matmul`, which natively supports mixed precision.

Differential Revision: https://reviews.llvm.org/D110792
2021-09-30 16:30:06 +00:00
Chris Lattner fb093c8314 [ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser.
The former is redundant because the later carries it as part of
its builder.  Add a getContext() helper method to DialectAsmParser
to make this more convenient, and stop passing the context around
explicitly.  This simplifies ODS generated parser hooks for attrs
and types.

This resolves PR51985

Recommit 4b32f8bac4 after fixing a dependency.

Differential Revision: https://reviews.llvm.org/D110796
2021-09-30 05:10:28 +00:00
Mehdi Amini 3310e0020c Revert "[ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser."
This reverts commit 4b32f8bac4.

Seems like the build is broken with -DDBUILD_SHARED_LIBS=ON
2021-09-30 05:01:17 +00:00
Chris Lattner 4b32f8bac4 [ODS/AsmParser] Don't pass MLIRContext with DialectAsmParser.
The former is redundant because the later carries it as part of
its builder.  Add a getContext() helper method to DialectAsmParser
to make this more convenient, and stop passing the context around
explicitly.  This simplifies ODS generated parser hooks for attrs
and types.

This resolves PR51985

Differential Revision: https://reviews.llvm.org/D110796
2021-09-29 21:36:05 -07:00
Matthias Springer 27451a05ed [mlir][vector] Fold transfer ops and tensor.extract/insert_slice.
* Fold vector.transfer_read and tensor.extract_slice.
* Fold vector.transfer_write and tensor.insert_slice.

Differential Revision: https://reviews.llvm.org/D110627
2021-09-30 09:28:00 +09:00
Marcel Koester 09cd4a71ed Introduced AllocationOpInterface to create deallocation operations on-the-fly that are compatible with the allocation operation implementing this interface.
Added interface implementations for AllocOp and CloneOp defined in the MemRef diallect.
Adapted the BufferDeallocation pass to be compatible with the interface introduced in this CL.

Differential Revision: https://reviews.llvm.org/D109350
2021-09-29 15:54:21 +02:00
Nicolas Vasilache 92ea624a13 [mlir][Linalg] Rewrite CodegenStrategy to populate a pass pipeline.
This revision retires a good portion of the complexity of the codegen strategy and puts the logic behind pass logic.

Differential revision: https://reviews.llvm.org/D110678
2021-09-29 13:35:45 +00:00
Rob Suderman 4f38f0640d [mlir][tosa] Add i32 to supported quantized type
Quantized int type should include I32 types as its the output of a quantizd
convolution or matmul operation.

Reviewed By: NatashaKnk

Differential Revision: https://reviews.llvm.org/D110651
2021-09-28 15:04:39 -07:00
bakhtiyar 55dfab39a2 Rename target block size to min task size for clarity.
Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D110604
2021-09-28 14:51:55 -07:00
Amy Zhuang 7ab14b8886 [mlir] Unroll-and-jam loops with iter_args.
Unroll-and-jam currently doesn't work when the loop being unroll-and-jammed
or any of its inner loops has iter_args. This patch modifies the
unroll-and-jam utility to support loops with iter_args.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D110085
2021-09-28 14:13:27 -07:00
Alexander Belyaev 9fb57c8c1d [mlir] Add min/max operations to Standard.
[RFC: Add min/max ops](https://llvm.discourse.group/t/rfc-add-min-max-operations/4353)

I was following the naming style for Arith dialect in
https://reviews.llvm.org/D110200,
i.e. similar to DivSIOp and DivUIOp I defined MaxSIOp, MaxUIOp.

When Arith PR is landed, I will migrate these ops as well.

Differential Revision: https://reviews.llvm.org/D110540
2021-09-28 09:40:22 +02:00
Tobias Gysi d20d0e145d [mlir][linalg] Finer-grained padding control.
Adapt the signature of the PaddingValueComputationFunction callback to either return the padding value or failure to signal padding is not desired.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110572
2021-09-27 19:21:37 +00:00
Sumesh Udayakumaran b2af2aeea6 [mlir] Mode for explicitly controlling the fusion kind
New mode option that allows for either running the default fusion kind that happens today or doing either of producer-consumer or sibling fusion. This will also be helpful to minimize the compile-time of the fusion tests.

Reviewed By: bondhugula, dcaballe

Differential Revision: https://reviews.llvm.org/D110102
2021-09-27 20:37:42 +03:00
Eugene Zhulenev 92db09cde0 [mlir] AsyncRuntime: use int64_t for ref counting operations
Workaround for SystemZ ABI problem: https://bugs.llvm.org/show_bug.cgi?id=51898

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D110550
2021-09-27 07:55:01 -07:00
Tobias Gysi e158b5634a [mlir][linalg] Make fusion on tensor rewriter friendly (NFC).
Let the calling pass or pattern replace the uses of the original root operation. Internally, the tileAndFuse still replaces uses and updates operands but only of newly created operations.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110169
2021-09-27 11:28:25 +00:00
Nicolas Vasilache 1b49a72de9 [mlir] Factor out constraint set creation from hoist padding.
This revision adds a

```
FlatAffineValueConstraints(ValueRange ivs, ValueRange lbs, ValueRange ubs)
```

method and use it in hoist padding.

Differential Revision: https://reviews.llvm.org/D110427
2021-09-27 10:11:35 +00:00
Nicolas Vasilache b74493ecea [mlir][Linalg] Refactor padding hoisting - NFC
This revision extracts padding hoisting in a new file and cleans it up in prevision of future improvements and extensions.

Differential Revision: https://reviews.llvm.org/D110414
2021-09-27 09:50:31 +00:00
Kunwar Shaanjeet Singh Grover 0f78ece169 [MLIR] Add functionality to remove redundant local variables
This patch adds functionality to FlatAffineConstraints to remove local
variables using equalities. This helps in keeping output representation of
FlatAffineConstraints smaller.

This patch is part of a series of patches aimed at generalizing affine
dependence analysis.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D110056
2021-09-25 16:10:43 +05:30
River Riddle ef764eeeb9 [mlir:ElementsAttr] Avoid crash on empty contiguous ranges
We currently, incorrectly, assume that a range always has at least
one element when building a contiguous range. This commit adds
a proper empty check to avoid crashing.

Differential Revision: https://reviews.llvm.org/D110457
2021-09-24 23:48:51 +00:00
Diego Caballero 2a876a711d [mlir] Create a generic reduction detection utility
This patch introduces a generic reduction detection utility that works
across different dialecs. It is mostly a generalization of the reduction
detection algorithm in Affine. The reduction detection logic in Affine,
Linalg and SCFToOpenMP have been replaced with this new generic utility.

The utility takes some basic components of the potential reduction and
returns: 1) the reduced value, and 2) a list with the combiner operations.
The logic to match reductions involving multiple combiner operations disabled
until we can properly test it.

Reviewed By: ftynse, bondhugula, nicolasvasilache, pifon2a

Differential Revision: https://reviews.llvm.org/D110303
2021-09-24 20:45:59 +00:00
River Riddle 531206310a [mlir:OpAsm] Factor out the common bits of (Op/Dialect)Asm(Parser/Printer)
This has a few benefits:
* It allows for defining parsers/printer code blocks that
  can be shared between operations and attribute/types.
* It removes the weird duplication of generic parser/printer hooks,
  which means that newly added hooks only require touching one class.

Differential Revision: https://reviews.llvm.org/D110375
2021-09-24 20:12:19 +00:00
River Riddle aca9bea199 [mlir:MemRef] Move DmaStartOp/DmaWaitOp to ODS
These are among the last operations still defined explicitly in C++. I've
tried to keep this commit as NFC as possible, but these ops
definitely need a non-NFC cleanup at some point.

Differential Revision: https://reviews.llvm.org/D110440
2021-09-24 19:35:28 +00:00
Lei Zhang e325ebb9c7 [mlir][tosa] Add some transpose folders
* If the input is a constant splat value, we just
  need to reshape it.
* If the input is a general constant with one user,
  we can also constant fold it, without bloating
  the IR.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D110439
2021-09-24 15:25:14 -04:00
River Riddle ef976337f5 [mlir:OpConversion] Remove the remaing usages of the deprecated matchAndRewrite methods
This commits updates the remaining usages of the ArrayRef<Value> based
matchAndRewrite/rewrite methods in favor of the new OpAdaptor
overload.

Differential Revision: https://reviews.llvm.org/D110360
2021-09-24 17:51:41 +00:00
River Riddle b54c724be0 [mlir:OpConversionPattern] Add overloads for taking an Adaptor instead of ArrayRef
This has been a TODO for a long time, and it brings about many advantages (namely nice accessors, and less fragile code). The existing overloads that accept ArrayRef are now treated as deprecated and will be removed in a followup (after a small grace period). Most of the upstream MLIR usages have been fixed by this commit, the rest will be handled in a followup.

Differential Revision: https://reviews.llvm.org/D110293
2021-09-24 17:51:41 +00:00
Alex Zinenko 5988a3b7a0 [mlir] Linalg: ensure tile-and-pad always creates padding as requested
Initially, the padding transformation and the related operation were only used
to guarantee static shapes of subtensors in tiled operations. The
transformation would not insert the padding operation if the shapes were
already static, and the overall code generation would actively remove such
"noop" pads. However, this transformation can be also used to pack data into
smaller tensors and marshall them into faster memory, regardless of the size
mismatches. In context of expert-driven transformation, we should assume that,
if padding is requested, a potentially padded tensor must be always created.
Update the transformation accordingly. To do this, introduce an optional
`packing` attribute to the `pad_tensor` op that serves as an indication that
the padding is an intentional choice (as opposed to side effect of type
normalization) and should be left alone by cleanups.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110425
2021-09-24 18:40:13 +02:00
Arjun P 4a57f5d1e1 [MLIR] PresburgerSet: support divisions in operations
Add support for intersecting, subtracting, complementing and checking equality of sets having divisions.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D110138
2021-09-24 15:36:47 +05:30
Alex Zinenko 3f89e339bb [mlir] add pad_tensor(tensor.cast) -> pad_tensor canonicalizer
This canonicalization pattern complements the tensor.cast(pad_tensor) one in
propagating constant type information when possible. It contributes to the
feasibility of pad hoisting.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110343
2021-09-24 12:03:47 +02:00
Matthias Springer f3f25ffc04 [mlir][linalg] Fix result type in FoldSourceTensorCast
* Do not discard static result type information that cannot be inferred from lower/upper padding.
* Add optional argument to `PadTensorOp::inferResultType` for specifying known result dimensions.

Differential Revision: https://reviews.llvm.org/D110380
2021-09-24 16:47:18 +09:00
Mehdi Amini 83f3c615dd Add missing storageType to AttrDef to ODS
This is only noticeable when using an attribute across dialects I think.
Previously the namespace would be ommited, but it wouldn't matter as
long as the generated code stays within a single namespace.

Differential Revision: https://reviews.llvm.org/D110367
2021-09-24 01:30:29 +00:00
Matthias Springer 8dc16ba8d2 [mlir][linalg] Merge all tiling passes into a single one.
Passes such as `linalg-tile-to-tiled-loop` are merged into `linalg-tile`.

Differential Revision: https://reviews.llvm.org/D110214
2021-09-24 10:16:46 +09:00
Aart Bik a924fcc7c3 [mlir][sparse] add sparse kernels test to sparse compiler test suite
This test makes sure kernels map to efficient sparse code, i.e. all
compressed for-loops, no co-iterating while loops.  In addition, this
revision removes the special constant folding inside the sparse
compiler in favor of Mahesh' new generic linalg folding. Thanks!

NOTE: relies on Mahesh fix, which needs to be rebased first

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D110001
2021-09-22 14:56:39 -07:00
Chris Lattner da93829b44 [DialectAsmPrinter] Add missing 'printAttributeWithoutType' member.
DialectAsmParser has a `parseAttribute` member that takes a
contextual type, but DialectAsmPrinter doesn't have the corresponding
member to take advantage of it.  As such, custom attribute
implementations can't really use it.  This adds the obvious missing
method which fills this hole.

Differential Revision: https://reviews.llvm.org/D110211
2021-09-21 18:45:24 -07:00
Christian Sigg 9149ae09bd Support value-typed references in iterator facade's operator->()
Add a PointerProxy similar to the existing iterator_facade_base::ReferenceProxy and return it from the arrow operator. This prevents iterator facades with a reference type that is not a true reference to take the address of a temporary.

Forward the reference type of the mapped_iterator to the iterator adaptor which in turn forwards it to the iterator facade. This fixes mlir::op_iterator::operator->() to take the address of a temporary.

Make some polishing changes to op_iterator and op_filter_iterator.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D109490
2021-09-21 20:42:22 +02:00
Tobias Gysi 9072f1b5f8 [mlir][linalg] Add isPermutation helper (NFC).
Add a helper method to check if an index vector contains a permutation of its indices. Additionally, refactor applyPermutationToVector to take int64_t.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D110135
2021-09-21 15:07:39 +00:00
Morten Borup Petersen 032cb1650f [MLIR][SCF] Add for-to-while loop transformation pass
This pass transforms SCF.ForOp operations to SCF.WhileOp. The For loop condition is placed in the 'before' region of the while operation, and indctuion variable incrementation + the loop body in the 'after' region. The loop carried values of the while op are the induction variable (IV) of the for-loop + any iter_args specified for the for-loop.
Any 'yield' ops in the for-loop are rewritten to additionally yield the (incremented) induction variable.

This transformation is useful for passes where we want to consider structured control flow solely on the basis of a loop body and the computation of a loop condition. As an example, when doing high-level synthesis in CIRCT, the incrementation of an IV in a for-loop is "just another part" of a circuit datapath, and what we really care about is the distinction between our datapath and our control logic (the condition variable).

Differential Revision: https://reviews.llvm.org/D108454
2021-09-21 09:09:54 +01:00
Kunwar Shaanjeet Singh Grover 0d12c99191 [MLIR] Add mergeLocalIds and mergeSymbolIds
This patch adds mergeLocalIds andmergeSymbolIds as public functions
for FlatAffineConstraints and FlatAffineValueConstraints respectively.

mergeLocalIds is also required to support divisions in intersection,
subtraction, equality checks, and complement for PresburgerSet.

This patch is part of a series of patches aimed at generalizing affine
dependence analysis.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D110045
2021-09-21 13:02:23 +05:30
Chris Lattner 58abc8c34b [OpAsmParser] Add a parseCommaSeparatedList helper and beef up Delimeter.
Lots of custom ops have hand-rolled comma-delimited parsing loops, as does
the MLIR parser itself.  Provides a standard interface for doing this that
is less error prone and less boilerplate.

While here, extend Delimiter to support <> and {} delimited sequences as
well (I have a use for <> in CIRCT specifically).

Differential Revision: https://reviews.llvm.org/D110122
2021-09-20 20:59:11 -07:00
River Riddle d80d3a358f [mlir] Refactor ElementsAttr into an AttrInterface
This revision refactors ElementsAttr into an Attribute Interface.
This enables a common interface with which to interact with
element attributes, without needing to modify the builtin
dialect. It also removes a majority (if not all?) of the need for
the current OpaqueElementsAttr, which was originally intended as
a way to opaquely represent data that was not representable by
the other builtin constructs.

The new ElementsAttr interface not only allows for users to
natively represent their data in the way that best suits them,
it also allows for efficient opaque access and iteration of the
underlying data. Attributes using the ElementsAttr interface
can directly expose support for interacting with the held
elements using any C++ data type they claim to support. For
example, DenseIntOrFpElementsAttr supports iteration using
various native C++ integer/float data types, as well as
APInt/APFloat, and more. ElementsAttr instances that refer to
DenseIntOrFpElementsAttr can use all of these data types for
iteration:

```c++
DenseIntOrFpElementsAttr intElementsAttr = ...;

ElementsAttr attr = intElementsAttr;
for (uint64_t value : attr.getValues<uint64_t>())
  ...;
for (APInt value : attr.getValues<APInt>())
  ...;
for (IntegerAttr value : attr.getValues<IntegerAttr>())
  ...;
```

ElementsAttr also supports failable range/iterator access,
allowing for selective code paths depending on data type
support:

```c++
ElementsAttr attr = ...;
if (auto range = attr.tryGetValues<uint64_t>()) {
  for (uint64_t value : *range)
    ...;
}
```

Differential Revision: https://reviews.llvm.org/D109190
2021-09-21 01:57:43 +00:00
River Riddle 0cb5d7fc7f [mlir] Add value_begin/value_end methods to DenseElementsAttr
Currently DenseElementsAttr only exposes the ability to get the full range of values for a given type T, but there are many situations where we just want the beginning/end iterator. This revision adds proper value_begin/value_end methods for all of the supported T types, and also cleans up a bit of the interface.

Differential Revision: https://reviews.llvm.org/D104173
2021-09-21 01:57:43 +00:00
River Riddle 4f21152af1 [mlir] Tighten verification of SparseElementsAttr
SparseElementsAttr currently does not perform any verfication on construction, with the only verification existing within the parser. This revision moves the parser verification to SparseElementsAttr, and also adds additional verification for when a sparse index is not valid.

Differential Revision: https://reviews.llvm.org/D109189
2021-09-21 01:57:42 +00:00
Chia-hung Duan bb2506061b [mlir-tblgen] Add DagNode StaticMatcher.
Some patterns may share the common DAG structures. Generate a static
function to do the match logic to reduce the binary size.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D105797
2021-09-20 23:37:42 +00:00
natashaknk 4edf46f72a [mlir][tosa] Remove the documentation requirement for elements of several binary elementwise ops to be of the same rank.
Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D110095
2021-09-20 15:00:38 -07:00
MaheshRavishankar 4cf9bf6c9f [mlir][MemRef] Compute unused dimensions of a rank-reducing subviews using strides as well.
For `memref.subview` operations, when there are more than one
unit-dimensions, the strides need to be used to figure out which of
the unit-dims are actually dropped.

Differential Revision: https://reviews.llvm.org/D109418
2021-09-20 11:05:30 -07:00
MaheshRavishankar 0b33890f45 [mlir][Linalg] Add ConvolutionOpInterface.
Add an interface that allows grouping together all covolution and
pooling ops within Linalg named ops. The interface currently
- the indexing map used for input/image access is valid
- the filter and output are accessed using projected permutations
- that all loops are charecterizable as one iterating over
  - batch dimension,
  - output image dimensions,
  - filter convolved dimensions,
  - output channel dimensions,
  - input channel dimensions,
  - depth multiplier (for depthwise convolutions)

Differential Revision: https://reviews.llvm.org/D109793
2021-09-20 10:41:10 -07:00
Mehdi Amini 5edd79fc97 Revert "[MLIR][SCF] Add for-to-while loop transformation pass"
This reverts commit 644b55d57e.

The added test is failing the bots.
2021-09-20 17:21:59 +00:00
Morten Borup Petersen 644b55d57e [MLIR][SCF] Add for-to-while loop transformation pass
This pass transforms SCF.ForOp operations to SCF.WhileOp. The For loop condition is placed in the 'before' region of the while operation, and indctuion variable incrementation + the loop body in the 'after' region. The loop carried values of the while op are the induction variable (IV) of the for-loop + any iter_args specified for the for-loop.
Any 'yield' ops in the for-loop are rewritten to additionally yield the (incremented) induction variable.

This transformation is useful for passes where we want to consider structured control flow solely on the basis of a loop body and the computation of a loop condition. As an example, when doing high-level synthesis in CIRCT, the incrementation of an IV in a for-loop is "just another part" of a circuit datapath, and what we really care about is the distinction between our datapath and our control logic (the condition variable).

Differential Revision: https://reviews.llvm.org/D108454
2021-09-20 16:57:50 +01:00
Tobias Gysi 6db928b8f3 [mlir][linalg] Fusion on tensors.
Add a new version of fusion on tensors that supports the following scenarios:
- support input and output operand fusion
- fuse a producer result passed in via tile loop iteration arguments (update the tile loop iteration arguments)
- supports only linalg operations on tensors
- supports only scf::for
- cannot add an output to the tile loop nest

The LinalgTileAndFuseOnTensors pass tiles the root operation and fuses its producers.

Reviewed By: nicolasvasilache, mravishankar

Differential Revision: https://reviews.llvm.org/D109766
2021-09-20 14:45:34 +00:00
Vladislav Vinogradov ec03bbe8a7 [mlir] Fix bug in partial dialect conversion
The discussion on forum:
https://llvm.discourse.group/t/bug-in-partial-dialect-conversion/4115

The `applyPartialConversion` didn't handle the operations, that were
marked as illegal inside dynamic legality callback.
Instead of reporting error, if such operation was not converted to legal set,
the method just added it to `unconvertedSet` in the same way as unknown operations.

This patch fixes that and handle dynamically illegal operations as well.

The patch includes 2 fixes for existing passes:

* `tensor-bufferize` - explicitly mark `std.return` as legal.
* `convert-parallel-loops-to-gpu` - ugly fix with marking visited operations
  to avoid recursive legality checks.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D108505
2021-09-20 10:39:10 +03:00
Vladislav Vinogradov 9a2255dfa0 [mlir][NFC] Add explicit "::mlir" namespace to tblgen generated code
Reviewed By: lattner, ftynse

Differential Revision: https://reviews.llvm.org/D109223
2021-09-20 10:37:50 +03:00
Arjun P 33afea5488 [MLIR] Simplex: rename num{Variables,Constraints} to getNum{Variables,Constraints}
As per the LLVM Coding Standards, function names should be verb phrases.
2021-09-18 22:39:35 +05:30
Arjun P 2b44a7325c [MLIR] Simplex: support adding new variables dynamically
Reviewed By: Groverkss

Differential Revision: https://reviews.llvm.org/D109962
2021-09-18 21:32:17 +05:30
Jacques Pienaar 0a1e569d37 [mlir-c] Add getting fused loc
For creating a fused loc using array of locations and metadata.

Differential Revision: https://reviews.llvm.org/D110022
2021-09-18 06:57:51 -07:00
Uday Bondhugula 57eda9becc [MLIR][GPU] Add constant propagator for gpu.launch op
Add a constant propagator for gpu.launch op in cases where the
grid/thread IDs can be trivially determined to take a single constant
value of zero.

Differential Revision: https://reviews.llvm.org/D109994
2021-09-18 12:02:46 +05:30
Geoffrey Martin-Noble 2cda4f8ed7
[mlir] Fix syntax example for tensor.from_elements
Parens are not used here
2021-09-17 17:23:11 -07:00
Aart Bik 46e77b5d10 [mlir][sparse] add a sparse quantized_matmul example to integration test
Note that this revision adds a very tiny bit of constant folding in the
sparse compiler lattice construction. Although I am generally trying to
avoid such canonicalizations (and rely on other passes to fix this instead),
the benefits of avoiding a very expensive disjunction lattice construction
justify having this special code (at least for now).

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D109939
2021-09-17 13:04:44 -07:00
Tobias Gysi 90b7817e03 [mlir][linalg] Add helper to update IndexOps after tiling (NFC).
Add the addTileLoopIvsToIndexOpResults method to shift the IndexOp results after tiling.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D109761
2021-09-17 15:17:33 +00:00
Arjun P 44db07f11f [MLIR] AffineStructures: support removing a range of constraints at once
Reviewed By: Groverkss, grosser

Differential Revision: https://reviews.llvm.org/D109892
2021-09-17 16:27:48 +05:30
Arjun P 6607bd9fd8 [MLIR] AffineStructures::removeIdRange: support specifying a range within an IdKind
Reviewed By: Groverkss, grosser

Differential Revision: https://reviews.llvm.org/D109896
2021-09-17 16:25:26 +05:30
Arjun P f263ea1571 [MLIR] Matrix: support resizing horizontally
Reviewed By: Groverkss

Differential Revision: https://reviews.llvm.org/D109897
2021-09-17 16:22:31 +05:30
Tobias Gysi 8f2db36b01 [mlir][OpDSL] Update op definitions to make shapes more concise (NFC).
Express the input shape definitions of convolution and pooling operations in terms of the output shapes, filter shapes, strides, and dilations.

Reviewed By: shabalin, rsuderman, stellaraccident

Differential Revision: https://reviews.llvm.org/D109815
2021-09-16 06:02:00 +00:00
Mogball cb8c30d35d [DRR] Explicit Return Types in Rewrites
Adds a new rewrite directive returnType that can be added at the end of an op's
argument list to explicitly specify return types.

```
(OpX $v0, $v1, (returnType "$_builder.getI32Type()"))
```

Pass in a bound value to copy its return type, or pass a native code call to
dynamically create new types.

```
(OpX $v0, $v1, (returnType $v0, (NativeCodeCall<"..."> $v1)))
```

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D109472
2021-09-15 14:25:29 -07:00
Uday Bondhugula f68939d3d9 [MLIR] Tighten type constraint on memref.global op def
Tighten the def of memref.global op to use the right kind of TypeAttr
(of MemRefType).

Differential Revision: https://reviews.llvm.org/D109822
2021-09-15 22:41:03 +05:30
Nicolas Vasilache 6fe77b1051 [mlir][Linalg] Fail comprehensive bufferization if a memref is returned.
Summary:

Reviewers:

Subscribers:

Differential revision: https://reviews.llvm.org/D109824
2021-09-15 15:11:17 +00:00
Nicolas Vasilache 660f281b5e [mlir][Linalg] Make codegen strategy late transformations opt-in
Summary: Making the late transformations opt-in results in less surprising behavior when composing multiple calls to the codegen strategy.

Reviewers:

Subscribers:

Differential revision: https://reviews.llvm.org/D109820
2021-09-15 11:02:14 +00:00
Mehdi Amini a32300a68f Make the --mlir-disable-threading command line option overrides the C++ API usage
This seems in-line with the intent and how we build tools around it.
Update the description for the flag accordingly.
Also use an injected thread pool in MLIROptMain, now we will create
threads up-front and reuse them across split buffers.

Differential Revision: https://reviews.llvm.org/D109802
2021-09-15 03:20:48 +00:00
cwz920716 500d4c45ba [MLIR] Use memref.copy ops in BufferResultsToOutParams pass.
Both copy/alloc ops are using memref dialect after this change.

Reviewed By: silvas, mehdi_amini

Differential Revision: https://reviews.llvm.org/D109480
2021-09-15 02:59:30 +00:00
Tobias Gysi 44a889778c [mlir][linalg] Fold ExtractSliceOps during tiling.
Add the makeComposedExtractSliceOp method that creates an ExtractSliceOp and folds chains of ExtractSliceOps by computing the sum of their offsets and by multiplying their strides.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D109601
2021-09-14 11:43:52 +00:00
Matthias Springer fb1def9c66 [mlir][linalg] New tiling option: Scalarize dynamic dims
This tiling option scalarizes all dynamic dimensions, i.e., it tiles all dynamic dimensions by 1.

This option is useful for linalg ops with partly dynamic tensor dimensions. E.g., such ops can appear in the partial iteration after loop peeling. After scalarizing dynamic dims, those ops can be vectorized.

Differential Revision: https://reviews.llvm.org/D109268
2021-09-14 10:40:50 +09:00
Matthias Springer 8faf35c0a5 [mlir][linalg] Add scf.for loop peeling to codegen strategy
Only scf.for loops are supported at the moment. linalg.tiled_loop support will be added in a subsequent commit.

Only static tensor sizes are supported. Loops for dynamic tensor sizes can be peeled, but the generated code is not optimal due to a missing canonicalization pattern.

Differential Revision: https://reviews.llvm.org/D109043
2021-09-14 10:35:01 +09:00
Benoit Jacob 340314c4dc Reorder mmt4d shapes:
* Revert https://reviews.llvm.org/D107307 so that both LHS and RHS have
  the same layout with K0 as the innermost dimension.

* Continuing from https://reviews.llvm.org/D107003, move also 'K'
  to the outer side, so that now the inter-tile dimensions as all outer,
  and the intra-tile dimensions are all inner.

Reviewed By: asaadaldien

Differential Revision: https://reviews.llvm.org/D109692
2021-09-13 12:09:22 -07:00
Mathieu Fehr 802bf02a73 [mlir] Allows to query traits from types and attributes
Types and attributes now have a `hasTrait` function that allow users to check
if a type defines a trait.
Also, AbstractType and AbstractAttribute has now a `hasTraitFn` field to carry
the implementation of the `hasTrait` function of the concrete type or attribute.
This patch also adds the remaining functions to access type and attribute traits
in TableGen.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D105202
2021-09-13 06:26:45 +00:00
Mehdi Amini 7fb2394a4f Add sanity check in MLIR ODS to catch case where an arguments/results/regions/successors names overlap
This is making a tablegen crash with a more friendly error.

Differential Revision: https://reviews.llvm.org/D109474
2021-09-13 06:21:25 +00:00
Rob Suderman b0532286fe [mlir][tosa] Add shape inference for tosa.while
Tosa.while shape inference requires repeatedly running shape inference across
the body of the loop until the types become static as we do not know the number
of iterations required by the loop body. Once the least specific arguments are
known they are propagated to both regions.

To determine the final end type, the least restrictive types are determined
from all yields.

Differential Revision: https://reviews.llvm.org/D108801
2021-09-10 13:11:53 -07:00
Stephan Herhut 5e6c170b3f [mlir][linalg] Fix bufferize pattern to allow unknown operations in body of generic
The original version of the bufferization pattern for linalg.generic would
manually clone operations within the region to the bufferized clone of the
operation. This triggers legality requirements on those operations in the
conversion infra. Instead, this now uses the rewriter to inline the region
instead, avoiding those legality requirements.

Differential Revision: https://reviews.llvm.org/D109581
2021-09-10 13:37:42 +02:00
Matthias Springer 0f3544d185 [mlir][scf] Loop peeling: Use scf.for for partial iteration
Generate an scf.for instead of an scf.if for the partial iteration. This is for consistency reasons: The peeling of linalg.tiled_loop also uses another loop for the partial iteration.

Note: Canonicalizations patterns may rewrite partial iterations to scf.if afterwards.

Differential Revision: https://reviews.llvm.org/D109568
2021-09-10 19:07:09 +09:00
Tobias Gysi 16488dc300 [mlir][linalg] Pass all operands to tile to the tile loop region builder (NFC).
Extend the signature of the tile loop nest region builder to take all operand values to use and not just the scf::For iterArgs. This change allows us to pass in all block arguments of TiledLoop and use them directly instead of replacing them after the loop generation.

Reviewed By: pifon2a

Differential Revision: https://reviews.llvm.org/D109569
2021-09-10 08:35:11 +00:00
Aart Bik 066d786ce0 [mlir][sparse] add folding to sparse_tensor.convert
folds conversion between identical types (with tests)

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D109545
2021-09-09 15:45:19 -07:00
thomasraoux 2a69790bad [mlir][sparse] Mark convert op as noSideEffect
Differential Revision: https://reviews.llvm.org/D109543
2021-09-09 14:39:09 -07:00
Aart Bik e2d3db42e5 [mlir][sparse] add casts to operations to lattice and exp builders
Further enhance the set of operations that can be handled by the sparse compiler

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D109413
2021-09-09 08:49:50 -07:00
Alex Zinenko 8b58ab8ccd [mlir] Factor type reconciliation out of Standard-to-LLVM conversion
Conversion to the LLVM dialect is being refactored to be more progressive and
is now performed as a series of independent passes converting different
dialects. These passes may produce `unrealized_conversion_cast` operations that
represent pending conversions between built-in and LLVM dialect types.
Historically, a more monolithic Standard-to-LLVM conversion pass did not need
these casts as all operations were converted in one shot. Previous refactorings
have led to the requirement of running the Standard-to-LLVM conversion pass to
clean up `unrealized_conversion_cast`s even though the IR had no standard
operations in it. The pass must have been also run the last among all to-LLVM
passes, in contradiction with the partial conversion logic. Additionally, the
way it was set up could produce invalid operations by removing casts between
LLVM and built-in types even when the consumer did not accept the uncasted
type, or could lead to cryptic conversion errors (recursive application of the
rewrite pattern on `unrealized_conversion_cast` as a means to indicate failure
to eliminate casts).

In fact, the need to eliminate A->B->A `unrealized_conversion_cast`s is not
specific to to-LLVM conversions and can be factored out into a separate type
reconciliation pass, which is achieved in this commit. While the cast operation
itself has a folder pattern, it is insufficient in most conversion passes as
the folder only applies to the second cast. Without complex legality setup in
the conversion target, the conversion infra will either consider the cast
operations valid and not fold them (a separate canonicalization would be
necessary to trigger the folding), or consider the first cast invalid upon
generation and stop with error. The pattern provided by the reconciliation pass
applies to the first cast operation instead. Furthermore, having a separate
pass makes it clear when `unrealized_conversion_cast`s could not have been
eliminated since it is the only reason why this pass can fail.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D109507
2021-09-09 16:51:24 +02:00
Alex Zinenko 1ce752b741 [mlir] support reductions in SCF to OpenMP conversion
OpenMP reductions need a neutral element, so we match some known reduction
kinds (integer add/mul/or/and/xor, float add/mul, integer and float min/max) to
define the neutral element and the atomic version when possible to express
using atomicrmw (everything except float mul). The SCF-to-OpenMP pass becomes a
module pass because it now needs to introduce new symbols for reduction
declarations in the module.

Reviewed By: chelini

Differential Revision: https://reviews.llvm.org/D107549
2021-09-09 13:04:27 +02:00
Matthias Springer c95a7246a3 [mlir][linalg] Tiling: Use loop ub in extract_slice size computation if possible
When tiling a LinalgOp, extract_slice/insert_slice pairs are inserted. To avoid going out-of-bounds when the tile size does not divide the shape size evenly (at the boundary), AffineMin ops are inserted. Some ops have assumptions regarding the dimensions of inputs/outputs. E.g., in a `A * B` matmul, `dim(A, 1) == dim(B, 0)`. However, loop bounds use either `dim(A, 1)` or `dim(B, 0)`.

With this change, AffineMin ops are expressed in terms of loop bounds instead of tensor sizes. (Both have the same runtime value.) This simplifies canonicalizations.

Differential Revision: https://reviews.llvm.org/D109267
2021-09-09 11:06:22 +09:00
Kunwar Shaanjeet Singh Grover dea76ccaf4 [MLIR] FlatAffineConstraints: Refactored computation of explicit representation for identifiers
This patch refactors the existing implementation of computing an explicit
representation of an identifier as a floordiv in terms of other identifiers and
exposes this computation as a public function.

The computation of this representation is required to support local identifiers
in PresburgerSet subtract, complement and isEqual.

Reviewed By: bondhugula, arjunp

Differential Revision: https://reviews.llvm.org/D106662
2021-09-08 20:24:46 +05:30
Arnab Dutta 1524b01541 [MLIR] Add loop coalesce utility for affine.for
Add loop coalesce utility for affine.for. This expects loops to have
been normalized a-priori. This works for both constant as well non
constant upper bounds having single/multiple result upper bound affine
map.

With contributions from Arnab Dutta and Uday Bondhugula.

Reviewed By: bondhugula, ayzhuang

Differential Revision: https://reviews.llvm.org/D108126
2021-09-08 18:02:23 +05:30
Mehdi Amini ee903a207b Improve error message when creating an op that isn't registered in the context
This prints a more helpful error for folks who aren't intrinsically
familiar with the system.

Differential Revision: https://reviews.llvm.org/D109378
2021-09-07 20:42:30 +00:00
Matthias Springer c57c4f888c [mlir][linalg] linalg.tiled_loop peeling
Differential Revision: https://reviews.llvm.org/D108270
2021-09-07 09:50:08 +09:00
Loren Maggiore 361458b1ce [mlir] create gpu memset op
Create a gpu memset op and corresponding CUDA and ROCm wrappers.

Reviewed By: herhut, lorenrose1013

Differential Revision: https://reviews.llvm.org/D107548
2021-09-04 08:13:04 +02:00
Mehdi Amini 78accf9f35 Make LLVM Linkage a first class attribute instead of using an integer attribute
This makes the IR more readable, in particular when this will be used on
the builtin func outside of the LLVM dialect.

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D109209
2021-09-03 21:21:46 +00:00
Alexander Belyaev 5ee5bbd0ff [mlir][linalg] Extend tiled_loop to SCF conversion to generate scf.parallel.
Differential Revision: https://reviews.llvm.org/D109230
2021-09-03 18:05:54 +02:00
Matthias Springer 4fa6c2734c [mlir][scf] Allow runtime type of iter_args to change
The limitation on iter_args introduced with D108806 is too restricting. Changes of the runtime type should be allowed.

Extends the dim op canonicalization with a simple analysis to determine when it is safe to canonicalize.

Differential Revision: https://reviews.llvm.org/D109125
2021-09-03 10:03:05 +09:00
Kiran Chandramohan 711aa35759 [MLIR][OpenMP] Add support for declaring critical construct names
Add an operation omp.critical.declare to declare names/symbols of
critical sections. Named omp.critical operations should use symbols
declared by omp.critical.declare. Having a declare operation ensures
that the names of critical sections are global and unique. In the
lowering flow to LLVM IR, the OpenMP IRBuilder creates unique names
for critical sections.

Reviewed By: ftynse, jeanPerier

Differential Revision: https://reviews.llvm.org/D108713
2021-09-02 14:31:19 +00:00
Marius Brehler 2f0750dd2e [mlir] Add Cpp emitter
This upstreams the Cpp emitter, initially presented with [1], from [2]
to MLIR core. Together with the previously upstreamed EmitC dialect [3],
the target allows to translate MLIR to C/C++.

[1] https://reviews.llvm.org/D76571
[2] https://github.com/iml130/mlir-emitc
[3] https://reviews.llvm.org/D103969

Co-authored-by: Jacques Pienaar <jpienaar@google.com>
Co-authored-by: Simon Camphausen <simon.camphausen@iml.fraunhofer.de>
Co-authored-by: Oliver Scherf <oliver.scherf@iml.fraunhofer.de>

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D104632
2021-09-02 13:51:05 +00:00
Alex Zinenko 8647e4c3a0 [mlir] support translating OpenMP loops with reductions
Use the recently introduced OpenMPIRBuilder facility to transate OpenMP
workshare loops with reductions to LLVM IR calling OpenMP runtime. Most of the
heavy lifting is done at the OpenMPIRBuilder. When other OpenMP dialect
constructs grow support for reductions, the translation can be updated to
operate on, e.g., an operation interface for all reduction containers instead
of workshare loops specifically. Designing such a generic translation for the
single operation that currently supports reductions is premature since we don't
know how the reduction modeling itself will be generalized.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D107343
2021-09-02 15:38:20 +02:00
Alexander Belyaev f68de11c10 [mlir][linalg] Expose function to create op on buffers during bufferization.
Differential Revision: https://reviews.llvm.org/D109140
2021-09-02 11:09:05 +02:00
Jacques Pienaar f7bf8a8658 [mlir][capi] Add NameLoc
Add method to get NameLoc. Treat null child location as unknown to avoid
needing to create UnknownLoc in C API where child loc is not needed.

Differential Revision: https://reviews.llvm.org/D108678
2021-09-01 16:16:35 -07:00
Weiwei Li a79d7c2c85 [mlir][SPIRV] Add Image Operands for Image Instructions
This patch is to add Image Operands in SPIR-V Dialect and also let ImageDrefGather to use Image Operands.

Image Operands are used in many image instructions. "Image Operands encodes what oprands follow, as per Image Operands". And ususally, they are optional to image instructions.

The format of image operands looks like:

    %0 = spv.ImageXXXX %1, ... %3 : f32 ["Bias|Lod"](%4, %5 : f32, f32) -> ...

This patch doesn’t implement all operands (see Section 3.14 in SPIR-V Spec) but provides a skeleton of it. There is TODO in verifyImageOperands function.

Co-authored: Alan Liu <alanliu.yf@gmail.com>

Reviewed by: antiagainst

Differential Revision: https://reviews.llvm.org/D108501
2021-09-02 04:14:17 +08:00
Mehdi Amini 43a894365e Remove deprecated registration APIs (NFC)
In D104421, we changed the API for pass registration.
Before you would write:

      void registerPass("my-pass", "My Pass Description.",
                        [] { return createMyPass(); });
while now you’d only write:

      void registerPass([] { return createMyPass(); });

If you’re using TableGen to define your pass registration, you shouldn’t have anything to do. If you’re using directly the C++ API here are some changes.
Your project may also be broken even if you use TableGen and you call the
generated registration API in case your pass implementation didn’t inherit from
the MyPassBase class generated by TableGen.

If you don't use TableGen, the "my-pass" and "My Pass Description." fields must
be provided by overriding methods on the pass itself:

  llvm::StringRef getArgument() const final { return "my-pass"; }
  llvm::StringRef getDescription() const final {
    return "My Pass Description.";
  }

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D104429
2021-09-01 18:53:30 +00:00
Tyler Augustine 7105512a34 Support alias.scope and noalias metadata lowering on intrinsics.
Builds on https://reviews.llvm.org/D107870 to support annotating intrinsics with alias.scope and noalias metadata.

Reviewed By: arpith-jacob, ftynse

Differential Revision: https://reviews.llvm.org/D109025
2021-09-01 16:54:20 +00:00
MaheshRavishankar b686fdbf92 [mlir][Linalg] Drop output tensor from `linalg.pad_tensor` op.
The output tensor was added for tiling purposes. With use of
`TilingInterface` for tiling pad operations, there is no need for an
explicit operand for the shape of result of `linalg.pad_tensor`
op. The interface allows the tiling pattern to query the value that
can be used for the "init" needed for tiling dynamically.

Differential Revision: https://reviews.llvm.org/D108613
2021-08-31 11:12:24 -07:00
Mehdi Amini 387f95541b Add a new interface allowing to set a default dialect to be used for printing/parsing regions
Currently the builtin dialect is the default namespace used for parsing
and printing. As such module and func don't need to be prefixed.
In the case of some dialects that defines new regions for their own
purpose (like SpirV modules for example), it can be beneficial to
change the default dialect in order to improve readability.

Differential Revision: https://reviews.llvm.org/D107236
2021-08-31 17:52:40 +00:00
Mehdi Amini c41b16c26b Change ASM Op printer to print the operation name in the framework instead of leaving it up to each individual operation
This aligns the printer with the parser contract: the operation isn't part of the user-controllable part of the syntax.

Differential Revision: https://reviews.llvm.org/D108804
2021-08-31 17:52:40 +00:00
Mehdi Amini fd87963eee Change dialect `printOperation()` hook to `getOperationPrinter()`
This makes the hook return a printer if available, instead of using LogicalResult  to
indicate if a printer was available (and invoked). This allows the caller to detect that
the dialect has a printer for a given operation without actually invoking the printer.
It'll be leveraged in a future revision to move printing the op name itself under control
of the ASMPrinter.

Differential Revision: https://reviews.llvm.org/D108803
2021-08-31 17:52:39 +00:00
marina kolpakova a.k.a. geexie 0080d2aa55 [mlir][gpu] folds memref.dim of gpu.alloc
implements canonicalization which folds memref.dim(gpu.alloc(%size), %idx) -> %size

Differential Revision: https://reviews.llvm.org/D108892
2021-08-31 12:33:10 +03:00
MaheshRavishankar ba72cfe734 [mlir] Add an interface to allow operations to specify how they can be tiled.
An interface to allow for tiling of operations is introduced. The
tiling of the linalg.pad_tensor operation is modified to use this
interface.

Differential Revision: https://reviews.llvm.org/D108611
2021-08-30 16:31:18 -07:00
Chris Lattner faf1c22408 [Builder] Eliminate the StringRef/StringAttr forms of getSymbolRefAttr.
The StringAttr version doesn't need a context, so we can just use the
existing `SymbolRefAttr::get` form.  The StringRef version isn't preferred
so we want to encourage people to use StringAttr.

There is an additional form of getSymbolRefAttr that takes a (SymbolTrait
implementing) operation.  This should also be moved, but I'll do that as
a separate patch.

Differential Revision: https://reviews.llvm.org/D108922
2021-08-30 16:05:36 -07:00
Stella Laurenzo 8e6c55c92c [mlir][python] Extend C/Python API to be usable for CFG construction.
* It is pretty clear that no one has tried this yet since it was both incomplete and broken.
* Fixes a symbol hiding issues keeping even the generic builder from constructing an operation with successors.
* Adds ODS support for successors.
* Adds CAPI `mlirBlockGetParentRegion`, `mlirRegionEqual` + tests (and missing test for `mlirBlockGetParentOperation`).
* Adds Python property: `Block.region`.
* Adds Python methods: `Block.create_before` and `Block.create_after`.
* Adds Python property: `InsertionPoint.block`.
* Adds new blocks.py test to verify a plausible CFG construction case.

Differential Revision: https://reviews.llvm.org/D108898
2021-08-30 08:28:00 -07:00
Alex Zinenko 9db95a67d1 Fix interface trait declaration in SymbolInterfaces.td
41d4aa7de6 introduced incorrect code in
extraTraitClassDeclaration: `this` refers to the trait class and not the
operation class so `->getContext()` is not valid. Use `$_op` instead.
2021-08-30 11:15:05 +02:00
Chris Lattner 41d4aa7de6 [SymbolRefAttr] Revise SymbolRefAttr to hold a StringAttr.
SymbolRefAttr is fundamentally a base string plus a sequence
of nested references.  Instead of storing the string data as
a copies StringRef, store it as an already-uniqued StringAttr.

This makes a lot of things simpler and more efficient because:
1) references to the symbol are already stored as StringAttr's:
   there is no need to copy the string data into MLIRContext
   multiple times.
2) This allows pointer comparisons instead of string
   comparisons (or redundant uniquing) within SymbolTable.cpp.
3) This allows SymbolTable to hold a DenseMap instead of a
   StringMap (which again copies the string data and slows
   lookup).

This is a moderately invasive patch, so I kept a lot of
compatibility APIs around.  It would be nice to explore changing
getName() to return a StringAttr for example (right now you have
to use getNameAttr()), and eliminate things like the StringRef
version of getSymbol.

Differential Revision: https://reviews.llvm.org/D108899
2021-08-29 21:54:47 -07:00
Matthias Springer d18ffd61d4 [mlir][SCF] Canonicalize dim(x) where x is an iter_arg
* Add `DimOfIterArgFolder`.
* Move existing cross-dialect canonicalization patterns to `LoopCanonicalization.cpp`.
* Rename `SCFAffineOpCanonicalization` pass to `SCFForLoopCanonicalization`.
* Expand documentaton of scf.for: The type of loop-carried variables may not change with iterations. (Not even the dynamic type.)

Differential Revision: https://reviews.llvm.org/D108806
2021-08-30 01:39:56 +00:00
Matthias Springer eedc997b7d [mlir][Analysis] Add batched version of FlatAffineConstraints::addId
* Add batched version of all `addId` variants, so that multiple IDs can be added at a time.
* Rename `addId` and variants to `insertId` and `appendId`. Most external users call `appendId`. Splitting `addId` into two functions also makes it possible to provide batched version for both. (Otherwise, the overloads are ambigious when calling `addId`.)

Differential Revision: https://reviews.llvm.org/D108532
2021-08-30 00:56:44 +00:00
Markus Böck 0235e3c7a6 [mlir][NFC] Fully qualify default value of Attributes `getStorageType()` in files generated by mlir-tblgen 2021-08-28 15:37:56 +02:00
Mehdi Amini 022538f276 Remove `const` from `const T &&` in debugString() helper to make it a universal reference (NFC)
It broke lvalue arguments otherwise:
2021-08-28 01:09:00 +00:00
Mehdi Amini 4387975170 Use a universal reference (&& instead of const &) for `debugString()` helper (NFC)
Some classes like mlir::Operation have a non-const print() method.
2021-08-28 00:41:41 +00:00
Mehdi Amini c0b70def21 Specify argument to be `const` for `debugString()` helper (NFC)
This allows using this helper with rvalues.
2021-08-28 00:10:53 +00:00
Butygin 1e35a7690d [mlir][spirv] Initial support for 64 bit index type and builtins
Differential Revision: https://reviews.llvm.org/D108516
2021-08-27 01:38:53 +03:00
Tobias Gysi 8e9808ca3a [mlir][linalg] Tune hasTensorSemantics/hasBufferSemantics methods.
Optimize performance by iterating all operands at once.

Reviewed By: benvanik

Differential Revision: https://reviews.llvm.org/D108716
2021-08-25 19:28:37 +00:00
Tobias Gysi 2b35b372fd [mlir][linalg] Tune getTiedIndexingMap method (NFC).
Optimize the performance by using the range directly.

Reviewed By: benvanik

Differential Revision: https://reviews.llvm.org/D108715
2021-08-25 18:44:01 +00:00
River Riddle c8d9e1ce43 [mlir][AttrTypeGen] Add support for specifying a "accessor" type of a parameter
This allows for using a different type when accessing a parameter than the
one used for storage. This allows for returning parameters by reference,
enables using more optimized/convient reference results, and more.

Differential Revision: https://reviews.llvm.org/D108593
2021-08-25 09:27:36 +00:00
River Riddle 9658b061dd [mlir] Update DialectAsmParser::parseString to use std::string instead of StringRef
This allows for parsing strings that have escape sequences, which require constructing
a string (as they can't be represented by looking at the Token contents directly).

Differential Revision: https://reviews.llvm.org/D108589
2021-08-25 09:27:35 +00:00
River Riddle aea3026ea7 [mlir] Move the Operation use iteration utilities to ResultRange
This allows for iterating and interacting with the uses of a specific subset of
results as opposed to just the full range.

Differential Revision: https://reviews.llvm.org/D108586
2021-08-25 09:27:35 +00:00
Matthias Springer a9cff97f94 [mlir][SCF] Generalize AffineMinSCFCanonicalization to min/max ops
* Add support for affine.max ops to SCF loop peeling pattern.
* Add support for affine.max ops to `AffineMinSCFCanonicalizationPattern`.
* Rename `AffineMinSCFCanonicalizationPattern` to `AffineOpSCFCanonicalizationPattern`.
* Rename `AffineMinSCFCanonicalization` pass to `SCFAffineOpCanonicalization`.

Differential Revision: https://reviews.llvm.org/D108009
2021-08-25 10:40:34 +09:00
Matthias Springer 2de2dbef2a [mlir][linalg] Replace AffineMinSCFCanonicalizationPattern with SCF reimplementation
Use the new canonicalization pattern in the SCF dialect.

Differential Revision: https://reviews.llvm.org/D107732
2021-08-25 08:52:56 +09:00
Matthias Springer 98aa694d0d [mlir][scf] Add general affine.min canonicalization pattern
This canonicalization simplifies affine.min operations inside "for loop"-like operations (e.g., scf.for and scf.parallel) based on two invariants:
* iv >= lb
* iv < lb + step * ((ub - lb - 1) floorDiv step) + 1

This commit adds a new pass `canonicalize-scf-affine-min` (instead of being a canonicalization pattern) to avoid dependencies between the Affine dialect and the SCF dialect.

Differential Revision: https://reviews.llvm.org/D107731
2021-08-25 07:32:30 +09:00
Tyler Augustine d25e91d7f6 Support alias.scope and noalias metadata
Introduces new Ops to represent 1. alias.scope metadata in LLVM, and 2. domains for these scopes. These correspond to the metadata described in https://llvm.org/docs/LangRef.html#noalias-and-alias-scope-metadata. Lists of scopes are modeled the same way as access groups - as an ArrayAttr on the Op (added in https://reviews.llvm.org/D97944).

Lowering 'noalias' attributes on function parameters is already supported. However, lowering `noalias` metadata on individual Ops is not, which is added in this change. LLVM uses the same keyword for these, but this change introduces a separate attribute name 'noalias_scopes' to represent this distinct concept.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D107870
2021-08-24 20:42:59 +02:00
Aart Bik a643bd3189 [mlir] add permutation utility
I found myself typing this code several times at different places
by now, so time to make this a general utility instead. Given
a permutation, it returns the permuted position of the input,
for example (i,j,k) -> (k,i,j) yields position 1 for input 0.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D108347
2021-08-24 08:07:40 -07:00
River Riddle 4e103a12d9 [mlir] Add support for VariadicOfVariadic operands
This revision adds native ODS support for VariadicOfVariadic operand
groups. An example of this is the SwitchOp, which has a variadic number
of nested operand ranges for each of the case statements, where the
number of case statements is variadic. Builtin ODS support allows for
generating proper accessors for the nested operand ranges, builder
support, and declarative format support. VariadicOfVariadic operands
are supported by providing a segment attribute to use to store the
operand groups, mapping similarly to the AttrSizedOperand trait
(but with a user defined attribute name).

`build` methods for VariadicOfVariadic operand expect inputs of the
form `ArrayRef<ValueRange>`. Accessors for the variadic ranges
return a new `OperandRangeRange` type, which represents a
contiguous range of `OperandRange`. In the declarative assembly
format, VariadicOfVariadic operands and types are by default
formatted as a comma delimited list of value lists:
`(<value>, <value>), (), (<value>)`.

Differential Revision: https://reviews.llvm.org/D107774
2021-08-23 20:32:31 +00:00
MaheshRavishankar 4aeeb91a92 [mlir][Linalg] Allow all build methods of Structured ops to specify additional attributes.
Differential Revision: https://reviews.llvm.org/D108338
2021-08-23 13:06:34 -07:00
River Riddle da12d88b1c [mlir][NFC] Add inlineRegion overloads that take a block iterator insert position
This allows for inlining into an empty block or to the beginning of a block. NFC as the existing implementations now foward to this overload.

Differential Revision: https://reviews.llvm.org/D108572
2021-08-23 19:49:53 +00:00
Krzysztof Drewniak 469172f3f4 [MLIR][Docs] Fix broken link to tuple type rationale
Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D108135
2021-08-23 18:35:36 +00:00
Matthias Springer bc194a5bb5 [mlir][SCF] Do not peel loops inside partial iterations
Do not apply loop peeling to loops that are contained in the partial iteration of an already peeled loop. This is to avoid code explosion when dealing with large loop nests. Can be controlled with a new pass option `skip-partial`.

Differential Revision: https://reviews.llvm.org/D108542
2021-08-23 21:35:46 +09:00
Stella Laurenzo a8de667af0 [mlir] Add op for NCHW conv2d.
* This is the native data layout for PyTorch and npcomp was using the prior version before cleanup.

Differential Revision: https://reviews.llvm.org/D108527
2021-08-22 17:27:33 -07:00
Stella Laurenzo 64e74e9d7c [mlir][linalg] Add script to update the LinalgNamedStructuredOps.yaml. nfc
Also adds banners to the files with update instructions.

Differential Revision: https://reviews.llvm.org/D108529
2021-08-22 16:54:51 -07:00
Stella Laurenzo e78b745cf2 [mlir][python] Makes C++ extension code relocatable by way of a macro.
* Resolves a TODO by making this configurable by downstreams.
* This seems to be the last thing allowing full use of the Python bindings as a library within another project (i.e. be embedding them).

Differential Revision: https://reviews.llvm.org/D108523
2021-08-22 13:46:14 -07:00
Rob Suderman 871c812483 [mlir][linalg] Finish refactor of TC ops to YAML
Multiple operations were still defined as TC ops that had equivalent versions
as YAML operations. Reducing to a single compilation path guarantees that
frontends can lower to their equivalent operations without missing the
optimized fastpath.

Some operations are maintained purely for testing purposes (mainly conv{1,2,3}D
as they are included as sole tests in the vectorizaiton transforms.

Differential Revision: https://reviews.llvm.org/D108169
2021-08-20 12:35:04 -07:00
Vladislav Vinogradov 9775c0c9f0 [mlir] Fix ControlFlowInterfaces implementation for Async dialect
* Add `RegionBranchTerminatorOpInterface` to `YieldOp`.
* Implement `getSuccessorEntryOperands` in `ExecuteOp`.
* Fix `getSuccessorRegions` implementation in `ExecuteOp`.

Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D108373
2021-08-20 12:14:45 +03:00
Rob Suderman 3205ee7e81 [mlir][tosa] Support UInt8 inputs and outputs for tosa.rescale
Tosa rescale can contain uint8 types. Added support for these types
using an unrealized conversion cast. Optimistically it would be better to
use bitcast however it does not support unsigned integers.

Differential Revision: https://reviews.llvm.org/D108427
2021-08-19 18:58:44 -07:00
MaheshRavishankar 16ffb283c5 Revert "[mlir][Linalg] Allow all build methods of Structured ops to specify additional attributes."
This reverts commit 95ddc8341a.

Differential Revision: https://reviews.llvm.org/D108396
2021-08-19 11:53:41 -07:00
MaheshRavishankar 95ddc8341a [mlir][Linalg] Allow all build methods of Structured ops to specify additional attributes.
Differential Revision: https://reviews.llvm.org/D108338
2021-08-19 11:14:35 -07:00
Matthias Springer 76a1861816 [mlir][SparseTensor] Split scf.for loop into masked/unmasked parts
Apply the "for loop peeling" pattern from SCF dialect transforms. This pattern splits scf.for loops into full and partial iterations. In the full iteration, all masked loads/stores are canonicalized to unmasked loads/stores.

Differential Revision: https://reviews.llvm.org/D107733
2021-08-19 21:53:11 +09:00
Matthias Springer 8e8b70aa84 [mlir][scf] Simplify affine.min ops after loop peeling
Simplify affine.min ops, enabling various other canonicalizations inside the peeled loop body.

affine.min ops such as:
```
map = affine_map<(d0)[s0, s1] -> (s0, -d0 + s1)>
%r = affine.min #affine.min #map(%iv)[%step, %ub]
```
are rewritten them into (in the case the peeled loop):
```
%r = %step
```

To determine how an affine.min op should be rewritten and to prove its correctness, FlatAffineConstraints is utilized.

Differential Revision: https://reviews.llvm.org/D107222
2021-08-19 17:24:53 +09:00
Matthias Springer 9329438244 [mlir][linalg] Remove ConstraintsSet class
The same functionality can be implemented with FlatAffineValueConstraints.

Differential Revision: https://reviews.llvm.org/D108179
2021-08-19 10:57:35 +09:00
Matthias Springer c777e51468 [mlir][Analysis][NFC] FlatAffineConstraints: Use BoundType enum in functions
Differential Revision: https://reviews.llvm.org/D108185
2021-08-19 10:33:42 +09:00
Chia-hung Duan 41e5dbe0fa Enables inferring return types for Shape op if possible
Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D102565
2021-08-18 21:36:55 +00:00
Butygin ddc3d51d58 [mlir][spirv] Add (InBounds)PtrAccessChain ops
Differential Revision: https://reviews.llvm.org/D108070
2021-08-18 17:59:21 +03:00
Jacques Pienaar b41bfb819d [mlir][ods] Fix packing in OperandOrAttribute
Wrong combiner was used which led to information loss.
2021-08-17 20:55:48 -07:00
Tobias Gysi 583a754248 [mlir][linalg] Remove duplicate methods (NFC).
Remove duplicate methods used to check iterator types.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D108102
2021-08-17 09:06:17 +00:00
Matthias Springer c19c51e357 [mlir][Analysis][NFC] Clean up FlatAffineValueConstraints
* Rename ids to values in FlatAffineValueConstraints.
* Overall cleanup of comments in FlatAffineConstraints and FlatAffineValueConstraints.

Differential Revision: https://reviews.llvm.org/D107947
2021-08-17 10:38:57 +09:00
Matthias Springer 4c4ab673f1 [mlir][Analysis][NFC] Split FlatAffineConstraints class
* Extract "value" functionality of `FlatAffineConstraints` into a new derived `FlatAffineValueConstraints` class. Current users of `FlatAffineConstraints` can use `FlatAffineValueConstraints` without additional code changes, thus NFC.
* `FlatAffineConstraints` no longer associates dimensions with SSA Values. All functionality that requires this, is moved to `FlatAffineValueConstraints`.
* `FlatAffineConstraints` no longer makes assumptions about where Values associated with dimensions are coming from.

Differential Revision: https://reviews.llvm.org/D107725
2021-08-17 10:09:17 +09:00
Geoffrey Martin-Noble e2c97d4484 [MLIR] Add a bitcast method to DenseElementsAttr
This method bitcasts a DenseElementsAttr elementwise to one of the same
shape with a different element type.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D107612
2021-08-16 17:13:35 -07:00
Rob Suderman f328f72e60 [mlir][tosa] Fixed depthwise conv parallel/reduction indices order
Reduction axis should come after all parallel axis to work with vectorization.

Reviewed By: NatashaKnk

Differential Revision: https://reviews.llvm.org/D108005
2021-08-16 14:06:22 -07:00
Robert Suderman 65532ea6dd [mlir][linalg] Clear unused linalg tc operations
These operations are not lowered to from any source dialect and are only
used for redundant tests. Removing these named ops, along with their
associated tests, will make migration to YAML operations much more
convenient.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D107993
2021-08-16 11:55:45 -07:00
tashuang.zk 2d45e332ba [MLIR][DISC] Revise ParallelLoopTilingPass with inbound_check mode
Expand ParallelLoopTilingPass with an inbound_check mode.

In default mode, the upper bound of the inner loop is from the min op; in
inbound_check mode, the upper bound of the inner loop is the step of the outer
loop and an additional inbound check will be emitted inside of the inner loop.

This was 'FIXME' in the original codes and a typical usage is for GPU backends,
thus the outer loop and inner loop can be mapped to blocks/threads in seperate.

Differential Revision: https://reviews.llvm.org/D105455
2021-08-16 14:02:53 +02:00
Stephen Neuendorffer 7776b19eed [MLIR] Move TestDialect to ::test namespace
While the changes are extensive, they basically fall into a few
categories:
1) Moving the TestDialect itself.
2) Updating C++ code in tablegen to explicitly use ::mlir, since it
will be put in a headers that shouldn't expect a 'using'.
3) Updating some generic MLIR Interface definitions to do the same thing.
4) Updating the Tablegen generator in a few places to be explicit about
namespaces
5) Doing the same thing for llvm references, since we no longer pick
up the definitions from mlir/Support/LLVM.h

Differential Revision: https://reviews.llvm.org/D88251
2021-08-14 13:24:41 -07:00
harsh-nod e33f301ec2 [mlir] Add support for moving reductions to outer most dimensions in vector.multi_reduction
The approach for handling reductions in the outer most
dimension follows that for inner most dimensions, outlined
below

First, transpose to move reduction dims, if needed
Convert reduction from n-d to 2-d canonical form
Then, for outer reductions, we emit the appropriate op
(add/mul/min/max/or/and/xor) and combine the results.

Differential Revision: https://reviews.llvm.org/D107675
2021-08-13 12:59:50 -07:00
Lorenzo Chelini e537a3adde [MLIR][Linalg] Fix typo 2021-08-13 18:00:14 +02:00
natashaknk ba0997ca09 [mlir][tosa] Fix depthwise_conv2D strides/dilation and name
Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D107997
2021-08-12 15:43:41 -07:00
Chia-hung Duan 62df4df41c [mlir-tblgen] Minor Refactor for StaticVerifierFunctionEmitter.
Move StaticVerifierFunctionEmitter to CodeGenHelper.h so that it can be
used for both ODS and DRR.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D106636
2021-08-12 20:53:05 +00:00
Tyler Augustine 3a2ff982d7 Support post-processing Ops in unrolled loop iterations
This can be useful when one needs to know which unrolled iteration an Op belongs to, for example, conveying noalias information among memory-affecting ops in parallel-access loops.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D107789
2021-08-11 23:11:10 +00:00
Rob Suderman 7de439b2be [mlir][tosa] Migrate tosa to more efficient linalg.conv
Existing linalg.conv2d is not well optimized for performance. Changed to a
version that is more aligned for optimziation. Include the corresponding
transposes to use this optimized version.

This also splits the conv and depthwise conv into separate implementations
to avoid overly complex lowerings.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D107504
2021-08-11 11:05:12 -07:00
Alex Zinenko 79b0576dd4 [mlir] Tighten LLVM_AnyNonAggregate ODS type constraint
The constraint was checking that the type is not an LLVM structure or array
type, but was not checking that it is an LLVM-compatible type, making it accept
incorrect types. As a result, some LLVM dialect ops could process values that
are not compatible with the LLVM dialect leading to further issues with
conversions and translations that assume all values are LLVM-compatible. Make
LLVM_AnyNonAggregate only accept LLVM-compatible types.

Reviewed By: cota, akuegel

Differential Revision: https://reviews.llvm.org/D107889
2021-08-11 16:30:19 +02:00
Matthias Springer 4b56e2ee1d [mlir][Analysis][NFC] Remove code duplication around getFlattenedAffineExprs
Remove code duplication in `addLowerOrUpperBound` and `composeMatchingMap`.

Differential Revision: https://reviews.llvm.org/D107814
2021-08-11 16:02:10 +09:00
Matthias Springer 9e6e08149c [mlir][Analysis][NFC] Reimplement FlatAffineConstraints::composeMap
Reimplement this function in terms of `composeMatchingMap`.

Also fix a bug in `composeMatchingMap` where local dims of `this` could be missing in `localCst`.

Differential Revision: https://reviews.llvm.org/D107813
2021-08-11 15:49:50 +09:00
Matthias Springer 97e41c004c [mlir][Analysis] Add FlatAffineConstraints::addLowerOrUpperBound
This function overload is similar to the existing `FlatAffineConstraints::addLowerOrUpperBound`. It constrains a dimension based on an affine map. However, in contrast to the other overloading, it does not attempt to align dimensions/symbols of the affine map with the dimensions/symbols of the constraint set. Instead, dimensions/symbols are expected to already be aligned.

Differential Revision: https://reviews.llvm.org/D107727
2021-08-11 15:13:48 +09:00
Matthias Springer 9832e1a079 [mlir][Analysis] Add alignAffineMapWithValues
This function aligns an affine map (and operands) with given dims and syms SSA values.

This is useful in conjunction with `FlatAffineConstraints::addLowerOrUpperBound`, which requires the `boundMap` to be aligned with the constraint set's dims and syms.

Differential Revision: https://reviews.llvm.org/D107728
2021-08-11 14:59:03 +09:00
Rob Suderman 2b2ebb6f98 [mlir][tosa] Add folders for trivial tosa operation cases
Some folding cases are trivial to fold away, specifically no-op cases where
an operation's input and output are the same. Canonicalizing these away
removes unneeded operations.

The current version includes tensor cast operations to resolve shape
discreprencies that occur when an operation's result type differs from the
input type. These are resolved during a tosa shape propagation pass.

Reviewed By: NatashaKnk

Differential Revision: https://reviews.llvm.org/D107321
2021-08-10 14:43:00 -07:00
natashaknk a1f46569a1 [mlir][tosa] Add quantized and unquantized versions for tosa.depthwise_conv2d lowering
Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D107855
2021-08-10 14:29:26 -07:00
Haruki Imai b34b1c6955 [mlir] Support normalizing memrefs with MemRef_ReinterpretCastOp
This patch enables normalizing memrefs with MemRef_ReinterpretCastOp by
adding MemRefsNormalizable trait in the Op definition.

Signed-off-by: Haruki Imai <imaihal@jp.ibm.com>

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D107425
2021-08-11 01:15:18 +05:30
Jacques Pienaar 093493032d [mlir] Enable specifying querying function in ValueShapeRange
This enables querying shapes/values as shapes without mutating the IR
directly (e.g., towards enabling doing inference in analysis &
application steps, inferring function shape with constant from callsite,
...). Add a new ShapeAdaptor that abstracts over whether shape is from
Type or ShapedTypeComponents or DenseIntElementsAttribute. This adds new
accessors to ValueShapeRange to get Shape and value as shape, but
doesn't restrict or remove the previous way of accessing Type via the
Value for now, that does mean a less refined shape could be accidentally
queried and will be restricted in follow up.

Currently restricted Value query to what can be represented as Shape. So
only supports cases where constant subgraph evaluation's output is a
shape. I had considered making it more general, but without TBD extern
attribute concept or some such a user cannot today uniformly avoid
overhead.

Update TOSA ops and also the shape inference pass.

Differential Revision: https://reviews.llvm.org/D107768
2021-08-10 11:44:20 -07:00
Matthias Springer ab03a84e2c [mlir][Analysis][NFC] Disambiguate FlatAffineConstraints constructor
The following constructor call (and others) used to be ambiguous:

```
FlatAffineConstraints constraints(0, 0, 0);
```

Differential Revision: https://reviews.llvm.org/D107726
2021-08-10 10:34:30 +09:00
Matthias Springer 27c2fa4f05 [mlir][Analysis] Revert D107221
This is in preparation of a larger refactoring that makes the changes in D107221 obsolete.

Differential Revision: https://reviews.llvm.org/D107724
2021-08-10 10:17:49 +09:00
Max Kudryavtsev 0b8cb87e0d [MLIR][STD] Add safe scalar constant propagation for FPTruncOp
Perform scalar constant propagation for FPTruncOp only if the resulting value can be represented without precision loss or rounding.

Example:
%cst = constant 1.000000e+00 : f32
%0 = fptrunc %cst : f32 to bf16
-->
%cst = constant 1.000000e+00 : bf16

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D107518
2021-08-06 16:31:29 -07:00
Markus Böck 5522ec00bc [mlir][NFC] Fix typos in DataLayoutInterfaces.td 2021-08-06 18:54:25 +02:00
Geoffrey Martin-Noble ca6baf1e1d [MLIR][std] Introduce bitcast operation
This patch introduces a bitcast operation to the standard dialect.
RFC: https://llvm.discourse.group/t/rfc-introduce-a-bitcast-op/3774

Reviewed By: silvas

Differential Revision: https://reviews.llvm.org/D105376
2021-08-06 08:47:51 -07:00
Alexander Belyaev aa2210a830 [linalg] Expose `rewriteAsPaddedOp` function.
Differential Revision: https://reviews.llvm.org/D107629
2021-08-06 12:08:12 +02:00
Vladislav Vinogradov 59f59d1c62 [mlir] Allow to override type/attr aliases from various hooks
Use new return type for `OpAsmDialectInterface::getAlias`:

* `AliasResult::NoAlias` if an alias was not provided.
* `AliasResult::OverridableAlias` if an alias was provided, but it might be overriden by other hook.
* `AliasResult::FinalAlias` if an alias was provided and it should be used (no other hooks will be checked).

In that case `AsmPrinter` will use either the first alias with `FinalAlias` result or
the last alias with `OverridableAlias` result (it depends on dialect array order).

Used `OverridableAlias` result for `BuiltinOpAsmDialectInterface`.

Use case: provide more informative alias for built-in attributes like `AffineMapAttr`
instead of generic "map<N>".

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D107437
2021-08-06 12:05:31 +03:00
Matthias Springer ac7c7cbfda [mlir] Update comment in Region.h
The file in which `Region::viewGraph` is defined has changed. This should have been updated with D106342.

Differential Revision: https://reviews.llvm.org/D107517
2021-08-05 10:57:34 +09:00