Commit Graph

5560 Commits

Author SHA1 Message Date
River Riddle c1c1df6347 [mlir] Fix region successor bug in forward dataflow analysis
We weren't properly visiting region successors when the terminator wasn't return like, which could create incorrect results in the analysis. This revision ensures that we properly visit region successors, to avoid optimistically assuming a value is constant when it isn't.

Differential Revision: https://reviews.llvm.org/D101783
2021-05-04 14:50:37 -07:00
Rob Suderman 1f7adf8cb1 [mlir][tosa] Fix tosa.concat by inserting linalg.fill after linalg.init
All linalg.init operations must be fed into a linalg operation before
subtensor. The inserted linalg.fill guarantees it executes correctly.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D101848
2021-05-04 14:26:28 -07:00
William S. Moses cb395b84b0 [MLIR] Add not icmp canonicalization documentation
See: https://reviews.llvm.org/D101710
2021-05-04 11:44:25 -04:00
William S. Moses 8e211bf1c8 [MLIR][SCF] Assume uses of condition in the body of scf.while is true
Differential Revision: https://reviews.llvm.org/D101801
2021-05-04 11:39:07 -04:00
William S. Moses 93297e4bac [MLIR] Replace a not of a comparison with appropriate comparison
Differential Revision: https://reviews.llvm.org/D101710
2021-05-04 11:23:29 -04:00
Tobias Gysi 05d2297b86 [mlir][linalg] Always lower index operations during loop lowering.
Ensure the index operations are lowered on all linalg loop lowering paths.

Differential Revision: https://reviews.llvm.org/D101827
2021-05-04 14:30:59 +00:00
Adrian Kuegel 93537fabce [mlir] Add lowering from math.expm1 to LLVM.
Differential Revision: https://reviews.llvm.org/D96776
2021-05-04 14:22:10 +02:00
Matthias Springer aa58281979 [mlir] Fix bug in TransferOpReduceRank when all dims are broadcasts
TransferReadOps that are a scalar read + broadcast are handled by TransferReadToVectorLoadLowering.

Differential Revision: https://reviews.llvm.org/D101808
2021-05-04 11:21:44 +09:00
natashaknk 07ce5c99d7 [mlir][tosa] Add lowerings for tosa.equal and tosa.arithmetic_right_shift
Lowerings equal and arithmetic_right_shift for elementwise ops to linalg dialect using linalg.generic

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D101804
2021-05-03 18:26:49 -07:00
Eugene Zhulenev 9b67096fe9 [mlir] Linalg: add vector transfer lowering patterns to the contraction lowering
This fixes a performance regression in vec-mat vectorization

Reviewed By: asaadaldien

Differential Revision: https://reviews.llvm.org/D101795
2021-05-03 16:21:51 -07:00
Emilio Cota 1c0374e770 [mlir] Add polynomial approximation for math::Log1p
This approximation matches the one in Eigen.

```
name                      old cpu/op  new cpu/op  delta
BM_mlir_Log1p_f32/10      83.2ns ± 7%  34.8ns ± 5%  -58.19%    (p=0.000 n=84+71)
BM_mlir_Log1p_f32/100      664ns ± 4%   129ns ± 4%  -80.57%    (p=0.000 n=82+82)
BM_mlir_Log1p_f32/1k      6.75µs ± 4%  0.81µs ± 3%  -88.07%    (p=0.000 n=88+79)
BM_mlir_Log1p_f32/10k     76.5µs ± 3%   7.8µs ± 4%  -89.84%    (p=0.000 n=80+80)
BM_eigen_s_Log1p_f32/10   70.1ns ±14%  72.6ns ±14%   +3.49%  (p=0.000 n=116+112)
BM_eigen_s_Log1p_f32/100   706ns ± 9%   717ns ± 3%   +1.60%   (p=0.018 n=117+80)
BM_eigen_s_Log1p_f32/1k   8.26µs ± 1%  8.26µs ± 1%     ~       (p=0.567 n=84+86)
BM_eigen_s_Log1p_f32/10k  92.1µs ± 5%  92.6µs ± 6%   +0.60%  (p=0.047 n=115+115)
BM_eigen_v_Log1p_f32/10   31.8ns ±24%  34.9ns ±17%   +9.72%    (p=0.000 n=98+96)
BM_eigen_v_Log1p_f32/100   169ns ±10%   177ns ± 5%   +4.66%   (p=0.000 n=119+81)
BM_eigen_v_Log1p_f32/1k   1.42µs ± 4%  1.46µs ± 8%   +2.70%   (p=0.000 n=93+113)
BM_eigen_v_Log1p_f32/10k  14.4µs ± 5%  14.9µs ± 8%   +3.61%  (p=0.000 n=115+110)
```

Reviewed By: ezhulenev, ftynse

Differential Revision: https://reviews.llvm.org/D101765
2021-05-03 15:11:37 -07:00
MaheshRavishankar a6e09391bb [mlir][Linalg] Add a utility method to get reassociations maps for reshape.
Given the source and destination shapes, if they are static, or if the
expanded/collapsed dimensions are unit-extent, it is possible to
compute the reassociation maps that can be used to reshape one type
into another. Add a utility method to return the reassociation maps
when possible.

This utility function can be used to fuse a sequence of reshape ops,
given the type of the source of the producer and the final result
type. This pattern supercedes a more constrained folding pattern added
to DropUnitDims pass.

Differential Revision: https://reviews.llvm.org/D101343
2021-05-03 14:40:15 -07:00
MaheshRavishankar fd15e2b825 [mlir][Linalg] Use rank-reduced versions of subtensor and subtensor insert when possible.
Convert subtensor and subtensor_insert operations to use their
rank-reduced versions to drop unit dimensions.

Differential Revision: https://reviews.llvm.org/D101495
2021-05-03 12:51:24 -07:00
thomasraoux 9621c1ef56 [mlir][linalg] Fix vectorization bug in vector transfer indexing map calculation
The current implementation had a bug as it was relying on the target vector
dimension sizes to calculate where to insert broadcast. If several dimensions
have the same size we may insert the broadcast on the wrong dimension. The
correct broadcast cannot be inferred from the type of the source and
destination vector.

Instead when we want to extend transfer ops we calculate an "inverse" map to the
projected permutation and insert broadcast in place of the projected dimensions.

Differential Revision: https://reviews.llvm.org/D101738
2021-05-03 12:16:38 -07:00
Frederik Gossen 456efbc0f1 [MLIR][Linalg] Avoid forward declaration in `Loops.cpp`
Differential Revision: https://reviews.llvm.org/D101771
2021-05-03 21:06:50 +02:00
Frederik Gossen ec339163a7 [MLIR][Linalg] Lower `linalg.tiled_loop` in a separate pass
Add dedicated pass `convert-linalg-tiled-loops-to-scf` to lower
`linalg.tiled_loop`s.

Differential Revision: https://reviews.llvm.org/D101768
2021-05-03 21:02:02 +02:00
Stella Laurenzo 9f3f6d7bd8 Move MLIR python sources to mlir/python.
* NFC but has some fixes for CMake glitches discovered along the way (things not cleaning properly, co-mingled depends).
* Includes previously unsubmitted fix in D98681 and a TODO to fix it more appropriately in a smaller followup.

Differential Revision: https://reviews.llvm.org/D101493
2021-05-03 18:36:48 +00:00
thomasraoux d51275cbc0 [mlir][spirv] Add support to convert std.splat op
Differential Revision: https://reviews.llvm.org/D101511
2021-05-03 10:57:40 -07:00
thomasraoux f44c76d6e9 [mlir][vector] Extend vector transfer unrolling to support permutations and broadcast
Differential Revision: https://reviews.llvm.org/D101637
2021-05-03 10:47:02 -07:00
thomasraoux 7417541fd8 [mlir][vector] Add canonicalization for extract/insert -> shapecast
Differential Revision: https://reviews.llvm.org/D101643
2021-05-03 10:41:15 -07:00
Benjamin Kramer 96a7900eb0 [mlir] Fix multidimensional lowering from std.select to llvm.select
The converter assumed that all operands have the same type, that's not
true for select.

Differential Revision: https://reviews.llvm.org/D101767
2021-05-03 19:30:49 +02:00
thomasraoux be8e2801a4 [mlir][vector][NFC] split TransposeOp lowerning out of contractLowering
Move TransposeOp lowering in its own populate function as in some cases
it is better to keep it during ContractOp lowering to better
canonicalize it rather than emiting scalar insert/extract.

Differential Revision: https://reviews.llvm.org/D101647
2021-05-03 10:23:45 -07:00
Stella Laurenzo b57d6fe42e [mlir][Python] Add casting constructor to Type and Attribute.
* This makes them consistent with custom types/attributes, whose constructors will do a type checked conversion. Of course, the base classes can represent everything so never error.
* More importantly, this makes it possible to subclass Type and Attribute out of tree in sensible ways.

Differential Revision: https://reviews.llvm.org/D101734
2021-05-03 10:12:03 -07:00
Frederik Gossen d2a291a5f8 [MLIR][Linalg] Lower `linalg.tiled_loop` to `scf` loops
Differential Revision: https://reviews.llvm.org/D101747
2021-05-03 18:47:12 +02:00
William S. Moses 039bdcc0a8 [MLIR] Canonicalize sub/add of a constant and another sub/add of a constant
Differential Revision: https://reviews.llvm.org/D101705
2021-05-03 11:49:23 -04:00
Benjamin Kramer cdeb4a8a64 [mlir] Allow lowering cmpi/cmpf with multidimensional vectors to LLVM
Differential Revision: https://reviews.llvm.org/D101535
2021-05-03 11:30:21 +02:00
William S. Moses 78720296f3 [MLIR] Canonicalization of Integer Cast Operations
1) Canonicalize IndexCast(SExt(x)) => IndexCast(x)
2) Provide constant folds of sign_extend and truncate

Differential Revision: https://reviews.llvm.org/D101714
2021-05-02 11:22:18 -04:00
eopXD 0c1ff26bd3 [mlir] [affine] add canonicalization for affine.vector_load, vector_store
Added canonicalization for vector_load and vector_store. An existing
pattern SimplifyAffineOp can be reused to compose maps that supplies
result into them. Added AffineVectorStoreOp and AffineVectorLoadOp
into static_assert of SimplifyAffineOp to allow operation to use it.

This fixes the bug filed: https://bugs.llvm.org/show_bug.cgi?id=50058

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D101691
2021-05-02 09:06:46 +05:30
Chris Lattner a4c8952e6d Microoptimize dominance a bit - NFC.
Don't get RegionKindInterface if we won't use it. Noticed by inspection.
2021-05-01 12:40:13 -07:00
Chris Lattner be8ad4e98e [Verifier] Slightly refactor code to reduce duplication, NFC. 2021-05-01 11:43:15 -07:00
Aart Bik 0a29219931 [mlir][sparse] sparse tensor type encoding migration (new home, new builders)
(1) migrates the encoding from TensorDialect into the new SparseTensorDialect
(2) replaces dictionary-based storage and builders with struct-like data

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D101669
2021-04-30 19:30:38 -07:00
Duncan P. N. Exon Smith 518d955f9d Support: Stop using F_{None,Text,Append} compatibility synonyms, NFC
Stop using the compatibility spellings of `OF_{None,Text,Append}`
left behind by 1f67a3cba9. A follow-up
will remove them.

Differential Revision: https://reviews.llvm.org/D101650
2021-04-30 11:00:03 -07:00
Ahmed Taei 499e89fc91 Add patterns to lower vector.multi_reduction into a sequence of vector.reduction
Three patterns are added to convert into vector.multi_reduction into a
sequence of vector.reduction as the following:

- Transpose the inputs so inner most dimensions are always reduction.
- Reduce rank of vector.multi_reduction into 2d with inner most
reduction dim (get the 2d canical form)
- 2D canonical form is converted into a sequence of vector.reduction.

There are two things we might worth in a follow up diff:

- An scf.for (maybe optionally) around vector.reduction instead of unrolling it.
- Breakdown the vector.reduction into a sequence of vector.reduction
(e.g tree-based reduction) instead of relying on how downstream dialects
handle it.
  Note: this will requires passing target-vector-length

Differential Revision: https://reviews.llvm.org/D101570
2021-04-30 10:52:21 -07:00
Aart Bik 319072f4e3 [mlir][sparse] migrate sparse operations into new sparse tensor dialect
This is the very first step toward removing the glue and clutter from linalg and
replace it with proper sparse tensor types. This revision migrates the LinalgSparseOps
into SparseTensorOps of a sparse tensor dialect. This also provides a new home for
sparse tensor related transformation.

NOTE: the actual replacement with sparse tensor types (and removal of linalg glue/clutter)
will follow but I am trying to keep the amount of changes per revision manageable.

Differential Revision: https://reviews.llvm.org/D101573
2021-04-29 15:52:35 -07:00
Rob Suderman be01b091af [mlir][tosa] Remove constant-0 dim expr values from TOSA lowerings
Constant-0 dim expr values should be avoided for linalg as it can prevent
fusion. This includes adding support for rank-0 reshapes.

Differential Revision: https://reviews.llvm.org/D101418
2021-04-29 15:06:03 -07:00
Mehdi Amini 086e0f05bf Revert "[mlir][sparse] migrate sparse operations into new sparse tensor dialect"
This reverts commit a6d92a9711.

The build with -DBUILD_SHARED_LIBS=ON is broken.
2021-04-29 20:59:41 +00:00
Benjamin Kramer b389c80963 [mlir] Fix lowering of multi-dimensional vector log1p to LLVM
This was using the untransformed operand, leading to invalid IR.

Differential Revision: https://reviews.llvm.org/D101531
2021-04-29 21:53:52 +02:00
Aart Bik a6d92a9711 [mlir][sparse] migrate sparse operations into new sparse tensor dialect
This is the very first step toward removing the glue and clutter from linalg and
replace it with proper sparse tensor types. This revision migrates the LinalgSparseOps
into SparseTensorOps of a sparse tensor dialect. This also provides a new home for
sparse tensor related transformation.

NOTE: the actual replacement with sparse tensor types (and removal of linalg glue/clutter)
will follow but I am trying to keep the amount of changes per revision manageable.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D101488
2021-04-29 12:09:10 -07:00
Alex Zinenko 28ab7ff2d7 [mlir] fix shared-lib build 2021-04-29 13:27:41 +02:00
Alex Zinenko 6841e6afba [mlir] support max/min lower/upper bounds in affine.parallel
This enables to express more complex parallel loops in the affine framework,
for example, in cases of tiling by sizes not dividing loop trip counts perfectly
or inner wavefront parallelism, among others. One can't use affine.max/min
and supply values to the nested loop bounds since the results of such
affine.max/min operations aren't valid symbols. Making them valid symbols
isn't an option since they would introduce selection trees into memref
subscript arithmetic as an unintended and undesired consequence. Also
add support for converting such loops to SCF. Drop some API that isn't used in
the core repo from AffineParallelOp since its semantics becomes ambiguous in
presence of max/min bounds. Loop normalization is currently unavailable for
such loops.

Depends On D101171

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D101172
2021-04-29 13:16:25 +02:00
Alex Zinenko 545fa37834 [mlir] Affine: parallelize affine loops with reductions
Introduce a basic support for parallelizing affine loops with reductions
expressed using iteration arguments. Affine parallelism detector now has a flag
to assume such reductions are parallel. The transformation handles a subset of
parallel reductions that are can be expressed using affine.parallel:
integer/float addition and multiplication. This requires to detect the
reduction operation since affine.parallel only supports a fixed set of
reduction operators.

Reviewed By: chelini, kumasento, bondhugula

Differential Revision: https://reviews.llvm.org/D101171
2021-04-29 13:16:24 +02:00
Lorenzo Chelini de94b1855c [mlir] Fix top-level comments (NFC) 2021-04-29 13:06:40 +02:00
Tres Popp b863af5a5e [mlir] Add LinalgTransforms dependency on Complex 2021-04-29 12:20:44 +02:00
Tres Popp 42e5f42215 [mlir] Support complex numbers in Linalg promotion
FillOp allows complex ops, and filling a properly sized buffer with
a default zero complex number is implemented.

Differential Revision: https://reviews.llvm.org/D99939
2021-04-29 11:58:57 +02:00
Alex Zinenko ac0a70f373 [mlir] Split out Python bindings entry point into a separate file
This will allow the bindings to be built as a library and reused in out-of-tree
projects that want to provide bindings on top of MLIR bindings.

Reviewed By: stellaraccident, mikeurbach

Differential Revision: https://reviews.llvm.org/D101075
2021-04-29 11:18:25 +02:00
Frederik Gossen eb56fa97de [MLIR][Shape] Fix `shape.broadcast` to standard lowering
Differential Revision: https://reviews.llvm.org/D101456
2021-04-29 10:09:15 +02:00
Nicolas Vasilache b6113db955 [mlir][Linalg] Generalize linalg vectorization
This revision adds support for vectorizing more general linalg operations with projected permutation maps.

This is achieved by eagerly broadcasting the intermediate vector to the common size
of the iteration domain of the linalg op. This allows a much more natural expression of
generalized vectorization but may introduce additional computations until all the
proper canonicalizations are implemented.

This generalization modifies the vector.transfer_read/write permutation logic and
exposes the fact that the logic employed in vector.contract was too ad-hoc.

As a consequence, changes occur in the permutation / transposition logic for contraction. In turn this prompts supporting more cases in the lowering of contract
to matrix intrinsics, which is required to make the corresponding tests pass.

Differential revision: https://reviews.llvm.org/D101165
2021-04-29 07:44:01 +00:00
Tobias Gysi c2be2cda8d [mlir][Python][Linalg] Adding const, capture, and index support to the OpDSL.
The patch extends the OpDSL with support for:
- Constant values
- Capture scalar parameters
- Access the iteration indices using the index operation
- Provide predefined floating point and integer types.

Up to now the patch only supports emitting the new nodes. The C++/yaml path is not fully implemented. The fill_rng_2d operation defined in emit_structured_generic.py makes use of the new DSL constructs.

Differential Revision: https://reviews.llvm.org/D101364
2021-04-29 07:24:47 +00:00
Mike Urbach 49745f87e6 [mlir][python] Add `destroy` method to PyOperation.
This adds a method to directly invoke `mlirOperationDestroy` on the
MlirOperation wrapped by a PyOperation.

Reviewed By: stellaraccident, mehdi_amini

Differential Revision: https://reviews.llvm.org/D101422
2021-04-28 19:30:05 -06:00
John Demme 32e2fec726 [mlir] Move PyConcreteType to header. NFC.
This allows out-of-tree users to derive PyConcreteType to bind custom
types.

The Type version of https://reviews.llvm.org/D101063/new/

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D101496
2021-04-28 16:40:56 -07:00
Mike Urbach 6ff74f96fd [mlir][python] Update `PyOpResult.owner` to get the parent object.
Previously, this API would return the PyObjectRef, rather than the
underlying PyOperation.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D101416
2021-04-28 14:39:59 -06:00
MaheshRavishankar 41849a9195 [mlir][Linalg] Avoid changing the rank of the result in canonicalizations of subtensor.
Canonicalizations for subtensor operations defaulted to use the
rank-reduced version of the operation, but the cast inserted to get
back the original type would be illegal if the rank was actually
reduced. Instead make the canonicalization not reduce the rank of the
operation.

Differential Revision: https://reviews.llvm.org/D101258
2021-04-28 11:33:26 -07:00
Alexander Belyaev fa0d044c44 [mlir] Fix canonicalization of tiled_loop if not all opresults fold.
The current canonicalization did not remap operation results correctly
and attempted to erase tiledLoop, which is incorrect if not all tensor
results are folded.
2021-04-28 19:57:48 +02:00
Frederik Gossen 511ffe17ed Revert "[MLIR][Shape] Concretize broadcast result type if possible"
This reverts commit dca5361035.
2021-04-28 17:16:02 +02:00
Nicolas Vasilache b87219f77e [mlir][python] Add basic python support for GPU dialect and passes
Differential Revision: https://reviews.llvm.org/D101449
2021-04-28 14:52:28 +00:00
Nicolas Vasilache e7db8408d0 [mlir][python] Add python support for async dialect and passes.
since the `async` keyword is reserved in python, the dialect is called async_dialect.

Differential Revision: https://reviews.llvm.org/D101447
2021-04-28 14:52:27 +00:00
Alexander Belyaev 9a66d33452 [mlir] Fix the postsubmit comments in https://reviews.llvm.org/D101445 2021-04-28 14:58:02 +02:00
Alexander Belyaev 29dbac0ae2 [mlir] Add folding for tensor inputs and memref.cast in linalg.tiled_loop.
Tensor inputs, if not used in the body of TiledLoopOp, can be removed.
memref::CastOp can be folded into TiledLoopOp as well.

Differential Revision: https://reviews.llvm.org/D101445
2021-04-28 14:36:07 +02:00
Adrian Kuegel 2ea7fb7b1c [MLIR] Add ComplexToStandard conversion pass.
So far, only a conversion for complex::AbsOp is done, but more will be added.

Differential Revision: https://reviews.llvm.org/D101442
2021-04-28 14:17:46 +02:00
Frederik Gossen dca5361035 [MLIR][Shape] Concretize broadcast result type if possible
As a canonicalization, infer the resulting shape rank if possible.

Differential Revision: https://reviews.llvm.org/D101377
2021-04-28 11:58:32 +02:00
Frederik Gossen cb393f4c99 [MLIR][Shape] Canonicalize casted extent tensor operands
Both, `shape.broadcast` and `shape.cstr_broadcastable` accept dynamic and static
extent tensors. If their operands are casted, we can use the original value
instead.

Differential Revision: https://reviews.llvm.org/D101376
2021-04-28 11:51:58 +02:00
Frederik Gossen 3e037f8f0e [MLIR][Shape] Derive more concrete type for `shape.shape_of`
Also create all extent tensor constants with const_shape op.

Differential Revision: https://reviews.llvm.org/D99197
2021-04-28 10:50:53 +02:00
Tobias Gysi 3071107cf0 [mlir][Python][Linalg] Fixing typos (NFC). 2021-04-28 07:38:36 +00:00
Ranjith Kumar H b65472d66d [MLIR] Add and propagate section attribute for LLVM_GlobalOp
Add a section attribute to LLVM_GlobalOp, during module translation attribute value is propagated to llvm

Reviewed By: sgrechanik, ftynse, mehdi_amini

Differential Revision: https://reviews.llvm.org/D100947
2021-04-28 04:15:49 +00:00
Mike Urbach 63d16d06f5 [mlir] Support setting operand values in C and Python APIs.
This adds `mlirOperationSetOperand` to the IR C API, similar to the
function to get an operand.

In the Python API, this adds `operands[index] = value` syntax, similar
to the syntax to get an operand with `operands[index]`.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D101398
2021-04-27 20:17:47 -06:00
Mike Urbach 3f3d1c901d [MLIR][Python] Add capsule methods for pybind11 to PyValue.
Add the `getCapsule()` and `createFromCapsule()` methods to the
PyValue class, as well as the necessary interoperability.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D101090
2021-04-27 20:14:16 -06:00
Ahmed Taei 7fe2063446 Handle the case of tile and pad a subset of the dimensions
This is useful in cases such as tile-distribute-and-pad where not all
dims are tiled

Differential Revision: https://reviews.llvm.org/D101319
2021-04-27 17:41:22 -07:00
Rob Suderman cc1ae54ebc [tosa][mlir] Fix FullyConnected to correctly order dimensions
MatMul and FullyConnected have transposed dimensions for the weights.
Also, removed uneeded tensor reshape for bias.

Differential Revision: https://reviews.llvm.org/D101220
2021-04-27 17:26:04 -07:00
Rob Suderman 8f190b13ba [mlir][tosa] Add tosa.negate lowerings for quantized cases
Quantized negation can be performed using higher bits operations.
Minimal bits are picked to perform the operation.

Differential Revision: https://reviews.llvm.org/D101225
2021-04-27 17:16:39 -07:00
River Riddle 82bcd98586 [mlir] Fix bug in ForwardDataFlowAnalysis solver
Explicitly check for uninitialized to prevent crashes in edge cases where the derived analysis creates a lattice element for a value that hasn't been visited yet.
2021-04-27 14:31:27 -07:00
Frederik Gossen 6176a8f909 [MLIR] Debug log IR after pattern applications
Like `print-ir-after-all` and `-before-all`, this allows to inspect IR for
debug purposes. While the former allow to inspect only between passes, this
change allows to follow the rewrites that happen within passes.

Differential Revision: https://reviews.llvm.org/D100940
2021-04-27 15:04:52 +02:00
Frederik Gossen f8d7bd996f [MLIR][Shape] Remove empty extent tensor operands
Empty extent tensor operands were only removed when they were defined as a
constant. Additionally, we can remove them if they are known to be empty by
their type `tensor<0xindex>`.

Differential Revision: https://reviews.llvm.org/D101351
2021-04-27 14:51:43 +02:00
Frederik Gossen 2b9b999d4d [MLIR][Shape] Replace single operand broadcasts with appropriate cast
Differential Revision: https://reviews.llvm.org/D101350
2021-04-27 14:48:56 +02:00
Alexander Belyaev 4b13b7581d [mlir] Add a pass to tile Linalg ops using `linalg.tiled_loop`.
Differential Revision: https://reviews.llvm.org/D101084
2021-04-27 12:33:28 +02:00
Frederik Gossen b003ebd603 [MLIR][Linalg] Generalize splat constant folding
Splat constant folding was limited to `std.constant` operations. Instead, use
the constant matcher and apply splat constant folding to any constant-like
operation that holds a splat attribute.

Differential Revision: https://reviews.llvm.org/D101301
2021-04-27 09:08:34 +02:00
River Riddle d07c90e395 [mlir] Refactor the forward dataflow propagation in SCCP into a generic framework
This revision takes the forward value propagation engine in SCCP and refactors it into a more generalized forward dataflow analysis framework. This framework allows for propagating information about values across the various control flow constructs in MLIR, and removes the need for users to reinvent the traversal (often not as completely). There are a few aspects of the traversal, that were conservative for SCCP, that should be relaxed to support the needs of different value analyses. To keep this revision simple, these conservative behaviors will be left in (Note that this won't produce an incorrect result, but may produce more conservative results than necessary in certain edge cases. e.g. region entry arguments for non-region branch interface operations). The framework also only focuses on computing lattices for values, given the SCCP origins, but this is something to relax as needed in the future.

Given that this logic is already in SCCP, a majority of this commit is NFC. The more interesting parts are the interface glue that clients interact with.

Differential Revision: https://reviews.llvm.org/D100915
2021-04-26 19:39:46 -07:00
Aart Bik 23c9e8bc25 [mlir][tensors] Introduce attribute interface/attribute for tensor encoding
The new "encoding" field in tensor types so far had no meaning. This revision introduces:

1. an encoding attribute interface in IR: for verification between tensors and encodings in general
2. an attribute in Tensor dialect; #tensor.sparse<dict> + concrete sparse tensors API

Active discussion:
https://llvm.discourse.group/t/rfc-introduce-a-sparse-tensor-type-to-core-mlir/2944/

Reviewed By: silvas, penpornk, bixia

Differential Revision: https://reviews.llvm.org/D101008
2021-04-26 18:31:54 -07:00
William S. Moses ca27260701 [MLIR] Add SCF.if Condition Canonicalizations
Add two canoncalizations for scf.if.
  1) A canonicalization that allows users of a condition within an if to assume the condition
     is true if in the true region, etc.
  2) A canonicalization that removes yielded statements that are equivalent to the condition
     or its negation

Differential Revision: https://reviews.llvm.org/D101012
2021-04-26 20:13:08 -04:00
Fangrui Song 18839be9c5 [ADT] Remove StatisticBase and make NoopStatistic empty
In LLVM_ENABLE_STATS=0 builds, `llvm::Statistic` maps to `llvm::NoopStatistic`
but has 3 mostly unused pointers. GlobalOpt considers that the pointers can
potentially retain allocated objects, so GlobalOpt cannot optimize out the
`NoopStatistic` variables (see D69428 for more context), wasting 23KiB for stage
2 clang.

This patch makes `NoopStatistic` empty and thus reclaims the wasted space.  The
clang size is even smaller than applying D69428 (slightly smaller in both .bss and
.text).
```
# This means the D69428 optimization on clang is mostly nullified by this patch.
HEAD+D69428: size(.bss) = 0x0725a8
HEAD+D101211: size(.bss) = 0x072238

# bloaty - HEAD+D69428 vs HEAD+D101211
# With D101211, we also save a lot of string table space (.rodata).
    FILE SIZE        VM SIZE
 --------------  --------------
  -0.0%     -32  -0.0%     -24    .eh_frame
  -0.0%    -336  [ = ]       0    .symtab
  -0.0%    -360  [ = ]       0    .strtab
  [ = ]       0  -0.2%    -880    .bss
  -0.0% -2.11Ki  -0.0% -2.11Ki    .rodata
  -0.0% -2.89Ki  -0.0% -2.89Ki    .text
  -0.0% -5.71Ki  -0.0% -5.88Ki    TOTAL
```

Note: LoopFuse is a disabled pass. For now this patch adds
`#if LLVM_ENABLE_STATS` so `OptimizationRemarkMissed` is skipped in
LLVM_ENABLE_STATS==0 builds.  If these `OptimizationRemarkMissed` are useful in
LLVM_ENABLE_STATS==0 builds, we can replace `llvm::Statistic` with
`llvm::TrackingStatistic`, or use a different abstraction to keep track of the strings.

Similarly, skip the code in `mlir/lib/Pass/PassStatistics.cpp` which
calls `getName`/`getDesc`/`getValue`.

Reviewed By: lattner

Differential Revision: https://reviews.llvm.org/D101211
2021-04-26 16:47:32 -07:00
Chris Lattner f0c22c3d58 [Verifier] Tidy up the code a bit, NFC.
This tidies up the code a bit:
 * Eliminate the ctx member, which doesn't need to be stored.
 * Rename verify(Operation) to make it more clear that it is
   doing more than verifyOperation and that the dominance check
   isn't being done multiple times.
 * Rename mayNotHaveTerminator which was confusing about whether
   it wasn't known whether it had a terminator, when it is really
   about whether it is legal to have a terminator.
 * Some minor optimizations: don't check for RegionKindInterface
   if there are no regions.  Don't do two passes over the
   operations in a block in OperationVerifier::verifyDominance when
   one will do.

The optimizations are actually a measurable (but minor) win in some
CIRCT cases.

Differential Revision: https://reviews.llvm.org/D101267
2021-04-26 11:54:02 -07:00
Frederik Gossen 88b8b88035 [MLIR] Remove empty shape operands from `cstr_broadcastable` ops
Differential Revision: https://reviews.llvm.org/D101170
2021-04-26 18:34:18 +02:00
Frederik Gossen 858d4885dc [MLIR][Shape] Ensure to preserve op type of `shape.broadcast`
Ensure to preserve the correct type during when folding and canonicalization.
`shape.broadcast` of of a single operand can only be folded away if the argument
type is correct.

Differential Revision: https://reviews.llvm.org/D101158
2021-04-26 17:55:39 +02:00
Butygin f22d381385 [mlir] Canonicalize AllocOp's with only store and dealloc uses
Differential Revision: https://reviews.llvm.org/D100268
2021-04-24 09:51:00 +03:00
natashaknk 6f720d5eca [mlir][tosa] Add tosa.gather lowering to linalg.indexed_generic
Lowering gather operation to linalg dialect.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D101200
2021-04-23 22:42:56 -07:00
Rob Suderman 97c571abbc [mlir][tosa] Add tosa.resize lowering to linalg generic
Includes tests and implementation for both integer and floating point values.
Both nearest neighbor and bilinear interpolation is included.

Differential Revision: https://reviews.llvm.org/D101009
2021-04-23 12:43:02 -07:00
Alexander Belyaev 5291a7a3c7 [mlir] Add block arguments for input/output operands of 'linalg.tiled_loop`.
Differential Revision: https://reviews.llvm.org/D101186
2021-04-23 20:55:20 +02:00
Matthias Springer 64f7fb5dfc [mlir] Support masked N-D vector transfer ops in ProgressiveVectorToSCF.
Mask vectors are handled similar to data vectors in N-D TransferWriteOp. They are copied into a temporary memory buffer, which can be indexed into with non-constant values.

Differential Revision: https://reviews.llvm.org/D101136
2021-04-23 18:23:51 +09:00
Matthias Springer 545f98efc7 [mlir] Support masked 1D vector transfer ops in ProgressiveVectorToSCF
Support for masked N-D vector transfer ops will be added in a subsequent commit.

Differential Revision: https://reviews.llvm.org/D101132
2021-04-23 18:08:50 +09:00
Matthias Springer a819e73393 [mlir] Support broadcast dimensions in ProgressiveVectorToSCF
This commit adds support for broadcast dimensions in permutation maps of vector transfer ops.

Also fixes a bug in VectorToSCF that generated incorrect in-bounds checks for broadcast dimensions.

Differential Revision: https://reviews.llvm.org/D101019
2021-04-23 18:01:32 +09:00
Matthias Springer f6a3e92e0a [mlir] Use SCF for loops in ProgressiveVectorToSCF
Use SCF for loops instead of Affine for loops.

Differential Revision: https://reviews.llvm.org/D101013
2021-04-23 17:54:12 +09:00
Matthias Springer ab154176bf [mlir] Support dimension permutations in ProgressiveVectorToSCF
This commit adds support for dimension permutations in permutation maps of vector transfer ops.

Differential Revision: https://reviews.llvm.org/D101007
2021-04-23 17:46:35 +09:00
Matthias Springer afaf36b69e [mlir] Handle strided 1D vector transfer ops in ProgressiveVectorToSCF
Strided 1D vector transfer ops are 1D transfers operating on a memref dimension different from the last one. Such transfer ops do not accesses contiguous memory blocks (vectors), but access memory in a strided fashion. In the absence of a mask, strided 1D vector transfer ops can also be lowered using matrix.column.major.* LLVM instructions (in a later commit).

Subsequent commits will extend the pass to handle the remaining missing permutation maps (broadcasts, transposes, etc.).

Differential Revision: https://reviews.llvm.org/D100946
2021-04-23 17:19:22 +09:00
Alex Zinenko 0b10fdedf9 [mlir] Move PyConcreteAttribute to header. NFC.
This allows out-of-tree users to derive PyConcreteAttribute to bind custom
attributes.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D101063
2021-04-22 16:11:59 +02:00
Alexander Belyaev 0724911d2a [mlir] Add `tensor.reshape`.
This operation a counterpart of `memref.reshape`.

RFC [Reshape Ops Restructuring](https://llvm.discourse.group/t/rfc-reshape-ops-restructuring/3310)

Differential Revision: https://reviews.llvm.org/D100971
2021-04-22 14:53:23 +02:00
Frederik Gossen f0c51cb2d4 [MLIR][Shape] Add canonicalizations for `shape.broadcast`
Eliminate empty shapes from the operands, partially fold all constant shape
operands, and fix normal folding.

Differential Revision: https://reviews.llvm.org/D100634
2021-04-22 14:11:23 +02:00
Tobias Gysi 0e777e4ad7 [mlir][linalg] remove interchange option on linalg to loop lowering.
The interchange option attached to the linalg to loop lowering affects only the loops and does not update the memory accesses generated in to body of the operation. Instead of performing the interchange during the loop lowering use the interchange pattern.

Differential Revision: https://reviews.llvm.org/D100758
2021-04-22 08:55:17 +00:00
thomasraoux d40a19c3a8 [mlir][linalg] Add pattern to push reshape after elementwise operation
This help expose more fusion opportunities.

Differential Revision: https://reviews.llvm.org/D100685
2021-04-21 21:22:39 -07:00
Rob Suderman 648dfdfc24 [mlir][tosa] Add tosa.avg_pool2d lowering
Added the float lowerings for avg pool with corresponding tests.

Differential Revision: https://reviews.llvm.org/D100793
2021-04-21 19:07:52 -07:00
Eugene Zhulenev 3f1e827abd [mlir] Linalg : do not forward memrefs to outputs when do bufferization
Example:
```
%0 = linalg.init_tensor : tensor<...>
%1 = linalg.generic ... outs(%0: tensor<...>)
%2 = linalg.generic ... outs(%0: tensor<...>)
```

Memref allocated as a result of `init_tensor` bufferization can be incorrectly overwritten by the second linalg.generic operation

Reviewed By: silvas

Differential Revision: https://reviews.llvm.org/D100921
2021-04-21 16:39:06 -07:00
River Riddle 751c14fc42 [mlir][mlir-lsp] Add a new C++ LSP server for MLIR named mlir-lsp-server
This commits adds a basic LSP server for MLIR that supports resolving references and definitions. Several components of the setup are simplified to keep the size of this commit down, and will be built out in later commits. A followup commit will add a vscode language client that communicates with this server, paving the way for better IDE experience when interfacing with MLIR files.

The structure of this tool is similar to mlir-opt and mlir-translate, i.e. the implementation is structured as a library that users can call into to implement entry points that contain the dialects/passes that they are interested in.

Note: This commit contains several files, namely those in `mlir-lsp-server/lsp`, that have been copied from the LSP code in clangd and adapted for use in MLIR. This copying was decided as the best initial path forward (discussed offline by several stake holders in MLIR and clangd) given the different needs of our MLIR server, and the one for clangd. If a strong desire/need for unification arises in the future, the existence of these files in mlir-lsp-server can be reconsidered.

Differential Revision: https://reviews.llvm.org/D100439
2021-04-21 14:44:37 -07:00
River Riddle f8ac31314b Add a new AsmParserState class to capture detailed source information for .mlir files
This information isn't useful for general compilation, but is useful for building tools that process .mlir files. This class will be used in a followup to start building an LSP language server for MLIR.

Differential Revision: https://reviews.llvm.org/D100438
2021-04-21 14:44:37 -07:00
Ahmed Taei 10d7924581 Fix FoldReshapeOpWithUnitExtent generating illegal reshape
This will prevent fusion that spains all dims and generates
(d0, d1, ...) -> () reshape that isn't legal

Differential Revision: https://reviews.llvm.org/D100805
2021-04-21 11:30:45 -07:00
Nico Weber 297a5b7cbc [mlir] hopefully final round of iwyu fixes after ba7a92c01e 2021-04-21 11:03:06 -04:00
Nico Weber 56f987fafe [mlir] yet more iwyu fixes after ba7a92c01e 2021-04-21 10:54:44 -04:00
Nico Weber 41711c0c14 [mlir] more iwyu fixes after ba7a92c01e 2021-04-21 10:50:59 -04:00
Nico Weber fce0ef277d [mlir] some iwyu fixes after ba7a92c01e 2021-04-21 10:48:54 -04:00
thomasraoux ded18708f9 [mlir][NFC] Refactor linalg substituteMin and AffineMinSCF canonizalizations
Break up the dependency between SCF ops and substituteMin helper and make a
more generic version of AffineMinSCFCanonicalization. This reduce dependencies
between linalg and SCF and will allow the logic to be used with other kind of
ops. (Like ID ops).

Differential Revision: https://reviews.llvm.org/D100321
2021-04-21 07:19:36 -07:00
Butygin 85740ee108 [mlir] Assume terminators in nested regions are always legal in FuncBufferizePass
Previously, any terminator without ReturnLike and BranchOpInterface traits (e.g. scf.condition) were causing pass to fail.

Differential Revision: https://reviews.llvm.org/D100832
2021-04-21 11:55:11 +03:00
Tobias Gysi 5a451e486f [mlir][linalg] adapt named op generalization to work with captures.
Instead of always running the region builder check if the generalized op has a region attached. If yes inline the existing region instead of calling the region builder. This change circumvents a problem with named operations that have a region builder taking captures and the generalization pass not knowing about this captures.

Differential Revision: https://reviews.llvm.org/D100880
2021-04-21 06:37:53 +00:00
Amy Zhuang 9194071626 [mlir] Support hoisting whole affine for loops in LICM
Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D100512
2021-04-20 18:07:06 -07:00
Matthias Springer dd5324467d [mlir] Disallow broadcast dimensions on TransferWriteOp.
The current implementation allows for TransferWriteOps with broadcasts that do not make sense. E.g., a broadcast could write a vector into a single (scalar) memory location, which is effectively the same as writing only the last element of the vector.

Differential Revision: https://reviews.llvm.org/D100842
2021-04-21 07:43:45 +09:00
thomasraoux b2e72cd38d [mlir][spirv] Support conversion of extract op from vector<1xT> type
Differential Revision: https://reviews.llvm.org/D100814
2021-04-20 09:11:41 -07:00
Hanhan Wang 7b7df8e85e [mlir][StandardToSPIRV] Add support for lowering std.xor on bool to SPIR-V
std.xor ops on bool are lowered to spv.LogicalNotEqual. For Boolean values, xor
and not-equal are the same thing.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D100817
2021-04-20 07:35:20 -07:00
Tobias Gysi b9715156ff [mlir][linalg] lower index operations during linalg to vector lowering.
The patch extends the vectorization pass to lower linalg index operations to vector code. It allocates constant 1d vectors that enumerate the indexes along the iteration dimensions and broadcasts/transposes these 1d vectors to the iteration space.

Differential Revision: https://reviews.llvm.org/D100373
2021-04-20 11:55:44 +00:00
Matthias Springer 7cc8106f67 [mlir] Progressively lower vector to SCF
Add a new ProgressiveVectorToSCF pass that lowers vector transfer ops to SCF by gradually unpacking one dimension at time. Unpacking stops at 1D, but can be configured to stop earlier, should the HW support (N>1)-d vectors.

The current implementation cannot handle permutation maps, masks, tensor types and unrolling yet. These will be added in subsequent commits. Once features are on par with VectorToSCF, this implementation will replace VectorToSCF.

Differential Revision: https://reviews.llvm.org/D100622
2021-04-20 18:49:36 +09:00
Tres Popp 34810e1b9c [mlir] Add patterns to lower Math operations to LLVM based libm calls.
Some Math operations do not have an equivalent in LLVM. In these cases,
allow a low priority fallback of calling the libm functions. This is to
give functionality and is not a performant option.

Differential Revision: https://reviews.llvm.org/D100367
2021-04-20 11:38:55 +02:00
KareemErgawy-TomTom 0b05207e45 [MLIR][LinAlg] Detensoring CF cost-model: look forward.
This patch extends the control-flow cost-model for detensoring by
implementing a forward-looking pass on block arguments that should be
detensored. This makes sure that if a (to-be-detensored) block argument
"escapes" its block through the terminator, then the successor arguments
are also detensored.

Reviewed By: silvas

Differential Revision: https://reviews.llvm.org/D100457
2021-04-20 09:01:43 +02:00
Tobias Gysi 39a604e3df [mlir][linalg] update fusion on tensors to support linalg index operations.
The patch replaces the index operations in the body of fused producers and linearizes the indices after expansion.

Differential Revision: https://reviews.llvm.org/D100479
2021-04-20 06:13:04 +00:00
Tobias Gysi d0774f7f0a [mlir][linalg] update drop unit dims to support linalg index operations.
Update the dimensions of the index operations to account for dropped dimensions and replace the index operations of dropped dimensions by zero.

Differential Revision: https://reviews.llvm.org/D100395
2021-04-20 04:54:00 +00:00
clementval c46a88625d [mlir][llvm] Add UnnamedAddr attribute to GlobalOp
This patch add the UnnamedAddr attribute for the GlobalOp in the LLVM
dialect. The attribute is also handled to and from LLVM IR.

This is meant to be used in a follow up patch to lower OpenACC/OpenMP ops to
call to kmp and tgt runtime calls (D100678).

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D100677
2021-04-19 21:45:14 -04:00
Nicolas Vasilache 1dc533cea4 [mlir][python] ExecutionEngine can dump to object file
Differential Revision: https://reviews.llvm.org/D100786
2021-04-19 19:33:27 +00:00
Tobias Gysi 27ad213680 [mlir][linalg] enable library call rewrites for linalg operations with index semantics.
The patch enables the library call lowering for linalg operations that contain index operations.

Differential Revision: https://reviews.llvm.org/D100537
2021-04-19 12:50:59 +00:00
Alex Zinenko 4acd8457d8 [mlir] Improve debug flag management in Python bindings
Expose the debug flag as a readable and assignable property of a
dedicated class instead of a write-only function. Actually test the fact
of setting the flag. Move test to a dedicated file, it has zero relation
to context_managers.py where it was added.

Arguably, it should be promoted from mlir.ir to mlir module, but we are
not re-exporting the latter and this functionality is purposefully
hidden so can stay in IR for now. Drop unnecessary export code.

Refactor C API and put Debug into a separate library, fix it to actually
set the flag to the given value.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D100757
2021-04-19 14:45:43 +02:00
Tobias Gysi 495e1d7e8a [mlir][linalg] adding pass to run the interchange pattern.
Instead of interchanging loops during the loop lowering this pass performs the interchange by permuting the indexing maps. It also updates the iterator types and the index accesses in the body of the operation.

Differential Revision: https://reviews.llvm.org/D100627
2021-04-19 12:19:15 +00:00
Tres Popp 31686d13dc Add default DataLayout support for complex numbers
Differential Revision: https://reviews.llvm.org/D100289
2021-04-19 11:36:12 +02:00
Nicolas Vasilache 843f1fc825 [mlir][scf] Add scf.for + tensor.cast canonicalization pattern
Fold scf.for iter_arg/result pairs that go through incoming/ougoing
a tensor.cast op pair so as to pull the tensor.cast inside the scf.for:

```
  %0 = tensor.cast %t0 : tensor<32x1024xf32> to tensor<?x?xf32>
  %1 = scf.for %i = %c0 to %c1024 step %c32 iter_args(%iter_t0 = %0)
     -> (tensor<?x?xf32>) {
    %2 = call @do(%iter_t0) : (tensor<?x?xf32>) -> tensor<?x?xf32>
    scf.yield %2 : tensor<?x?xf32>
  }
  %2 = tensor.cast %1 : tensor<?x?xf32> to tensor<32x1024xf32>
  use_of(%2)
```

folds into:

```
  %0 = scf.for %arg2 = %c0 to %c1024 step %c32 iter_args(%arg3 = %arg0)
      -> (tensor<32x1024xf32>) {
    %2 = tensor.cast %arg3 : tensor<32x1024xf32> to tensor<?x?xf32>
    %3 = call @do(%2) : (tensor<?x?xf32>) -> tensor<?x?xf32>
    %4 = tensor.cast %3 : tensor<?x?xf32> to tensor<32x1024xf32>
    scf.yield %4 : tensor<32x1024xf32>
  }
  use_of(%0)
```

Differential Revision: https://reviews.llvm.org/D100661
2021-04-16 16:50:21 +00:00
thomasraoux 3fc0fbefc8 [mlir][vector] Move transferOp on tensor opt to folder/canonicalization
Move the existing optimization for transfer op on tensor to folder and
canonicalization. This handles the write after write case and read after write
and also add write after read case.

Differential Revision: https://reviews.llvm.org/D100597
2021-04-16 08:13:10 -07:00
Mats Petersson 517c3aee4d [OpenMP IRBuilder, MLIR] Add support for OpenMP do schedule dynamic
The implementation supports static schedule for Fortran do loops. This
implements the dynamic variant of the same concept.

Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D97393
2021-04-16 16:09:49 +01:00
Javier Setoain b739bada9d [mlir][ArmSVE] Cleanup dialect registration
ArmSVE dialect is behind the recent changes in how the Vector dialect
interacts with backend vector dialects and the MLIR -> LLVM IR
translation module. This patch cleans up ArmSVE initialization within
Vector and removes the need for an LLVMArmSVE dialect.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D100171
2021-04-16 15:56:51 +02:00
Nicolas Vasilache caa159f044 [mlir][python] Add simple debugging and printing helpers
Differential Revision: https://reviews.llvm.org/D100643
2021-04-16 13:47:46 +00:00
Frederik Gossen c2c65585c5 [MLIR] Fix `isValidIndex`
Differential Revision: https://reviews.llvm.org/D100635
2021-04-16 14:58:54 +02:00
Frederik Gossen 3a5a610e27 [MLIR][Shape] Expose `getShapeVec` and add support for extent tensors
Differential Revision: https://reviews.llvm.org/D100636
2021-04-16 13:59:20 +02:00
Nicolas Vasilache b5f3a128bf [mlir][Python][Linalg] Add support for captures in body builder.
When Linalg named ops support was added, captures were omitted
from the body builder. This revision adds support for captures
which allows us to write FillOp in a more idiomatic fashion using
the _linalg_ops_ext mixin support.

This raises an issue in the generation of `_linalg_ops_gen.py` where
```
  @property
  def result(self):
    return self.operation.results[0] if len(self.operation.results) > 1 else None
```.
The condition should be `== 1`.

This will be fixed in a separate commit.

Differential Revision: https://reviews.llvm.org/D100363
2021-04-16 08:47:26 +00:00
Nicolas Vasilache 8cf650c554 [mlir][linalg] Add support for WAW fusion on tensors.
Differential Revision: https://reviews.llvm.org/D100603
2021-04-16 08:22:09 +00:00
Ahmed Taei 0e2f9b61fd Fix tile-and-pad when padding doesn't span all dimension
Without this tile-and-pad will never terminate if pad-fails.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D97720
2021-04-15 20:17:40 -07:00
Prashant Kumar 102fd1cb8b Add support for numpy arrays to memref conversions.
This offers the ability to pass numpy arrays to the corresponding
memref argument.

Reviewed By: mehdi_amini, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D100077
2021-04-15 23:41:26 +00:00
River Riddle 706c9c5ce0 [mlir] Add support for walking locations similarly to Operations
This allows for walking all nested locations of a given location, and is generally useful when processing locations.

Differential Revision: https://reviews.llvm.org/D100437
2021-04-15 16:09:34 -07:00
River Riddle 4efb7754e0 [mlir][NFC] Add a using directive for llvm::SetVector
Differential Revision: https://reviews.llvm.org/D100436
2021-04-15 16:09:34 -07:00
Aart Bik 916f3e16bd [mlir][vector][avx] add AVX dot product to X86Vector dialect with lowering
In the long run, we want to unify the dot product codegen solutions between
all target architectures, but this intrinsic enables experimenting with AVX
specific implementations in the meantime.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D100593
2021-04-15 15:01:39 -07:00
River Riddle b1260109fb [mlir][AsmPrinter] Fix multi-threaded segfault by using explicit null stream per thread
We were using llvm::nulls, but that isn't thread safe so we switch to giving each thread it's own null stream.

Differential Revision: https://reviews.llvm.org/D100578
2021-04-15 12:26:59 -07:00
Alexander Belyaev cf761904a2 [mlir] Add verification for `linalg.tiled_loop` op.
Differential Revision: https://reviews.llvm.org/D100555
2021-04-15 20:50:36 +02:00
Alexander Belyaev 67f60bcc75 [mlir] Expose `updateBoundsForCyclicDistribution` in Linalg/Utils.h.
Differential Revision: https://reviews.llvm.org/D100580
2021-04-15 20:47:37 +02:00
River Riddle 94662ee0c1 [mlir] Add support for adding attribute+type traits/interfaces to tablegen defs
This matches the current support provided to operations, and allows attaching traits, interfaces, and using the DeclareInterfaceMethods utility. This was missed when attribute/type generation was first added.

Differential Revision: https://reviews.llvm.org/D100233
2021-04-15 11:41:51 -07:00
Aart Bik 92b0a9d7d4 [mlir][sparse] remove restriction on vectorization of index type
Rationale:
Now that vector<?xindex> is allowed, the restriction on vectorization
of index types in the sparse compiler can be removed. Also needs
generalization of scatter/gather index types.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D100522
2021-04-15 10:27:04 -07:00
Chia-hung Duan 6b0cef3e02 Refactor the architecture of mlir-reduce
Add iterator for ReductionNode traversal and use range to indicate the
region we would like to keep. Refactor the interaction between
Pass/Tester/ReductionNode.
Now it'll be easier to add new traversal type and OpReducer

Reviewed By: jpienaar, rriddle

Differential Revision: https://reviews.llvm.org/D99713
2021-04-14 13:40:44 -07:00
Mehdi Amini a33b647100 Revert "Refactor the architecture of mlir-reduce"
This reverts commit a32846b1d0.

The build is broken with -DBUILD_SHARED_LIBS=ON:

tools/mlir/lib/Reducer/CMakeFiles/obj.MLIRReduce.dir/Tester.cpp.o: In function `mlir::Tester::isInteresting(mlir::ModuleOp) const':
Tester.cpp:(.text._ZNK4mlir6Tester13isInterestingENS_8ModuleOpE+0xa8): undefined reference to `mlir::OpPrintingFlags::OpPrintingFlags()'
Tester.cpp:(.text._ZNK4mlir6Tester13isInterestingENS_8ModuleOpE+0xc6): undefined reference to `mlir::Operation::print(llvm::raw_ostream&, mlir::OpPrintingFlags)'
2021-04-14 19:25:43 +00:00
Jacques Pienaar a32846b1d0 Refactor the architecture of mlir-reduce
Add iterator for ReductionNode traversal and use range to indicate the region we would like to keep. Refactor the interaction between Pass/Tester/ReductionNode.
Now it'll be easier to add new traversal type and OpReducer

Reviewed By: jpienaar, rriddle

Differential Revision: https://reviews.llvm.org/D99713
2021-04-14 10:16:30 -07:00
Tobias Gysi ce82843f72 [mlir][linalg] update fusion to support linalg index operations.
The patch updates the linalg fusion pass to add the tile offsets to the indices.

Differential Revision: https://reviews.llvm.org/D100456
2021-04-14 15:32:42 +00:00
Hanhan Wang 7c4de2e9b9 [mlir][StandardToSPIRV] Add support for lowering memref<?xi1> to SPIR-V
Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D100452
2021-04-14 07:22:49 -07:00
Tres Popp d80178f7c1 [mlir] Change verification order to prevent null dereference
Differential Revision: https://reviews.llvm.org/D100390
2021-04-14 09:33:17 +02:00