Commit Graph

5426 Commits

Author SHA1 Message Date
Rob Suderman be01b091af [mlir][tosa] Remove constant-0 dim expr values from TOSA lowerings
Constant-0 dim expr values should be avoided for linalg as it can prevent
fusion. This includes adding support for rank-0 reshapes.

Differential Revision: https://reviews.llvm.org/D101418
2021-04-29 15:06:03 -07:00
Mehdi Amini 086e0f05bf Revert "[mlir][sparse] migrate sparse operations into new sparse tensor dialect"
This reverts commit a6d92a9711.

The build with -DBUILD_SHARED_LIBS=ON is broken.
2021-04-29 20:59:41 +00:00
Benjamin Kramer b389c80963 [mlir] Fix lowering of multi-dimensional vector log1p to LLVM
This was using the untransformed operand, leading to invalid IR.

Differential Revision: https://reviews.llvm.org/D101531
2021-04-29 21:53:52 +02:00
Aart Bik a6d92a9711 [mlir][sparse] migrate sparse operations into new sparse tensor dialect
This is the very first step toward removing the glue and clutter from linalg and
replace it with proper sparse tensor types. This revision migrates the LinalgSparseOps
into SparseTensorOps of a sparse tensor dialect. This also provides a new home for
sparse tensor related transformation.

NOTE: the actual replacement with sparse tensor types (and removal of linalg glue/clutter)
will follow but I am trying to keep the amount of changes per revision manageable.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D101488
2021-04-29 12:09:10 -07:00
Alex Zinenko 28ab7ff2d7 [mlir] fix shared-lib build 2021-04-29 13:27:41 +02:00
Alex Zinenko 6841e6afba [mlir] support max/min lower/upper bounds in affine.parallel
This enables to express more complex parallel loops in the affine framework,
for example, in cases of tiling by sizes not dividing loop trip counts perfectly
or inner wavefront parallelism, among others. One can't use affine.max/min
and supply values to the nested loop bounds since the results of such
affine.max/min operations aren't valid symbols. Making them valid symbols
isn't an option since they would introduce selection trees into memref
subscript arithmetic as an unintended and undesired consequence. Also
add support for converting such loops to SCF. Drop some API that isn't used in
the core repo from AffineParallelOp since its semantics becomes ambiguous in
presence of max/min bounds. Loop normalization is currently unavailable for
such loops.

Depends On D101171

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D101172
2021-04-29 13:16:25 +02:00
Alex Zinenko 545fa37834 [mlir] Affine: parallelize affine loops with reductions
Introduce a basic support for parallelizing affine loops with reductions
expressed using iteration arguments. Affine parallelism detector now has a flag
to assume such reductions are parallel. The transformation handles a subset of
parallel reductions that are can be expressed using affine.parallel:
integer/float addition and multiplication. This requires to detect the
reduction operation since affine.parallel only supports a fixed set of
reduction operators.

Reviewed By: chelini, kumasento, bondhugula

Differential Revision: https://reviews.llvm.org/D101171
2021-04-29 13:16:24 +02:00
Lorenzo Chelini de94b1855c [mlir] Fix top-level comments (NFC) 2021-04-29 13:06:40 +02:00
Tres Popp b863af5a5e [mlir] Add LinalgTransforms dependency on Complex 2021-04-29 12:20:44 +02:00
Tres Popp 42e5f42215 [mlir] Support complex numbers in Linalg promotion
FillOp allows complex ops, and filling a properly sized buffer with
a default zero complex number is implemented.

Differential Revision: https://reviews.llvm.org/D99939
2021-04-29 11:58:57 +02:00
Alex Zinenko ac0a70f373 [mlir] Split out Python bindings entry point into a separate file
This will allow the bindings to be built as a library and reused in out-of-tree
projects that want to provide bindings on top of MLIR bindings.

Reviewed By: stellaraccident, mikeurbach

Differential Revision: https://reviews.llvm.org/D101075
2021-04-29 11:18:25 +02:00
Frederik Gossen eb56fa97de [MLIR][Shape] Fix `shape.broadcast` to standard lowering
Differential Revision: https://reviews.llvm.org/D101456
2021-04-29 10:09:15 +02:00
Nicolas Vasilache b6113db955 [mlir][Linalg] Generalize linalg vectorization
This revision adds support for vectorizing more general linalg operations with projected permutation maps.

This is achieved by eagerly broadcasting the intermediate vector to the common size
of the iteration domain of the linalg op. This allows a much more natural expression of
generalized vectorization but may introduce additional computations until all the
proper canonicalizations are implemented.

This generalization modifies the vector.transfer_read/write permutation logic and
exposes the fact that the logic employed in vector.contract was too ad-hoc.

As a consequence, changes occur in the permutation / transposition logic for contraction. In turn this prompts supporting more cases in the lowering of contract
to matrix intrinsics, which is required to make the corresponding tests pass.

Differential revision: https://reviews.llvm.org/D101165
2021-04-29 07:44:01 +00:00
Tobias Gysi c2be2cda8d [mlir][Python][Linalg] Adding const, capture, and index support to the OpDSL.
The patch extends the OpDSL with support for:
- Constant values
- Capture scalar parameters
- Access the iteration indices using the index operation
- Provide predefined floating point and integer types.

Up to now the patch only supports emitting the new nodes. The C++/yaml path is not fully implemented. The fill_rng_2d operation defined in emit_structured_generic.py makes use of the new DSL constructs.

Differential Revision: https://reviews.llvm.org/D101364
2021-04-29 07:24:47 +00:00
Mike Urbach 49745f87e6 [mlir][python] Add `destroy` method to PyOperation.
This adds a method to directly invoke `mlirOperationDestroy` on the
MlirOperation wrapped by a PyOperation.

Reviewed By: stellaraccident, mehdi_amini

Differential Revision: https://reviews.llvm.org/D101422
2021-04-28 19:30:05 -06:00
John Demme 32e2fec726 [mlir] Move PyConcreteType to header. NFC.
This allows out-of-tree users to derive PyConcreteType to bind custom
types.

The Type version of https://reviews.llvm.org/D101063/new/

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D101496
2021-04-28 16:40:56 -07:00
Mike Urbach 6ff74f96fd [mlir][python] Update `PyOpResult.owner` to get the parent object.
Previously, this API would return the PyObjectRef, rather than the
underlying PyOperation.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D101416
2021-04-28 14:39:59 -06:00
MaheshRavishankar 41849a9195 [mlir][Linalg] Avoid changing the rank of the result in canonicalizations of subtensor.
Canonicalizations for subtensor operations defaulted to use the
rank-reduced version of the operation, but the cast inserted to get
back the original type would be illegal if the rank was actually
reduced. Instead make the canonicalization not reduce the rank of the
operation.

Differential Revision: https://reviews.llvm.org/D101258
2021-04-28 11:33:26 -07:00
Alexander Belyaev fa0d044c44 [mlir] Fix canonicalization of tiled_loop if not all opresults fold.
The current canonicalization did not remap operation results correctly
and attempted to erase tiledLoop, which is incorrect if not all tensor
results are folded.
2021-04-28 19:57:48 +02:00
Frederik Gossen 511ffe17ed Revert "[MLIR][Shape] Concretize broadcast result type if possible"
This reverts commit dca5361035.
2021-04-28 17:16:02 +02:00
Nicolas Vasilache b87219f77e [mlir][python] Add basic python support for GPU dialect and passes
Differential Revision: https://reviews.llvm.org/D101449
2021-04-28 14:52:28 +00:00
Nicolas Vasilache e7db8408d0 [mlir][python] Add python support for async dialect and passes.
since the `async` keyword is reserved in python, the dialect is called async_dialect.

Differential Revision: https://reviews.llvm.org/D101447
2021-04-28 14:52:27 +00:00
Alexander Belyaev 9a66d33452 [mlir] Fix the postsubmit comments in https://reviews.llvm.org/D101445 2021-04-28 14:58:02 +02:00
Alexander Belyaev 29dbac0ae2 [mlir] Add folding for tensor inputs and memref.cast in linalg.tiled_loop.
Tensor inputs, if not used in the body of TiledLoopOp, can be removed.
memref::CastOp can be folded into TiledLoopOp as well.

Differential Revision: https://reviews.llvm.org/D101445
2021-04-28 14:36:07 +02:00
Adrian Kuegel 2ea7fb7b1c [MLIR] Add ComplexToStandard conversion pass.
So far, only a conversion for complex::AbsOp is done, but more will be added.

Differential Revision: https://reviews.llvm.org/D101442
2021-04-28 14:17:46 +02:00
Frederik Gossen dca5361035 [MLIR][Shape] Concretize broadcast result type if possible
As a canonicalization, infer the resulting shape rank if possible.

Differential Revision: https://reviews.llvm.org/D101377
2021-04-28 11:58:32 +02:00
Frederik Gossen cb393f4c99 [MLIR][Shape] Canonicalize casted extent tensor operands
Both, `shape.broadcast` and `shape.cstr_broadcastable` accept dynamic and static
extent tensors. If their operands are casted, we can use the original value
instead.

Differential Revision: https://reviews.llvm.org/D101376
2021-04-28 11:51:58 +02:00
Frederik Gossen 3e037f8f0e [MLIR][Shape] Derive more concrete type for `shape.shape_of`
Also create all extent tensor constants with const_shape op.

Differential Revision: https://reviews.llvm.org/D99197
2021-04-28 10:50:53 +02:00
Tobias Gysi 3071107cf0 [mlir][Python][Linalg] Fixing typos (NFC). 2021-04-28 07:38:36 +00:00
Ranjith Kumar H b65472d66d [MLIR] Add and propagate section attribute for LLVM_GlobalOp
Add a section attribute to LLVM_GlobalOp, during module translation attribute value is propagated to llvm

Reviewed By: sgrechanik, ftynse, mehdi_amini

Differential Revision: https://reviews.llvm.org/D100947
2021-04-28 04:15:49 +00:00
Mike Urbach 63d16d06f5 [mlir] Support setting operand values in C and Python APIs.
This adds `mlirOperationSetOperand` to the IR C API, similar to the
function to get an operand.

In the Python API, this adds `operands[index] = value` syntax, similar
to the syntax to get an operand with `operands[index]`.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D101398
2021-04-27 20:17:47 -06:00
Mike Urbach 3f3d1c901d [MLIR][Python] Add capsule methods for pybind11 to PyValue.
Add the `getCapsule()` and `createFromCapsule()` methods to the
PyValue class, as well as the necessary interoperability.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D101090
2021-04-27 20:14:16 -06:00
Ahmed Taei 7fe2063446 Handle the case of tile and pad a subset of the dimensions
This is useful in cases such as tile-distribute-and-pad where not all
dims are tiled

Differential Revision: https://reviews.llvm.org/D101319
2021-04-27 17:41:22 -07:00
Rob Suderman cc1ae54ebc [tosa][mlir] Fix FullyConnected to correctly order dimensions
MatMul and FullyConnected have transposed dimensions for the weights.
Also, removed uneeded tensor reshape for bias.

Differential Revision: https://reviews.llvm.org/D101220
2021-04-27 17:26:04 -07:00
Rob Suderman 8f190b13ba [mlir][tosa] Add tosa.negate lowerings for quantized cases
Quantized negation can be performed using higher bits operations.
Minimal bits are picked to perform the operation.

Differential Revision: https://reviews.llvm.org/D101225
2021-04-27 17:16:39 -07:00
River Riddle 82bcd98586 [mlir] Fix bug in ForwardDataFlowAnalysis solver
Explicitly check for uninitialized to prevent crashes in edge cases where the derived analysis creates a lattice element for a value that hasn't been visited yet.
2021-04-27 14:31:27 -07:00
Frederik Gossen 6176a8f909 [MLIR] Debug log IR after pattern applications
Like `print-ir-after-all` and `-before-all`, this allows to inspect IR for
debug purposes. While the former allow to inspect only between passes, this
change allows to follow the rewrites that happen within passes.

Differential Revision: https://reviews.llvm.org/D100940
2021-04-27 15:04:52 +02:00
Frederik Gossen f8d7bd996f [MLIR][Shape] Remove empty extent tensor operands
Empty extent tensor operands were only removed when they were defined as a
constant. Additionally, we can remove them if they are known to be empty by
their type `tensor<0xindex>`.

Differential Revision: https://reviews.llvm.org/D101351
2021-04-27 14:51:43 +02:00
Frederik Gossen 2b9b999d4d [MLIR][Shape] Replace single operand broadcasts with appropriate cast
Differential Revision: https://reviews.llvm.org/D101350
2021-04-27 14:48:56 +02:00
Alexander Belyaev 4b13b7581d [mlir] Add a pass to tile Linalg ops using `linalg.tiled_loop`.
Differential Revision: https://reviews.llvm.org/D101084
2021-04-27 12:33:28 +02:00
Frederik Gossen b003ebd603 [MLIR][Linalg] Generalize splat constant folding
Splat constant folding was limited to `std.constant` operations. Instead, use
the constant matcher and apply splat constant folding to any constant-like
operation that holds a splat attribute.

Differential Revision: https://reviews.llvm.org/D101301
2021-04-27 09:08:34 +02:00
River Riddle d07c90e395 [mlir] Refactor the forward dataflow propagation in SCCP into a generic framework
This revision takes the forward value propagation engine in SCCP and refactors it into a more generalized forward dataflow analysis framework. This framework allows for propagating information about values across the various control flow constructs in MLIR, and removes the need for users to reinvent the traversal (often not as completely). There are a few aspects of the traversal, that were conservative for SCCP, that should be relaxed to support the needs of different value analyses. To keep this revision simple, these conservative behaviors will be left in (Note that this won't produce an incorrect result, but may produce more conservative results than necessary in certain edge cases. e.g. region entry arguments for non-region branch interface operations). The framework also only focuses on computing lattices for values, given the SCCP origins, but this is something to relax as needed in the future.

Given that this logic is already in SCCP, a majority of this commit is NFC. The more interesting parts are the interface glue that clients interact with.

Differential Revision: https://reviews.llvm.org/D100915
2021-04-26 19:39:46 -07:00
Aart Bik 23c9e8bc25 [mlir][tensors] Introduce attribute interface/attribute for tensor encoding
The new "encoding" field in tensor types so far had no meaning. This revision introduces:

1. an encoding attribute interface in IR: for verification between tensors and encodings in general
2. an attribute in Tensor dialect; #tensor.sparse<dict> + concrete sparse tensors API

Active discussion:
https://llvm.discourse.group/t/rfc-introduce-a-sparse-tensor-type-to-core-mlir/2944/

Reviewed By: silvas, penpornk, bixia

Differential Revision: https://reviews.llvm.org/D101008
2021-04-26 18:31:54 -07:00
William S. Moses ca27260701 [MLIR] Add SCF.if Condition Canonicalizations
Add two canoncalizations for scf.if.
  1) A canonicalization that allows users of a condition within an if to assume the condition
     is true if in the true region, etc.
  2) A canonicalization that removes yielded statements that are equivalent to the condition
     or its negation

Differential Revision: https://reviews.llvm.org/D101012
2021-04-26 20:13:08 -04:00
Fangrui Song 18839be9c5 [ADT] Remove StatisticBase and make NoopStatistic empty
In LLVM_ENABLE_STATS=0 builds, `llvm::Statistic` maps to `llvm::NoopStatistic`
but has 3 mostly unused pointers. GlobalOpt considers that the pointers can
potentially retain allocated objects, so GlobalOpt cannot optimize out the
`NoopStatistic` variables (see D69428 for more context), wasting 23KiB for stage
2 clang.

This patch makes `NoopStatistic` empty and thus reclaims the wasted space.  The
clang size is even smaller than applying D69428 (slightly smaller in both .bss and
.text).
```
# This means the D69428 optimization on clang is mostly nullified by this patch.
HEAD+D69428: size(.bss) = 0x0725a8
HEAD+D101211: size(.bss) = 0x072238

# bloaty - HEAD+D69428 vs HEAD+D101211
# With D101211, we also save a lot of string table space (.rodata).
    FILE SIZE        VM SIZE
 --------------  --------------
  -0.0%     -32  -0.0%     -24    .eh_frame
  -0.0%    -336  [ = ]       0    .symtab
  -0.0%    -360  [ = ]       0    .strtab
  [ = ]       0  -0.2%    -880    .bss
  -0.0% -2.11Ki  -0.0% -2.11Ki    .rodata
  -0.0% -2.89Ki  -0.0% -2.89Ki    .text
  -0.0% -5.71Ki  -0.0% -5.88Ki    TOTAL
```

Note: LoopFuse is a disabled pass. For now this patch adds
`#if LLVM_ENABLE_STATS` so `OptimizationRemarkMissed` is skipped in
LLVM_ENABLE_STATS==0 builds.  If these `OptimizationRemarkMissed` are useful in
LLVM_ENABLE_STATS==0 builds, we can replace `llvm::Statistic` with
`llvm::TrackingStatistic`, or use a different abstraction to keep track of the strings.

Similarly, skip the code in `mlir/lib/Pass/PassStatistics.cpp` which
calls `getName`/`getDesc`/`getValue`.

Reviewed By: lattner

Differential Revision: https://reviews.llvm.org/D101211
2021-04-26 16:47:32 -07:00
Chris Lattner f0c22c3d58 [Verifier] Tidy up the code a bit, NFC.
This tidies up the code a bit:
 * Eliminate the ctx member, which doesn't need to be stored.
 * Rename verify(Operation) to make it more clear that it is
   doing more than verifyOperation and that the dominance check
   isn't being done multiple times.
 * Rename mayNotHaveTerminator which was confusing about whether
   it wasn't known whether it had a terminator, when it is really
   about whether it is legal to have a terminator.
 * Some minor optimizations: don't check for RegionKindInterface
   if there are no regions.  Don't do two passes over the
   operations in a block in OperationVerifier::verifyDominance when
   one will do.

The optimizations are actually a measurable (but minor) win in some
CIRCT cases.

Differential Revision: https://reviews.llvm.org/D101267
2021-04-26 11:54:02 -07:00
Frederik Gossen 88b8b88035 [MLIR] Remove empty shape operands from `cstr_broadcastable` ops
Differential Revision: https://reviews.llvm.org/D101170
2021-04-26 18:34:18 +02:00
Frederik Gossen 858d4885dc [MLIR][Shape] Ensure to preserve op type of `shape.broadcast`
Ensure to preserve the correct type during when folding and canonicalization.
`shape.broadcast` of of a single operand can only be folded away if the argument
type is correct.

Differential Revision: https://reviews.llvm.org/D101158
2021-04-26 17:55:39 +02:00
Butygin f22d381385 [mlir] Canonicalize AllocOp's with only store and dealloc uses
Differential Revision: https://reviews.llvm.org/D100268
2021-04-24 09:51:00 +03:00
natashaknk 6f720d5eca [mlir][tosa] Add tosa.gather lowering to linalg.indexed_generic
Lowering gather operation to linalg dialect.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D101200
2021-04-23 22:42:56 -07:00