Commit Graph

5577 Commits

Author SHA1 Message Date
Kazu Hirata 51fd157635 Remove duplicate forward declarations (NFC) 2022-01-08 11:56:42 -08:00
Groverkss 0e19186c82 [MLIR][NFC] Move PresburgerSet to Presburger/ directory
This patch moves PresburgerSet to Presburger/ directory. This patch is purely
mechincal, it only moves and renames functionality and tests.

This patch is part of a series of patches to move presburger functionality to
Presburger/ directory.

Reviewed By: arjunp

Differential Revision: https://reviews.llvm.org/D116836
2022-01-08 15:39:40 +05:30
Groverkss 74903059b2 [MLIR] Add IntegerPolyhedron::getUniverse
This operation already exists in FlatAffineConstraints but is added to
IntegerPolyhedron to keep consistancy in available methods.
2022-01-08 02:59:56 +05:30
Groverkss 49b754b5c6 [MLIR][NFC] Move presburger functionality from FlatAffineConstraints to IntegerPolyhedron
This patch moves all presburger functionality from FlatAffineConstraints to
IntegerPolyhedron. This patch is purely mechanical, it only moves and renames
functionality and tests.

This patch is part of a series of patches to move presburger functionality to
Presburger/ directory.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D116681
2022-01-08 01:22:49 +05:30
Kazu Hirata fb7cf90071 Use nullptr instead of 0 or NULL (NFC)
Identified with modernize-use-nullptr.
2022-01-07 10:17:29 -08:00
Matthias Springer 089b910abc [mlir][linalg][bufferize][NFC] Add `analyzeOp` helper function
This function runs just the analysis of Comprehensive Bufferize, but does not bufferize the IR yet.

This is in preparation of fixing CallOp bufferization. Also needed for unifying Comprehensive Bufferize and core bufferization; the new partial bufferization can simply run bufferization without an analysis.

Differential Revision: https://reviews.llvm.org/D116456
2022-01-08 01:33:41 +09:00
Matthias Springer d9184ab1a5 [mlir][linalg][bufferize][NFC] Simplify buffer API of BufferizationState
Instead of `lookupBuffer` and `getResultBuffer`, there is now a single `getBuffer` function. This simplifies the `BufferizableOpInterface` API and is less confusing to users. They could previously have called the wrong function.

Furthermore, since `getBuffer` now takes an `OpOperand &` instead of a `Value`, users can no longer accidentally use one of the previous two functions incorrectly, which would have resulted in missing buffer copies.

Differential Revision: https://reviews.llvm.org/D116455
2022-01-08 01:12:18 +09:00
Matthias Springer 8e2b6aac32 [mlir][linalg][bufferize][NFC] Analyze OpOperands instead of OpResults
With this change, the analysis takes a look at OpOperands instead of OpResults. OpOperands can bufferize out-of-place (even if they have no aliasing OpResults). The analysis does no longer care about OpResults.

Previously, only OpResults could bufferize out-of-place, so OpOperands that have no aliasing OpResults were never copied by Comprehensive Bufferize. This does not fit wwell with the new CallOp bufferization that is introduced in a subsequent change. In essence, called FuncOps can then be treated as "black boxes" that may read/write to any bbArg, even if they do not return anything.

Differential Revision: https://reviews.llvm.org/D115706
2022-01-08 01:00:30 +09:00
gysit cf05668c17 [mlir][OpDSL] Rename `PrimFn` to `ArithFn`.
The revision renames `PrimFn` to `ArithFn`. The name resembles the newly introduced arith dialect that implements most of the arithmetic functions. An exception are log/exp that are part of the math dialect.

Depends On D115239

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D115240
2022-01-07 12:38:03 +00:00
gysit 15757ea80a [mlir][OpDSL] Add `TypeFn` class.
This revision introduces a the `TypeFn` class that similar to the `PrimFn` class contains an extensible set of type conversion functions. Having the same mechanism for both type conversion functions and arithmetic functions improves code consistency. Additionally, having an explicit function class and function name is a prerequisite to specify a conversion or arithmetic function via attribute. In a follow up commits, we will introduce function attributes to make OpDSL operations more generic. In particular, the goal is to handle signed and unsigned computation in one operations. Today, there is a linalg.matmul and a linalg.matmul_unsigned.

The commit implements the following changes:
- Introduce the class of type conversion functions `TypeFn`
- Replace the hardwired cast and cast_unsigned ops by the `TypeFn` counterparts
- Adapt the python and C++ code generation paths to support the new cast operations

Example:
```
cast(U, A[D.m, D.k])
```
changes to
```
TypeFn.cast(U, A[D.m, D.k])
```

Depends On D115237

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D115239
2022-01-07 12:26:47 +00:00
Alex Zinenko f50cfc44d6 [mlir] Require struct indices in LLVM::GEPOp to be constant
Recent commits added a possibility for indices in LLVM dialect GEP operations
to be supplied directly as constant attributes to ensure they remain such until
translation to LLVM IR happens. Make this required for indexing into LLVM
struct types to match LLVM IR requirements, otherwise the translation would
assert on constructing such IR.

For better compatibility with MLIR-style operation construction interface,
allow GEP operations to be constructed programmatically using Values pointing
to known constant operations as struct indices.

Depends On D116758

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D116759
2022-01-07 09:56:05 +01:00
Alex Zinenko 43ff4a6d55 [mlir] Add ConstantLike trait to LLVM::ConstantOp
This make LLVM dialect constants to work with `m_constant` matches. Implement
the folding hook for this operation as required by the trait. This in turn
allows LLVM::ConstantOp to properly participate in constant-folding.

Depends On D116757

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D116758
2022-01-07 09:56:03 +01:00
Alex Zinenko cafaa35036 [mlir] Make it possible to directly supply constant values to LLVM GEPOp
In LLVM IR, the GEP indices that correspond to structures are required to be
i32 constants. MLIR models constants as just values defined by special
operations, and there is no verification that it is the case for structure
indices in GEP. Furthermore, some common transformations such as control flow
simplification may lead to the operands becoming non-constant. Make it possible
to directly supply constant values to LLVM GEPOp to guarantee they remain
constant until the translation to LLVM IR. This is not yet a requirement and
the verifier is not modified, this will be introduced separately.

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D116757
2022-01-07 09:56:01 +01:00
Kazu Hirata e6075b2c2c [mlir] Fix a warning
This patch fixes:

  mlir/include/mlir/Dialect/Linalg/Transforms/Transforms.h:913:30:
  error: private field 'options' is not used
  [-Werror,-Wunused-private-field]
2022-01-06 18:41:07 -08:00
Nicolas Vasilache 9cd7e880fd [mlir][Linalg] NFC - Modernize more transformation patterns.
Differential Revision: https://reviews.llvm.org/D116763
2022-01-06 17:40:23 -05:00
Matthias Springer 698896cd6c [mlir][linalg][bufferize][NFC] Change allocationFn return type to FailureOr<Value>
In addition, all functions that call `allocationFn` now return FailureOr<Value>. This resolves a few TODOs in the code base.

Differential Revision: https://reviews.llvm.org/D116452
2022-01-07 06:33:19 +09:00
Nicolas Vasilache 4a661602ef [mlir][Linalg] NFC - Modernize APIs and get rid of unnecessary tiling paterns.
Tiling patterns can be reduced to a single pattern by using interface-based patterns.

Differential Revision: https://reviews.llvm.org/D116733
2022-01-06 16:27:35 -05:00
Matthias Springer 75d65293ca [mlir][linalg][bufferize][NFC] Clean up comments and minor code refactorings
Differential Revision: https://reviews.llvm.org/D116451
2022-01-07 06:23:01 +09:00
Matthias Springer 0f5edb9d63 [mlir][linalg][bufferize] Add `createDealloc` options
If `createDealloc` is deactivated (enabled by default), newly allocated buffers are not deallocated anymore. In such a case, the missing deallocations can be inserted by the existing "BufferDeallocation" pass.

This change is needed for unifying core bufferization and Comprehensive Bufferize. Core bufferization has a separate pass for generating deallocations.

Note: In the future, this will evolve towards generating deallocation ops only for buffer allocations that do not escape block boundaries (i.e., that are in destination passing style).

Differential Revision: https://reviews.llvm.org/D116450
2022-01-07 06:13:57 +09:00
Matthias Springer bf9d8d9dfb [mlir][linalg][bufferize][NFC] Rename functions in BufferizationState
The old function names (e.g., `replaceOp`) could have been confusing to users because they sound similar to rewriter functions, but have slightly different semantics.

Differential Revision: https://reviews.llvm.org/D116449
2022-01-07 05:28:58 +09:00
Matthias Springer 15c7e3ee15 [mlir][linalg][bufferize][NFC] Use RewritePatterns instead of custom traversal
This change simplifies BufferizableOpInterface and other functions. Overall, the API will get smaller: Functions related to custom IR traversal are deleted entirely. This will makes it easier to write BufferizableOpInterface implementations.

This is also in preparation of unifying Comprehensive Bufferize and core bufferization. While Comprehensive Bufferize could theoretically maintain its own IR traversal, there is no reason to do so, because all bufferize implementations in BufferizableOpInterface have to support partial bufferization anyway. And we can share a larger part of the code base between the two bufferizations.

Differential Revision: https://reviews.llvm.org/D116448
2022-01-07 00:56:54 +09:00
Matthias Springer 2975407bd4 [mlir][linalg][bufferize][NFC] Pass BufferizationState as const reference
This is mostly for documentation purposes: Passing the object as a const reference signifies that analysis decisions cannot be changed after the analysis.

Differential Revision: https://reviews.llvm.org/D116742
2022-01-07 00:18:46 +09:00
Nicolas Vasilache 2c4a56c418 [mlir][Linalg] NFC - Modernize padding pattern
Differential Revision: https://reviews.llvm.org/D116739
2022-01-06 08:59:35 -05:00
Matthias Springer 0e5f258452 [mlir][linalg][bufferize][NFC] Simplify InsertSliceOp bufferization
No need to keep track of equivalent extract_slice / insert_slice tensors during bufferization. Just emit a copy, it will fold away.

Note: The analysis still keeps track of equivalent tensors to make the correct inplace bufferization decisions.

Differential Revision: https://reviews.llvm.org/D116684
2022-01-06 17:35:45 +09:00
William S. Moses 358d020017 [MLIR][LLVM] Add simple folders for bitcast/addrspacecast/gep
Add 5 simple folders
* bitcast(x : T0, T0) -> x
* addrcast(x : T0, T0) -> x
* bitcast(bitcast(x : T0, T1), T0) -> x
* addrcast(addrcast(x : T0, T1), T0) -> x
* gep %x:T, 0 -> %x:T

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116715
2022-01-05 21:17:32 -05:00
Mogball b0774e5f50 [mlir][ods] ODS ops get an `extraClassDefinition`
Extra definitions are placed in the generated source file for each op class. The substitution `$cppClass` is replaced by the op's C++ class name.

This is useful when declaring but not defining methods in TableGen base classes:

```
class BaseOp<string mnemonic>
    : Op<MyDialect, mnemonic, [DeclareOpInterfaceMethods<SomeInterface>] {
  let extraClassDeclaration = [{
    // ZOp is declared at at the bottom of the file and is incomplete here
    ZOp getParent();
  }];
  let extraClassDefinition = [{
    int $cppClass::someInterfaceMethod() {
      return someUtilityFunction(*this);
    }
    ZOp $cppClass::getParent() {
      return dyn_cast<ZOp>(this->getParentOp());
    }
  }];
}
```

Certain things may prevent defining these functions inline, in the declaration. In this example, `ZOp` in the same dialect is incomplete at the function declaration because ops classes are declared in alphabetical order. Alternatively, functions may be too big to be desired as inlined, or they may require dependencies that create cyclic includes, or they may be calling a templated utility function that one may not want to expose in a header. If the functions are not inlined, then inheriting from the base class N times means that each function will need to be defined N times. With `extraClassDefinitions`, they only need to be defined once.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D115783
2022-01-06 01:43:26 +00:00
Groverkss dde7388ad5 [MLIR] Add clearAndCopyFrom to IntegerPolyhedron
This patch adds clearAndCopyFrom to IntegerPolyhedron. This requires moving
LLVM-style RTTI from FlatAffineConstraints to IntegerPolyhedron.

This patch is part of a series of patches to move presburger math to Presburger
directory.

Reviewed By: arjunp

Differential Revision: https://reviews.llvm.org/D116533
2022-01-05 23:39:26 +05:30
Alex Zinenko 06cc2f2f12 [mlir] Align LLVM_Type ODS constraint on type verifiers
Verify only the outer type being LLVM-compatible, the elemental types if
present are already checked by the type verifiers. This makes some LLVM dialect
operations compatible with mixed-dialect types that appear during progressive
lowering.

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D116671
2022-01-05 19:00:56 +01:00
Nicolas Vasilache 9a7d111f4f [mlir][Linalg] NFC - Modernize transformation APIs.
Differential Revision: https://reviews.llvm.org/D116665
2022-01-05 11:01:40 -05:00
Alex Zinenko 66d4090d9b [mlir] Introduce Python bindings for the quantization dialect
So far, only the custom dialect types are exposed.

The build and packaging is same as for Linalg and SparseTensor, and in
need of refactoring that is beyond the scope of this patch.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D116605
2022-01-05 16:26:31 +01:00
Alex Zinenko 9bcf13bf3e [mlir] Introduce C API for the Quantization dialect types
Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D116546
2022-01-05 16:20:29 +01:00
Alex Zinenko d716cfc4fa [mlir] Use public PybindAdaptors in Linalg dialect bindings
Previously, the Python bindings for the Linalg dialect relied on the internal
implementation of core bindings. Most of that functionality was moved, and the
remaining one does not need access to the implementation: it used to accept a
dialect pointer as argument, but it can always be extracted from the operation
that it also accepts; operations are available through PybindAdaptors in an
opaque way. Change the bindings in that direction.

This enables the decoupling of the Linalg dialect Python extension from the
core IR Python extension.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D116649
2022-01-05 16:18:30 +01:00
Matthias Springer ed5e3590a3 [mlir][linalg][bufferize][NFC] Remove RewriterBase from BufferizationState
This change simplifies BufferizationState. Having `rewriter` in BufferizationState could be confusing to users because a rewriter is also passed to each `bufferize` function and it is not obvious (by looking at the API) that these two rewriters are the same.

Differential Revision: https://reviews.llvm.org/D116444
2022-01-06 00:04:43 +09:00
Nicolas Vasilache c7dd0bf41d [mlir][vector] NFC - Split out transfer split patterns
Differential Revision: https://reviews.llvm.org/D116648
2022-01-05 08:38:04 -05:00
Matthias Springer 6c6bba7436 [mlir][linalg][bufferize][NFC] Use RewriterBase instead of OpBuilder
This is in preparation of unifying core bufferization and Comprehensive Bufferize.

Differential Revision: https://reviews.llvm.org/D116102
2022-01-05 21:05:42 +09:00
Matthias Springer 46e316651f [mlir][linalg][bufferize][NFC] Refactor BufferizationOption ownership
Pass unique_ptr<BufferizationOption> to the bufferization. This allows the bufferization to enqueue additional PostAnalysisSteps. When running bufferization a second time, a new BufferizationOptions must be constructed.

Differential Revision: https://reviews.llvm.org/D116101
2022-01-05 20:24:54 +09:00
Nicolas Vasilache 11b67aaffb [mlir][scf] NFC - refactor the implementation of outlineIfOp
This revision refactors the implementation of outlineIfOp to expose
a finer-grain functionality `outlineSingleBlockRegion` that will be
reused in other contexts.

Differential Revision: https://reviews.llvm.org/D116591
2022-01-05 05:02:26 -05:00
wren romano bc04a47038 [mlir][sparse] adding OverheadType::kIndex
Depends On D115008

This change opens the way for D115012, and removes some corner cases in `CodegenUtils.cpp`. The `SparseTensorAttrDefs.td` already specifies that we allow `0` bitwidth for the two overhead types and that it is interpreted to mean the architecture's native width.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D115010
2022-01-04 16:15:54 -08:00
Benjamin Kramer 41760a6b40 [mlir] Make Value's constructor constexpr. NFCI.
This allows clang to flag unused Values in more cases, so remove them.
2022-01-04 21:04:13 +01:00
Groverkss 4ca510b1d1 [MLIR] Remove dependency on IR for Simplex
This patch removes unnecessary dependency on IR for Simplex. This patch allows
users to use Presburger library without depending on MLIRIR.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D116530
2022-01-03 16:23:11 +05:30
Mehdi Amini 1461bd13c9 Revert "Define a `cppAccessorType` to const-ref in APFloatParameter and update ODS emitter to use it for verifier signatures"
This reverts commit 89af17c0c7.

This broke the gcc5 build.
2022-01-03 06:32:50 +00:00
William S. Moses 834cf3be22 [MLIR][Arith] Canonicalize and/or with ext
Replace and(ext(a),ext(b)) with ext(and(a,b)). This both reduces one instruction, and results in the computation (and/or) being done on a smaller type.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116519
2022-01-03 01:25:30 -05:00
Groverkss 29120a5130 [MLIR][NFC] Fix clang-tidy errors in Analysis/Presburger/
This patch fixes clang-tidy errors related to different parameter names
in header file and source file in Analysis/Presburger/ directory.
2022-01-03 11:42:48 +05:30
Groverkss 8506c8c13b [MLIR] Move LinearTransform to Presburger/
This patch moves LinearTransform to Presburger/ and makes it use
IntegerPolyhedron instead of FlatAffineConstraints. Also modifies its usage in
`FlatAffineConstraints::findIntegerSample` to support the changes.

This patch is part of a series of patches for moving presburger math functionality into Presburger directory.

Reviewed By: arjunp

Differential Revision: https://reviews.llvm.org/D116311
2022-01-03 11:11:09 +05:30
William S. Moses 1bb9f4e482 [MLIR] Create folders for extsi/extui
Create folders/canonicalizers for extsi/extui. Specifically,

extui(extui(x)) -> extui(x)
extsi(extsi(x)) -> extsi(x)
extsi(extui(x)) -> extui(x)

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116515
2022-01-03 00:11:23 -05:00
Mehdi Amini 89af17c0c7 Define a `cppAccessorType` to const-ref in APFloatParameter and update ODS emitter to use it for verifier signatures
This reduce an unnecessary amount of copy of non-trivial objects, like
APFloat.

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D116505
2022-01-03 04:57:11 +00:00
Mehdi Amini 3f127d8aaa Use `= default` for the ValueShapeRange copy-constructor (NFC)
This makes it trivially copyable.
2022-01-02 23:43:52 +00:00
Mehdi Amini ad5d7ace34 Apply clang-tidy fixes for readability-const-return-type to MLIR (NFC)
Reviewed By: rriddle, Mogball

Differential Revision: https://reviews.llvm.org/D116251
2022-01-02 01:51:39 +00:00
Mehdi Amini 1fc096af1e Apply clang-tidy fixes for performance-unnecessary-value-param to MLIR (NFC)
Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D116250
2022-01-02 01:45:18 +00:00
Mehdi Amini 104a827ea6 Move LinalgDetensorize pass option from .cpp file to the .td declaration (NFC) 2022-01-01 21:19:31 +00:00
Mehdi Amini 07b264d1f0 Pass the LLVMTypeConverter by reference in UnrankedMemRefBuilder (NFC)
This is a fairly large structure (952B according to Coverity), it was
already passed by reference in most places but not consistently.
2022-01-01 02:01:41 +00:00
Mehdi Amini bb6109aae6 Pass the LLVMTypeConverter by reference in MemRefBuilder (NFC)
This is a fairly large structure (952B according to Coverity), it was
already passed by reference in most places but not consistently.
2022-01-01 01:56:50 +00:00
Mehdi Amini a9f13f8065 Fix a few unitialized class members in MLIR (NFC)
Flagged by Coverity.
2022-01-01 01:40:36 +00:00
Mehdi Amini 8637be74a0 Remove redundant return after return in CodegenStrategy (NFC)
Reported by Coverity
2022-01-01 01:14:27 +00:00
Markus Böck 3536d24a1a [mlir][LLVMIR] Add `llvm.eh.typeid.for` intrinsic
MLIR already exposes landingpads, the invokeop and the personality function on LLVM functions. With this intrinsic it should be possible to implement exception handling via the exception handling mechanisms provided by the Itanium ABI.

Differential Revision: https://reviews.llvm.org/D116436
2022-01-01 02:03:00 +01:00
MaheshRavishankar 59442a5460 [mlir][Linalg] Change signature of `get(Parallel/Reduce/Window)Dims` method.
These method currently takes a SmallVector<AffineExpr> & as an
argument to return the dims as AffineExpr. This creation of
AffineExpr objects is unnecessary.

Differential Revision: https://reviews.llvm.org/D116422
2021-12-30 14:02:15 -08:00
long.chen d295dd10f2 [MLIR] Add explicit `using` to disambiguate between multiple implementations from base classes (NFC)
Both of DenseElementsAttr and ElementsAttrTrait define the method of
getElementType, this commit makes it available on
DenseIntOrFPElementsAttr and DenseStringElementsAttr.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116389
2021-12-30 19:47:33 +00:00
William S. Moses a6a583dae4 [MLIR] Move AtomicRMW into MemRef dialect and enum into Arith
Per the discussion in https://reviews.llvm.org/D116345 it makes sense
to move AtomicRMWOp out of the standard dialect. This was accentuated by the
need to add a fold op with a memref::cast. The only dialect
that would permit this is the memref dialect (keeping it in the standard dialect
or moving it to the arithmetic dialect would require those dialects to have a
dependency on the memref dialect, which breaks linking).

As the AtomicRMWKind enum is used throughout, this has been moved to Arith.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D116392
2021-12-30 14:31:33 -05:00
Jacques Pienaar 4a8cef157b [mlir] Change SCF/Complex to prefixed (NFC)
See
https://llvm.discourse.group/t/psa-ods-generated-accessors-will-change-to-have-a-get-prefix-update-you-apis/4476
2021-12-30 09:57:51 -08:00
William S. Moses 180455ae5e [MLIR][LLVM] Expose powi intrinsic to MLIR
Expose the powi intrinsic to the LLVM dialect within MLIR

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116364
2021-12-29 13:09:35 -05:00
Kazu Hirata 773ea16eba [AST] Fix a warning
This patch fixes:

  mlir/include/mlir/Tools/PDLL/AST/Types.h:54:3: error: definition of
  implicit copy assignment operator for 'Type' is deprecated because
  it has a user-declared copy constructor [-Werror,-Wdeprecated-copy]
2021-12-28 22:52:56 -08:00
William S. Moses 99fc000c87 [MLIR] Expose atomicrmw and/or
LLVM (dialect and IR) have atomics for and/or. This patch enables atomic_rmw ops in the standard dialect for and/or that lower to these (in addition to the existing atomics such as addi, etc).

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116345
2021-12-29 00:23:28 -05:00
William S. Moses ca8997eb7f [MLIR] Add constant folder for fptosi and friends
This patch adds constant folds for FPToSI/FPToUI/SIToFP/UIToFP

Reviewed By: mehdi_amini, bondhugula

Differential Revision: https://reviews.llvm.org/D116321
2021-12-28 23:50:01 -05:00
Rob Suderman f0cb77d7d5 [mlir][tosa] Resubmit split tosa-to-linalg named ops out of pass
Includes dependency fix that resulted in canonicalizer pass not linking in.

Linalg named ops lowering are moved to a separate pass. This allows TOSA
canonicalizers to run between named-ops lowerings and the general TOSA
lowerings. This allows the TOSA canonicalizers to run between lowerings.

Differential Revision: https://reviews.llvm.org/D116057
2021-12-28 11:22:58 -08:00
Groverkss 5f22f248d8 [MLIR] Use IntegerPolyhedron in Simplex instead of FlatAffineConstraints
This patch replaces usage of FlatAffineConstraints in Simplex with
IntegerPolyhedron. This removes dependency of Simplex on FlatAffineConstraints
and puts it on IntegerPolyhedron, which is part of Presburger library.

Reviewed By: arjunp

Differential Revision: https://reviews.llvm.org/D116287
2021-12-27 19:06:35 +05:30
Groverkss 3f22d492ac [MLIR] Move `print()` and `dump()` from FlatAffineConstraints to IntegerPolyhedron.
This patch moves `FlatAffineConstraints::print` and
`FlatAffineConstraints::dump()` to IntegerPolyhedron.

Reviewed By: arjunp

Differential Revision: https://reviews.llvm.org/D116289
2021-12-27 18:40:49 +05:30
Arjun P 4fe5cfe53e [MLIR] Add forgotten directory Support to unittests cmake
The Support directory was removed from the unittests cmake when the directory
was removed in 204c3b5516. Subsequent commits
added the directory back but seem to have missed adding it back to the cmake.

This patch also removes MLIRSupportIndentedStream from the list of linked
libraries to avoid an ODR violation (it's already part of MLIRSupport which
is also being linked here). Otherwise ASAN complains:

```
=================================================================
==102592==ERROR: AddressSanitizer: odr-violation (0x7fbdf214eee0):
  [1] size=120 'vtable for mlir::raw_indented_ostream' /home/arjun/llvm-project/mlir/lib/Support/IndentedOstream.cpp
  [2] size=120 'vtable for mlir::raw_indented_ostream' /home/arjun/llvm-project/mlir/lib/Support/IndentedOstream.cpp
These globals were registered at these points:
  [1]:
    #0 0x28a71d in __asan_register_globals (/home/arjun/llvm-project/build/tools/mlir/unittests/Support/MLIRSupportTests+0x28a71d)
    #1 0x7fbdf214a61b in asan.module_ctor (/home/arjun/llvm-project/build/lib/libMLIRSupportIndentedOstream.so.14git+0x661b)

  [2]:
    #0 0x28a71d in __asan_register_globals (/home/arjun/llvm-project/build/tools/mlir/unittests/Support/MLIRSupportTests+0x28a71d)
    #1 0x7fbdf2061c4b in asan.module_ctor (/home/arjun/llvm-project/build/lib/libMLIRSupport.so.14git+0x11bc4b)

==102592==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY AddressSanitizer: odr-violation: global 'vtable for mlir::raw_indented_ostream' at /home/arjun/llvm-project/mlir/lib/Support/IndentedOstream.cpp
==102592==ABORTING
```

This patch also fixes a build issue with `DebugAction::classof` under Windows.

This commit re-lands this patch, which was previously reverted in
2132906836 due to a buildbot failure that
turned out to be because of a flaky test.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D116027
2021-12-27 14:42:48 +05:30
Arjun P 2132906836 Revert "[MLIR] Add forgotten directory Support to unittests cmake"
This reverts commit 0c553cc1af.

This caused a buildbot failure (https://lab.llvm.org/buildbot#builders/197/builds/888).

```
******************** TEST 'ScudoStandalone-Unit :: ./ScudoUnitTest-aarch64-Test/ScudoCommonTest.ResidentMemorySize' FAILED ********************
Script:
--
/home/tcwg-buildbot/worker/clang-aarch64-sve-vla/stage1/projects/compiler-rt/lib/scudo/standalone/tests/./ScudoUnitTest-aarch64-Test --gtest_filter=ScudoCommonTest.ResidentMemorySize
--
Note: Google Test filter = ScudoCommonTest.ResidentMemorySize
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from ScudoCommonTest
[ RUN      ] ScudoCommonTest.ResidentMemorySize
/home/tcwg-buildbot/worker/clang-aarch64-sve-vla/llvm/compiler-rt/lib/scudo/standalone/tests/common_test.cpp:49: Failure
Expected: (getResidentMemorySize()) > (OnStart + Size - Threshold), actual: 707358720 vs 943153152
[  FAILED  ] ScudoCommonTest.ResidentMemorySize (21709 ms)
[----------] 1 test from ScudoCommonTest (21709 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (21709 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] ScudoCommonTest.ResidentMemorySize
 1 FAILED TEST
********************
```
2021-12-26 13:59:23 +05:30
Arjun P 0c553cc1af [MLIR] Add forgotten directory Support to unittests cmake
The Support directory was removed from the unittests cmake when the directory
was removed in 204c3b5516. Subsequent commits
added the directory back but seem to have missed adding it back to the cmake.

This patch also removes MLIRSupportIndentedStream from the list of linked
libraries to avoid an ODR violation (it's already part of MLIRSupport which
is also being linked here). Otherwise ASAN complains:

```
=================================================================
==102592==ERROR: AddressSanitizer: odr-violation (0x7fbdf214eee0):
  [1] size=120 'vtable for mlir::raw_indented_ostream' /home/arjun/llvm-project/mlir/lib/Support/IndentedOstream.cpp
  [2] size=120 'vtable for mlir::raw_indented_ostream' /home/arjun/llvm-project/mlir/lib/Support/IndentedOstream.cpp
These globals were registered at these points:
  [1]:
    #0 0x28a71d in __asan_register_globals (/home/arjun/llvm-project/build/tools/mlir/unittests/Support/MLIRSupportTests+0x28a71d)
    #1 0x7fbdf214a61b in asan.module_ctor (/home/arjun/llvm-project/build/lib/libMLIRSupportIndentedOstream.so.14git+0x661b)

  [2]:
    #0 0x28a71d in __asan_register_globals (/home/arjun/llvm-project/build/tools/mlir/unittests/Support/MLIRSupportTests+0x28a71d)
    #1 0x7fbdf2061c4b in asan.module_ctor (/home/arjun/llvm-project/build/lib/libMLIRSupport.so.14git+0x11bc4b)

==102592==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY AddressSanitizer: odr-violation: global 'vtable for mlir::raw_indented_ostream' at /home/arjun/llvm-project/mlir/lib/Support/IndentedOstream.cpp
==102592==ABORTING
```

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D116027
2021-12-26 02:22:00 +05:30
Groverkss 5b2e611b73 [MLIR][FlatAffineConstraints][NFC] Move some static functions to be available to Presburger/
This patch moves some static functions from AffineStructures.cpp to
Presburger/Utils.cpp and some to be private members of FlatAffineConstraints
(which will later be moved to IntegerPolyhedron) to allow for a smoother
transition for moving FlatAffineConstraints math functionality to
Presburger/IntegerPolyhedron.

This patch is part of a series of patches for moving math functionality to
Presburger directory.

Reviewed By: arjunp, bondhugula

Differential Revision: https://reviews.llvm.org/D115869
2021-12-25 22:36:23 +05:30
William S. Moses 2709fd1520 [MLIR][LLVM] Add MemmoveOp to LLVM Dialect
LLVM Dialect in MLIR doesn't have a memmove op. This adds one.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116274
2021-12-24 19:48:04 -05:00
Groverkss 27a0718ad0 Revert "[MLIR][FlatAffineConstraints][NFC] Move some static functions to be available to Presburger/"
This reverts commit 6c0eaefaf8.
2021-12-25 00:39:27 +05:30
Groverkss 6c0eaefaf8 [MLIR][FlatAffineConstraints][NFC] Move some static functions to be available to Presburger/
This patch moves some static functions from AffineStructures.cpp to
Presburger/Utils.cpp and some to be private members of FlatAffineConstraints
(which will later be moved to IntegerPolyhedron) to allow for a smoother
transition for moving FlatAffineConstraints math functionality to
Presburger/IntegerPolyhedron.

This patch is part of a series of patches for moving math functionality to
Presburger directory.

Reviewed By: arjunp, bondhugula

Differential Revision: https://reviews.llvm.org/D115869
2021-12-25 00:11:35 +05:30
Mogball 41a64338cc [mlir] Add getNumThreads to MLIRContext
Querying threads directly from the thread pool fails if there is no thread pool or if multithreading is not enabled. Returns 1 by default.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116259
2021-12-24 02:02:54 +00:00
Mehdi Amini 735fe1da6b Revert "[mlir][tosa] Split tosa-to-linalg named ops out of pass"
This reverts commit 313de31fbb.

There is a missing CMake dependency, building with shared libraries is
broken:

55.509 [45/4/3061] Linking CXX shared library lib/libMLIRTosaToLinalg.so.14git
FAILED: lib/libMLIRTosaToLinalg.so.14git
...
TosaToLinalgPass.cpp: undefined reference to `mlir::createCanonicalizerPass()'
2021-12-24 00:09:15 +00:00
Rob Suderman 313de31fbb [mlir][tosa] Split tosa-to-linalg named ops out of pass
Linalg named ops lowering are moved to a separate pass. This allows TOSA
canonicalizers to run between named-ops lowerings and the general TOSA
lowerings. This allows the TOSA canonicalizers to run between lowerings.

Reviewed By: NatashaKnk

Differential Revision: https://reviews.llvm.org/D116057
2021-12-23 12:23:19 -08:00
Mogball 42ac4f3dc6 [mlir] Canonicalizer constructor should accept disabled/enabled patterns
There is no way to programmatically configure the list of disabled and enabled patterns in the canonicalizer pass, other than the duplicate the whole pass. This patch exposes the `disabledPatterns` and `enabledPatterns` options.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116055
2021-12-22 19:08:31 +00:00
Stephan Herhut 8761f5ebf7 [mlir][Support] Avoid multiplication in floorDiv / ceilDiv
Using comparisons instead avoids potential overflow.

Differential Revision: https://reviews.llvm.org/D116096
2021-12-21 11:50:40 +01:00
MaheshRavishankar 4142932a83 [mlir][Linalg] Move named op conversions out of canonicalizations.
These conversions are better suited to be applied at whole tensor
level. Applying these as canonicalizations end up triggering such
canonicalizations at all levels of the stack which might be
undesirable. For example some of the resulting code patterns wont
bufferize in-place and need additional stack buffers. Best is to be
more deliberate in when these canonicalizations apply.

Differential Revision: https://reviews.llvm.org/D115912
2021-12-20 10:19:05 -08:00
Jacques Pienaar c0342a2de8 [mlir] Switching accessors to prefixed form (NFC)
Makes eventual prefixing flag flip smaller change.
2021-12-20 08:03:43 -08:00
Mehdi Amini 7f9e9c7fc3 Move getAsmBlockArgumentNames from OpAsmDialectInterface to OpAsmOpInterface
This method is more suitable as an opinterface: it seems intrinsic to
individual instances of the operation instead of the dialect.
Also remove the restriction on the interface being applicable to the entry block only.

Differential Revision: https://reviews.llvm.org/D116018
2021-12-20 07:18:01 +00:00
Arjun P 4fa96b7eca [MLIR] Simplex: split some basic functionality out into a SimplexBase class
This is a purely mechanical patch moving some functionality out from the
`Simplex` class out into a `SimplexBase` class. This pavees the way for
a future patch adding support for lexicographic optimization with a class
`LexSimplex`, which will inherit from `SimplexBase`. Inheriting directly
from `Simplex` would bring many additional functions that would not work in
`LexSimplex` because it operates slighty differently from `Simplex`. So We
split out only the basic functionality it needs to inherit into `SimplexBase`.

Reviewed By: Groverkss

Differential Revision: https://reviews.llvm.org/D115831
2021-12-19 22:24:40 +05:30
bakhtiyar ec0e4545ca Make AsyncParallelForRewrite parameterizable with a cost model which drives deciding the parallelization granularity.
Reviewed By: ezhulenev, mehdi_amini

Differential Revision: https://reviews.llvm.org/D115423
2021-12-19 08:41:01 -08:00
Aaron DeBattista 64f694acaf [mlir][tosa] Move tosa canonicalizers to optional optimization pass
TOSA's canonicalizers that change dense operations should be moved to a
seperate optimization pass to avoid canonicalizing to operations not supported
for relevant backends.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D115890
2021-12-16 23:33:54 -08:00
Mogball 319d8cf685 [mlir][ods] Added EnumAttr, an AttrDef implementation of enum attributes
`EnumAttr` is a pure TableGen implementation of enum attributes using `AttrDef`. This is meant as a drop-in replacement for `StrEnumAttr`, which is soon to be deprecated. `StrEnumAttr` is often used over `IntEnumAttr` because its more readable in MLIR assembly formats. However, storing and manipulating strings is not efficient. Defining `StrEnumAttr` can also be awkward and relies on a lot of special logic in `EnumsGen`, and has some hidden sharp edges.

Also, `EnumAttr` stores the enum directly,  removing the need to convert to/from integers when calling attribute getters on ops.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D115181
2021-12-17 02:55:28 +00:00
not-jenni f9cefc7b90 [mlir][tosa] Add tosa.max_pool2d as no-op canonicalization
When the input and output of a pool2d op are both 1x1, it can be canonicalized to a no-op

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D115908
2021-12-16 15:27:26 -08:00
Alexander Belyaev f77e9f8768 [mlir] Extend `tensor.from_elements` to support N-D case.
RFC: https://llvm.discourse.group/t/rfc-extend-tensor-fromelementsop-to-n-d/4715

Differential Revision: https://reviews.llvm.org/D115821
2021-12-16 14:52:41 +01:00
Diego Caballero 32fe1a8a25 [mlir][GPU] Extend GPU kernel outlining to generate DL specification
This patch extends the GPU kernel outlining pass so that it can take in
an optional data layout specification that will be attached to the GPU
module operation generated. If the data layout specification is not provided
the default data layout is used instead.

Reviewed By: herhut, mehdi_amini

Differential Revision: https://reviews.llvm.org/D115722
2021-12-16 11:35:53 +00:00
Diego Caballero 0ca00c3538 [mlir][vector] Remove default value in populateVectorMultiReductionLoweringPatterns
Having a default value for the lowering strategy of the multi-reduction op has proven
to be unexpected by users. This patch is dropping the default value so that users have
to explicitly choose the lowering strategy to be applied.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D115805
2021-12-16 09:45:34 +00:00
Matthias Springer ec8628b1d6 [mlir][linalg][bufferize][NFC] Pass BufferizationState into all op interface methods
This allows op interface implementations to make decisions based on dialect-specific bufferization state.

This is in preparation of fixing conflict detection of CallOps in ModuleBufferization.

Differential Revision: https://reviews.llvm.org/D115705
2021-12-16 11:45:13 +09:00
River Riddle 3ee44cb775 [PDLL] Add a `rewrite` statement to enable complex rewrites
The `rewrite` statement allows for rewriting a given root
operation with a block of nested rewriters. The root operation is
not implicitly erased or replaced, and any transformations to it
must be expressed within the nested rewrite block. The inner body
may contain any number of other rewrite statements, variables, or
expressions.

Differential Revision: https://reviews.llvm.org/D115299
2021-12-16 02:08:13 +00:00
River Riddle 12eebb8e37 [PDLL] Add a `replace` rewrite statement for replacing operations
This statement acts as a companion to the existing `erase`
statement, and is the corresponding PDLL construct for the
`PatternRewriter::replaceOp` C++ API. This statement replaces a
given operation with a set of values.

Differential Revision: https://reviews.llvm.org/D115298
2021-12-16 02:08:13 +00:00
River Riddle f62a57a3f0 [PDLL] Add support for tuple types and expressions
Tuples are used to group multiple elements into a single
compound value. The values in a tuple can be of any type, and
do not need to be of the same type. There is also no limit to
the number of elements held by a tuple.

Tuples will be used to support multiple results from
Constraints and Rewrites (added in a followup), and will also
make it easier to support more complex primitives (such as
range based maps that can operate on multiple values).

Differential Revision: https://reviews.llvm.org/D115297
2021-12-16 02:08:13 +00:00
River Riddle 02670c3f38 [PDLL] Add support for `op` Operation expressions
An operation expression in PDLL represents an MLIR operation. In
the match section of a pattern, this expression models one of
the input operations to the pattern. In the rewrite section of
a pattern, this expression models one of the operations to
create. The general structure of the operation expression is very
similar to that of the "generic form" of textual MLIR assembly:

```
let root = op<my_dialect.foo>(operands: ValueRange) {attr = attr: Attr} -> (resultTypes: TypeRange);
```

For now we only model the components that are within PDL, as PDL
gains support for blocks and regions so will this expression.

Differential Revision: https://reviews.llvm.org/D115296
2021-12-16 02:08:12 +00:00
River Riddle d7e7fdf3aa [PDLL] Add support for literal Attribute and Type expressions
This allows for using literal attributes and types within PDLL,
which simplifies building both constraints and rewriters. For
example, checking if an attribute is true is as simple as
`attr<"true">`.

Differential Revision: https://reviews.llvm.org/D115295
2021-12-16 02:08:12 +00:00
River Riddle 11d26bd143 [mlir][PDLL] Add an initial frontend for PDLL
This is a new pattern rewrite frontend designed from the ground
up to support MLIR constructs, and to target PDL. This frontend
language was proposed in https://llvm.discourse.group/t/rfc-pdll-a-new-declarative-rewrite-frontend-for-mlir/4798

This commit starts sketching out the base structure of the
frontend, and is intended to be a minimal starting point for
building up the language. It essentially contains support for
defining a pattern, variables, and erasing an operation. The
features mentioned in the proposal RFC (including IDE support)
will be added incrementally in followup commits.

I intend to upstream the documentation for the language in a
followup when a bit more of the pieces have been landed.

Differential Revision: https://reviews.llvm.org/D115093
2021-12-16 02:08:12 +00:00
Jacques Pienaar aed288d6df [mlir] Flip Complex & SCF dialects to _Both (NFC)
Following
https://llvm.discourse.group/t/psa-ods-generated-accessors-will-change-to-have-a-get-prefix-update-you-apis/4476
2021-12-15 08:21:38 -08:00
Mogball c7103810bd [mlir][scf] Add getNumRegionInvocations to IfOp
Implements the RegionBranchOpInterface method getNumRegionInvocations to `scf::IfOp` so that, when the condition is constant, the number of region executions can be analyzed by `NumberOfExecutions`.

Reviewed By: jpienaar, ftynse

Differential Revision: https://reviews.llvm.org/D115087
2021-12-15 14:56:20 +00:00
Matthias Springer 417014170b [mlir][linalg][bufferize] Replace remaining bvm usage with new API
* Call `replaceOp` instead of `mapBuffer`.
* Remove bvm and all helper functions around bvm.
* Simplify FuncOp bufferization and rely on existing functionality to generate ToMemrefOps for function BlockArguments.

Differential Revision: https://reviews.llvm.org/D115515
2021-12-15 23:21:39 +09:00
Markus Böck 1d10bddfa3 [mlir][LLVMIR] Add `llvm.umin` and `llvm.umax` intrinsics
Ops for the signed counterparts "llvm.smin" and "llvm.smax" already exist. This patch adds the unsigned versions as well.

Differential Revision: https://reviews.llvm.org/D115796
2021-12-15 13:54:31 +01:00
gysit b7f2c108eb [mlir][linalg] Replace LinalgOps.h and LinalgTypes.h by a single header.
After removing the range type, Linalg does not define any type. The revision thus consolidates the LinalgOps.h and LinalgTypes.h into a single Linalg.h header. Additionally, LinalgTypes.cpp is renamed to LinalgDialect.cpp to follow the convention adopted by other dialects such as the tensor dialect.

Depends On D115727

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D115728
2021-12-15 12:15:03 +00:00
Javier Setoain a4830d14ed [mlir][RFC] Add scalable dimensions to VectorType
With VectorType supporting scalable dimensions, we don't need many of
the operations currently present in ArmSVE, like mask generation and
basic arithmetic instructions. Therefore, this patch also gets
rid of those.

Having built-in scalable vector support also simplifies the lowering of
scalable vector dialects down to LLVMIR.

Scalable dimensions are indicated with the scalable dimensions
between square brackets:

        vector<[4]xf32>

Is a scalable vector of 4 single precission floating point elements.

More generally, a VectorType can have a set of fixed-length dimensions
followed by a set of scalable dimensions:

        vector<2x[4x4]xf32>

Is a vector with 2 scalable 4x4 vectors of single precission floating
point elements.

The scale of the scalable dimensions can be obtained with the Vector
operation:

        %vs = vector.vscale

This change is being discussed in the discourse RFC:

https://llvm.discourse.group/t/rfc-add-built-in-support-for-scalable-vector-types/4484

Differential Revision: https://reviews.llvm.org/D111819
2021-12-15 09:31:37 +00:00
Matthias Springer 7161aa06ef [mlir][linalg][bufferize] Reimplementation of scf.for bufferization
Instead of modifying the existing scf.for op, create a new op with memref OpOperands/OpResults and delete the old op.

New allocations / other memrefs can now be yielded from the loop. This functionality is deactivated by default and guarded against by AssertDestinationPassingStyle.

This change also introduces `replaceOp`, which will be utilized by all other `bufferize` implementations in future commits. Bufferization will then no longer rely on old (pre-bufferize) ops to DCE away. Instead old ops are deleted on the spot. This improves debuggability because there won't be any duplicate ops anymore (bufferized + not-yet-bufferized) when dumping IR during bufferization. It is also less fragile because unbufferized IR can no longer silently "hang around" due to an implementation bug.

Differential Revision: https://reviews.llvm.org/D114926
2021-12-15 18:29:22 +09:00
Julian Gross 524d6a2d6a [mlir] Added documentation for bufferization to memref conversion pass.
Added documentation to clearify the purpose of the bufferization to memref pass
and added some remarks.

Differential Revision: https://reviews.llvm.org/D115326
2021-12-15 10:16:41 +01:00
gysit 9912bed730 [mlir][linalg] Remove RangeOp and RangeType.
Remove the RangeOp and the RangeType that are not actively used anymore. After removing RangeType, the LinalgTypes header only includes the generated dialect header.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D115727
2021-12-15 07:19:10 +00:00
Thomas Raoux 7d97678df7 [mlir][linalg] Break up linalg vectorization pre-condition
Break up the vectorization pre-condition into the part checking for
static shape and the rest checking if the linalg op is supported by
vectorization. This allows checking if an op could be vectorized if it
had static shapes.

Differential Revision: https://reviews.llvm.org/D115754
2021-12-14 13:38:14 -08:00
Aart Bik ef244ad620 [mlir][sparse] fixed typos
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D115667
2021-12-14 08:33:06 -08:00
Matthias Springer 81eece7f26 [mlir][linalg][bufferize] Debug output as IR attributes
Instead of printing analysis debug information to stderr, annotate the IR. This makes it easier to understand decisions made by the analysis, especially in larger input IR.

Differential Revision: https://reviews.llvm.org/D115575
2021-12-14 21:29:43 +09:00
Alexander Belyaev 15f8f3e20a [mlir] Split std.rank into tensor.rank and memref.rank.
Move `std.rank` similarly to how `std.dim` was moved to TensorOps and MemRefOps.

Differential Revision: https://reviews.llvm.org/D115665
2021-12-14 10:15:55 +01:00
Markus Böck ef5be2bb16 [mlir] Implement `DataLayoutTypeInterface` for `LLVMArrayType`
Implementation of the interface allows querying the size and alignments of an LLVMArrayType as well as query the size and alignment of a struct containing an LLVMArrayType.
The implementation should yield the same results as llvm::DataLayout, including support for over aligned element types.
There is no customization point for adjusting an arrays alignment; it is simply taken from the element type.

Differential Revision: https://reviews.llvm.org/D115704
2021-12-14 09:35:45 +01:00
Benoit Jacob aba437ceb2 [mlir][Vector] Patterns flattening vector transfers to 1D
This is the second part of https://reviews.llvm.org/D114993 after slicing
into 2 independent commits.

This is needed at the moment to get good codegen from 2d vector.transfer
ops that aim to compile to SIMD load/store instructions but that can
only do so if the whole 2d transfer shape is handled in one piece, in
particular taking advantage of the memref being contiguous rowmajor.

For instance, if the target architecture has 128bit SIMD then we would
expect that contiguous row-major transfers of <4x4xi8> map to one SIMD
load/store instruction each.

The current generic lowering of multi-dimensional vector.transfer ops
can't achieve that because it peels dimensions one by one, so a transfer
of <4x4xi8> becomes 4 transfers of <4xi8>.

The new patterns here are only enabled for now by
 -test-vector-transfer-flatten-patterns.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114993
2021-12-13 22:39:41 +00:00
Benoit Jacob 0aea49a730 [mlir][Vector] Patterns flattening vector transfers to 1D
This is the first part of https://reviews.llvm.org/D114993 which has been
split into small independent commits.

This is needed at the moment to get good codegen from 2d vector.transfer
ops that aim to compile to SIMD load/store instructions but that can
only do so if the whole 2d transfer shape is handled in one piece, in
particular taking advantage of the memref being contiguous rowmajor.

For instance, if the target architecture has 128bit SIMD then we would
expect that contiguous row-major transfers of <4x4xi8> map to one SIMD
load/store instruction each.

The current generic lowering of multi-dimensional vector.transfer ops
can't achieve that because it peels dimensions one by one, so a transfer
of <4x4xi8> becomes 4 transfers of <4xi8>.

The new patterns here are only enabled for now by
 -test-vector-transfer-flatten-patterns.

Reviewed By: nicolasvasilache
2021-12-13 21:49:04 +00:00
Stella Laurenzo c10995a8ad Re-apply [NFC] Generalize a couple of passes so they can operate on any FunctionLike op.
* Generalizes passes linalg-detensorize, linalg-fold-unit-extent-dims, convert-elementwise-to-linalg.
* I feel that more work could be done in the future (i.e. make FunctionLike into a proper OpInterface and extend actions in dialect conversion to be trait based), and this patch would be a good record of why that is useful.
* Note for downstreams:
  * Since these passes are now generic, they do not automatically nest with pass managers set up for implicit nesting.
  * The Detensorize pass must run on a FunctionLike, and this requires explicit nesting.
* Addressed missed comments from the original and per-suggestion removed the assert on FunctionLike in ElementwiseToLinalg and DropUnitDims.cpp, which also is what was causing the integration test to fail.

This reverts commit aa8815e42e.

Differential Revision: https://reviews.llvm.org/D115671
2021-12-13 13:33:00 -08:00
Mehdi Amini aa8815e42e Revert "[NFC] Generalize a couple of passes so they can operate on any FunctionLike op."
This reverts commit 34696e6542.

A test is crashing on the mlir-nvidia bot.
2021-12-13 20:41:25 +00:00
Stella Laurenzo 34696e6542 [NFC] Generalize a couple of passes so they can operate on any FunctionLike op.
* Generalizes passes linalg-detensorize, linalg-fold-unit-extent-dims, convert-elementwise-to-linalg.
* I feel that more work could be done in the future (i.e. make FunctionLike into a proper OpInterface and extend actions in dialect conversion to be trait based), and this patch would be a good record of why that is useful.
* Note for downstreams:
  * Since these passes are now generic, they do not automatically nest with pass managers set up for that.
  * If running them over nested functions, you must nest explicitly. Upstream has adopted this style but *-opt still has some uses of implicit pipelines via args. See tests for argument changes needed.

Differential Revision: https://reviews.llvm.org/D115645
2021-12-13 12:01:53 -08:00
Markus Böck 664cc9312c [mlir] Implement `DataLayoutTypeInterface` for `LLVMStructType`
Using this implementation of the interface it is possible to query the size, ABI alignment as well as the preferred alignment of a struct. It should yield the same results as LLVMs `llvm::DataLayout` on an equivalent `llvm::StructType`, including for packed structs.

Additionally it is also possible to increase the ABI and preferred alignment using a data layout entry with the type `llvm.struct<()>, which serves the same functionality as the `a:` component in LLVMs data layout string.

Differential Revision: https://reviews.llvm.org/D115600
2021-12-13 15:09:16 +01:00
Nicolas Vasilache 408553dd96 [mlir][Vector] Support 0-D vectors in `CreateMaskOp`
The 0-D case gets lowered in almost the same way that the 1-D case does
in VectorCreateMaskOpConversion. I also had to slightly update the
verifier for the op to always require exactly 1 operand in the 0-D case.

Depends On D115220

Reviewed by: ftynse

Differential revision: https://reviews.llvm.org/D115221
2021-12-12 13:32:29 +00:00
Michal Terepeta a0c930d312 [mlir][Vector] Support 0-D vectors in `CmpIOp`
Following the example of `VectorOfAnyRankOf`, I've done a few changes in the
`.td` files to help with adding the support for the 0-D case gradually.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115220
2021-12-12 13:28:26 +00:00
Jacques Pienaar feeee78afc [mlir] Flip dialects to _Prefixed
Following
https://llvm.discourse.group/t/psa-ods-generated-accessors-will-change-to-have-a-get-prefix-update-you-apis/4476
these have been flipped to both for ~4 weeks, flipping to _Prefixed.

Differential Revision: https://reviews.llvm.org/D115585
2021-12-11 14:21:20 -08:00
Lei Zhang 3ed47bcc96 [mlir][spirv] Propagate LogicalResult in (de)serialization
`(void)` was added when LogicalResult was marked as non
discard. This commit cleans them up to properly propagate
failures.

Reviewed By: scotttodd

Differential Revision: https://reviews.llvm.org/D115541
2021-12-10 19:20:49 -05:00
Lei Zhang 1bfa40a5d6 [mlir][spirv] Change default subgroup size
This should really come from a matching target environment. But
as a default, it can be handy (to avoid always listing the full
resource limits attribute in IR, etc.). It's common to see 32
so use that as the subgroup size.

Reviewed By: scotttodd

Differential Revision: https://reviews.llvm.org/D115534
2021-12-10 19:20:49 -05:00
Lei Zhang b289266cb2 [mlir][spirv] Add serialization control to emit symbol name
In SPIR-V, symbol names are encoded as `OpName` instructions.
They are not semantic impacting and can be omitted, which can
reduce the binary size.

Reviewed By: scotttodd

Differential Revision: https://reviews.llvm.org/D115531
2021-12-10 19:20:49 -05:00
Arjun P 98db55f108 [MLIR] IntegerPolyhedron: introduce getNumIdKind to replace calls to assertAtMostNumIdKind
Introduce a function `getNumIdKind` that returns the number of ids of the
specified kind. Remove the function `assertAtMostNumIdKind` and instead just
directly assert the inequality with a call to `getNumIdKind`.
2021-12-11 03:42:46 +05:30
Uday Bondhugula bc657b2eef [MLIR][NFC] Move out affine scalar replacement utility to affine utils
NFC. Move out and expose affine scalar replacement utility through
affine utils. Renaming misleading forwardStoreToLoad ->
affineScalarReplace. Update a stale doc comment.

Differential Revision: https://reviews.llvm.org/D115495
2021-12-11 03:26:42 +05:30
River Riddle 233e9476d8 [mlir:PDL] Allow non-bound pdl.attribute/pdl.type operations that create constants
This allows for passing in these attributes/types to constraints/rewrites as arguments.

Differential Revision: https://reviews.llvm.org/D114817
2021-12-10 19:38:43 +00:00
River Riddle 98f5bd3489 [mlir:PDL] Adjust the assembly format for AttributeOp to avoid conflicts with DictionaryAttr
Switch the attribute creation operations to use attr-dict-with-
keyword to avoid conflicts (in the case of pdl.attribute) and
confusion(in the case of pdl_interp.create_attribute) with
having a DictionaryAttr as a value and specifying the
attributes of the operation itself (as a dictionary).

Differential Revision: https://reviews.llvm.org/D114815
2021-12-10 19:38:42 +00:00
River Riddle 9debc35f02 [mlir:PDL] Fix assembly format for pdl.apply_native_rewrite
The results of a rewrite are optional, but we currently require
them to be present in the assembly format. This commit
makes the results component in the format optional.

Differential Revision: https://reviews.llvm.org/D114814
2021-12-10 19:38:42 +00:00
Mogball d658a4bb97 [mlir][ir] OpRewritePattern should accept generatedNames
Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D115514
2021-12-10 19:35:05 +00:00
Mogball 0845635eda [mlir][ir] Custom ops' parse/print fall back to dialect hooks
Custom ops that have no parser or printer should fall back to the dialect's parser and/or printer hooks. This avoids the need to define parsers and printers that simply dispatch to the dialect hook.

Reviewed By: mehdi_amini, rriddle

Differential Revision: https://reviews.llvm.org/D115481
2021-12-10 19:34:25 +00:00
Alexander Belyaev b618880e7b [mlir] Move `linalg.tensor_expand/collapse_shape` to TensorDialect.
RFC: https://llvm.discourse.group/t/rfc-reshape-ops-restructuring/3310

linalg.fill gets a canonicalizer, because `FoldFillWithTensorReshape` cannot be moved to tensorops (it uses linalg::FillOp inside). Before it was listed as a canonicalization pattern for the reshape operations, now it became a canonicalization for FillOp.

Differential Revision: https://reviews.llvm.org/D115502
2021-12-10 12:11:48 +01:00
Aart Bik 2bce0c1c7f [mlir][sparse] minor corrections and updates in sparse compiler doc
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D115467
2021-12-09 14:06:02 -08:00
Shraiysh Vaishay d4865393b5 [NFC][mlir][OpenMP] Added documentation for omp.atomic ops
This patch adds the documentation for the operations `omp.atomic.read`,
`omp.atomic.write` and `omp.atomic.update`.

Reviewed By: peixin

Differential Revision: https://reviews.llvm.org/D115445
2021-12-09 21:46:38 +05:30
Krzysztof Drewniak e1da62910e [MLIR][GPU] Define gpu.printf op and its lowerings
- Define a gpu.printf op, which can be lowered to any GPU printf() support (which is present in CUDA, HIP, and OpenCL). This op only supports constant format strings and scalar arguments
- Define the lowering of gpu.pirntf to a call to printf() (which is what is required for AMD GPUs when using OpenCL) as well as to the hostcall interface present in the AMD Open Compute device library, which is the interface present when kernels are running under HIP.
- Add a "runtime" enum that allows specifying which of the possible runtimes a ROCDL kernel will be executed under or that the runtime is unknown. This enum controls how gpu.printf is lowered

This change does not enable lowering for Nvidia GPUs, but such a lowering should be possible in principle.

And:
[MLIR][AMDGPU] Always set amdgpu-implicitarg-num-bytes=56 on kernels

This is something that Clang always sets on both OpenCL and HIP kernels, and failing to include it causes mysterious crashes with printf() support.

In addition, revert the max-flat-work-group-size to (1, 256) to avoid triggering bugs in the AMDGPU backend.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D110448
2021-12-09 15:54:31 +00:00
Groverkss 6f9afad6d3 [MLIR] Move Presburger Math from FlatAffineConstraints to Presburger/IntegerPolyhedron
This patch factors out math functionality that is a subset of Presburger arithmetic and moves it from FlatAffineConstraints to Presburger/IntegerPolyhedron. This patch only moves some parts of the functionality planned to be moved, with subsequent patches moving more functionality. There are three main reasons for this:

1. This split makes the Presburger Library easier and more flexible to use
    across MLIR, by not depending on IR.
2. This split allows the Presburger library to be developed independently from
    Affine Analysis, with Affine Analysis using this library.
3. With more functionality being upstreamed to the Presburger Library, the
    mlir/Analysis directory will be cluttered with Presburger library components
    since they depend on math functionality from FlatAffineConstraints. Moving this
    functionality to the Presburger directory allows keeping the new functionality
    in the Presburger directory.

This patch is part of an ongoing effort to make the Presburger Library easier to use. The motivation for this effort is the feedback received at the LLVM conference from Mehdi and others.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D114674
2021-12-09 16:42:06 +05:30
Michel Weber 45ea542dd8 [MLIR] Introduce coalesce for PresburgerSet
This patch provides functionality for simplifying `PresburgerSet`s by checking if any `FlatAffineConstraints` in the set is contained in another, and removing such redundant FACs.

This is part of a series of patches to provide functionality for [integer set coalescing](http://impact.gforge.inria.fr/impact2015/papers/impact2015-verdoolaege.pdf) in MLIR.

Reviewed By: arjunp

Differential Revision: https://reviews.llvm.org/D110617
2021-12-09 15:46:31 +05:30
Shraiysh Vaishay d82c1f4e4b [MLIR][OpenMP] Added omp.atomic.update
This patch supports the atomic construct (update) following section 2.17.7 of OpenMP 5.0 standard. Also added tests and verifier for the same.

Reviewed By: kiranchandramohan, peixin

Differential Revision: https://reviews.llvm.org/D112982
2021-12-09 15:21:24 +05:30
Rob Suderman 23149d522b [mlir] Added ctlz and cttz to math dialect and LLVM dialect
Count leading/trailing zeros are an existing LLVM intrinsic. Added LLVM
support for the intrinsics with lowerings from the math dialect to LLVM
dialect.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115206
2021-12-08 14:32:15 -08:00
Butygin d8fce785de [mlir][spirv] math.erf OpenCL lowering
Differential Revision: https://reviews.llvm.org/D115335
2021-12-08 21:59:46 +03:00
Thomas Raoux 579c1ff67d [mlir][nvvm] Add async copy ops to nvvm dialect
Differential Revision: https://reviews.llvm.org/D115314
2021-12-08 09:42:20 -08:00
Matthias Springer 847710f7b7 [mlir][linalg][bufferize] Add dialect filter to BufferizationOptions
This adds a new option `dialectFilter` to BufferizationOptions. Only ops from dialects that are allow-listed in the filter are bufferized. Other ops are left unbufferized. Note: This option requires `allowUnknownOps = true`.

To make use of `dialectFilter`, BufferizationOptions or BufferizationState must be passed to various helper functions.

The purpose of this change is to provide a better infrastructure for partial bufferization, which will be fully activated in a subsequent change.

Differential Revision: https://reviews.llvm.org/D114691
2021-12-08 23:51:18 +09:00
Mehdi Amini be0a7e9f27 Adjust "end namespace" comment in MLIR to match new agree'd coding style
See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differential Revision: https://reviews.llvm.org/D115309
2021-12-08 06:05:26 +00:00
Mehdi Amini ee0908703d Change the printing/parsing behavior for Attributes used in declarative assembly format
The new form of printing attribute in the declarative assembly is eliding the `#dialect.mnemonic` prefix to only keep the `<....>` part.

Differential Revision: https://reviews.llvm.org/D113873
2021-12-08 02:02:37 +00:00
Aart Bik e1b9d80532 [mlir][sparse] add a few more sparse output tests (for generated IR)
also fixes two typos in IR doc

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D115288
2021-12-07 15:31:29 -08:00
Aart Bik 4f2ec7f983 [mlir][sparse] finalize sparse output in the presence of reductions
This revision implements sparse outputs (from scratch) in all cases where
the loops can be reordered with all but one parallel loops outer. If the
inner parallel loop appears inside one or more reductions loops, then an
access pattern expansion is required (aka. workspaces in TACO speak).

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D115091
2021-12-07 10:54:29 -08:00
Lei Zhang 7709b23bef [mlir][scf] NFC: create dedicated files for affine utils
These functions are generic utility functions that operates on
affine ops within SCF regions. Moving them to their own files
for a better code structure, instead of mixing with loop
specialization logic.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D115245
2021-12-07 10:55:32 -05:00
Nicolas Vasilache 61ba9f9110 [mlir][Linalg] NFC - Extend the TilingInterface to allow better composition with out-of-tree dialects.
Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D115233
2021-12-07 13:06:27 +00:00
Matthias Springer 4ccbf1d2fb [mlir][linalg][bufferize] Fix forward declaration 2021-12-07 20:13:24 +09:00
Matthias Springer 958ae8b2d4 [mlir][linalg][bufferize] Bufferize Operation* instead of FuncOp
This change mainly changes the API. There is no mentioning of FuncOps in ComprehensiveBufferize anymore.

Also, bufferize methods of the op interface are called for ops without tensor operands/results if they have a region.

Differential Revision: https://reviews.llvm.org/D115212
2021-12-07 19:53:44 +09:00
not-jenni 5911a29aa9 [mlir][tosa] Add tosa.depthwise_conv2d as tosa.mul canonicalization
For a 1x1 weight and stride of 1, the input/weight can be reshaped and
multiplied elementwise then reshaped back

Reviewed By: rsuderman, KoolJBlack

Differential Revision: https://reviews.llvm.org/D115207
2021-12-06 17:28:52 -08:00
Matthias Springer 7ce427e3bc [mlir][linalg][bufferize][NFC] Clean up BufferizationState
Make fields private and clean up the interface. In particular, BufferizableOpInterface::bufferize no longer has access to `aliasInfo`. This was potentially dangerous because some of the ops registered in BufferizationAliasInfo may have been deleted.

Differential Revision: https://reviews.llvm.org/D114931
2021-12-07 10:05:39 +09:00
Rob Suderman 05e33d846f [mlir][tosa] Resubmit add tosa.conv2d as tosa.fully_connected canonicalization
Fixed the tosa.conv2d to tosa.fully_connected canonicalization for incorrect
output channels. Included uptes to tests to include checks for the result
shapes during canonicalization.

This allows conv2d to transform to the simpler fully_connected operation.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D115170
2021-12-06 15:33:07 -08:00
Rob Suderman c5fef77bc3 [mlir] Add CtPop to MathOps with lowering to LLVM
math.ctpop maths to the llvm.ctpop intrinsic.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D114998
2021-12-06 11:54:20 -08:00
Alex Zinenko d64b3e47ba [mlir] Avoid needlessly converting LLVM named structs with compatible elements
Conversion of LLVM named structs leads to them being renamed since we cannot
modify the body of the struct type once it is set. Previously, this applied to
all named struct types, even if their element types were not affected by the
conversion. Make this behvaior only applicable when element types are changed.
This requires making the LLVM dialect type-compatibility check recursively look
at the element types (arguably, it should have been doing than since the moment
the LLVM dialect type system stopped being closed). In addition, have a more
lax check for outer types only to avoid repeated check when necessary (e.g.,
parser, verifiers that are going to also look at the inner type).

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D115037
2021-12-06 13:42:11 +01:00
Matthias Springer e761c49a14 [mlir][linalg][bufferize][NFC] Utilize isWritable for FuncOps
This is a cleanup of ModuleBufferization. Instead of storing information about writable function arguments in BufferizationAliasInfo, we can use isWritable and make the decision there, based on dialect-specifc bufferization state.

Differential Revision: https://reviews.llvm.org/D114930
2021-12-06 18:36:54 +09:00
Matthias Springer e9fb4dc9e9 [mlir][linalg][bufferize] Remove buffer equivalence from bufferize
Remove all function calls related to buffer equivalence from bufferize implementations.

Add a new PostAnalysisStep for scf.for that ensures that yielded values are equivalent to the corresponding BBArgs. (This was previously checked in `bufferize`.) This will be relaxed in a subsequent commit.

Note: This commit changes two test cases. These were broken by design
and should not have passed. With the new scf.for PostAnalysisStep, this
bug was fixed.

Differential Revision: https://reviews.llvm.org/D114927
2021-12-06 17:48:31 +09:00
MaheshRavishankar 3ec6b1bfac [mlir] Add default implementations for methods in `TilingInterface`.
Adding the default implementation of `getLoopIteratorTypes` and
`getLoopBounds` allows ExternalModels to override these methods.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D115101
2021-12-06 08:35:55 +00:00
Matthias Springer cb4d0bf997 [mlir][linalg][bufferize][NFC] Collect equivalent FuncOp BBArgs in PostAnalysisStep
Collect equivalent BBArgs right after the equivalence analysis of the FuncOp and before bufferizing. This is in preparation of decoupling bufferization from aliasInfo.

Also gather equivalence info for CallOps, which was missing in the
previous commit.

Differential Revision: https://reviews.llvm.org/D114847
2021-12-06 17:31:39 +09:00
Michal Terepeta caf89c0db6 [mlir][Vector] Support 0-D vectors in `ConstantMaskOp`
To support creating both a mask with just a single `true` and `false` values,
I had to relax the restriction in the verifier that the rank is always equal to
the length of the attribute array, in other words, we now allow:

- `vector.constant_mask [0] : vector<i1>` which gets lowered to
  `arith.constant dense<false> : vector<i1>`
- `vector.constant_mask [1] : vector<i1>` which gets lowered to
  `arith.constant dense<true> : vector<i1>`

(the attribute list for the 0-D case must be a singleton containing
either `0` or `1`)

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D115023
2021-12-06 08:03:04 +00:00
Chia-hung Duan b8c6b15283 [mlir] Support collecting logs from notifyMatchFailure().
Let the user registers their own handler to processing the matching
failure information.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D110896
2021-12-04 04:35:24 +00:00
Matthias Springer 5fa0b3561a [mlir][linalg][bufferize] Implement equivalence analysis
Instead of checking buffer equivalence during bufferization, gather buffer equivalence information right after the analysis. This is in preparation of decoupling bufferization from BufferizationAliasInfo.

This change also fixes equivalence analysis for scf.if op results, which was not fully implemented. scf.if op results are equivalent to their corresponding yield values if both yield values are equivalent.

Differential Revision: https://reviews.llvm.org/D114774
2021-12-04 11:52:04 +09:00
Matthias Springer 6db200736c [mlir][linalg][bufferize][NFC] Use same OpBuilder throughout bufferization
Also set insertion point right before calling `bufferize`. No need to put an InsertionGuard anymore.

Differential Revision: https://reviews.llvm.org/D114928
2021-12-04 09:57:26 +09:00
natashaknk e2d8b60742 Revert "[mlir][tosa] Add tosa.conv2d as fully_connected canonicalization"
This reverts commit 13bdb7ab4a. The commit introduced/uncovered an unintended bug in models containing Conv2D.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D115079
2021-12-03 14:35:48 -08:00
Matthias Springer e359a1e548 [mlir][linalg][bufferize][NFC] Map only tensors in BufferizationState
BufferizationState had map/lookup overloads for non-tensor values. This was necessary for IREE. There is now a better way to do this, so these overloads can be removed.

Differential Revision: https://reviews.llvm.org/D114929
2021-12-03 23:07:09 +09:00
Matthias Springer ed8c63115e [mlir][linalg][bufferize][NFC] Provide default implementation of getAliasingOpOperand
This simplifies op interface implementations.

Differential Revision: https://reviews.llvm.org/D115025
2021-12-03 22:36:22 +09:00
Matthias Springer ad1ba42f68 [mlir][linalg][bufferize] Allow unbufferizable ops in input
Allow ops that are not bufferizable in the input IR. (Deactivated by default.)

bufferization::ToMemrefOp and bufferization::ToTensorOp are generated at the bufferization boundaries.

Differential Revision: https://reviews.llvm.org/D114669
2021-12-03 20:20:46 +09:00
Matthias Springer 867cd948ac [mlir][linalg][bufferize][NFC] Move BufferizationOptions to op interface
Also store a reference to BufferizationOptions in BufferizationState. This is in preparation of adding support for partial bufferization.

Differential Revision: https://reviews.llvm.org/D114661
2021-12-03 19:51:34 +09:00
Michal Terepeta 1423e8bf5d [mlir][Vector] Support 0-D vectors in `BitCastOp`
The implementation only allows to bit-cast between two 0-D vectors. We could
probably support casting from/to vectors like `vector<1xf32>`, but I wasn't
convinced that this would be important and it would require breaking the
invariant that `BitCastOp` works only on vectors with equal rank.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114854
2021-12-03 08:55:59 +00:00
Matthias Springer d30fcadf07 [mlir][linalg][bufferize] Op interface implementation for Bufferization dialect ops
This change provides `BufferizableOpInterface` implementations for ops from the Bufferization dialects. These ops are needed at the bufferization boundaries for partial bufferization.

Differential Revision: https://reviews.llvm.org/D114618
2021-12-03 16:25:44 +09:00
Ulysse Beaugnon e45705ad50 [MLIR] Use a shared uniquer for affine maps and integer sets.
Affine maps and integer sets previously relied on a single lock for creating unique instances. In a multi-threaded setting, this lock becomes a contention point. This commit updates AffineMap and IntegerSet to use StorageUniquer instead. StorageUniquer internally uses sharded locks and thread-local caches to reduce contention. It is already used for affine expressions, types and attributes. On my local machine, this gives me a 5X speedup for an application that manipulates a lot of affine maps and integer sets.

This commit also removes the integer set uniquer threshold. The threshold was used to avoid adding integer sets with a lot of constraints to the hash_map containing unique instances, but the constraints and the integer set were still allocated in the same allocator and never freed, thus not saving any space expect for the hash-map entry.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D114942
2021-12-02 23:49:32 +01:00
Groverkss d257f7c1bf [MLIR][FlatAffineConstraints] Remove duplicate divisions while merging local ids
This patch implements detecting duplicate local identifiers by extracting their
division representation while merging local identifiers.

For example, given the FACs A, B:

```
A: (x, y)[s0] : (exists d0 = [x / 4], d1 = [y / 4]: d0 <= s0, d1 <= s0, x + y >= 2)
B: (x, y)[s0] : (exists d0 = [x / 4], d1 = [y / 4]: d0 <= s0, d1 <= s0, x + y >= 5)
```

The intersection of A and B without this patch would lead to the following FAC:

```
(x, y)[s0] : (exists d0 = [x / 4], d1 = [y / 4], d2 = [x / 4], d3 = [x / 4]: d0 <= s0, d1 <= s0, d2 <= s0, d3 <= s0, x + y >= 2, x + y >= 5)
```

after this patch, merging of local ids will detect that `d0 = d2` and `d1 = d3`,
and the intersection of these two FACs will be (after removing duplicate constraints):

```
(x, y)[s0] : (exists d0 = [x / 4], d1 = [y / 4] : d0 <= s0, d1 <= s0, x + y >= 2, x + y >= 5)
```

This reduces the number of constraints by 2 (constraints) + 4 (2 constraints for each extra division) for this case.

This is used to reduce the output size representation of operations like
PresburgerSet::subtract, PresburgerSet::intersect which require merging local
variables.

Reviewed By: arjunp, bondhugula

Differential Revision: https://reviews.llvm.org/D112867
2021-12-03 03:44:47 +05:30
Groverkss cff427ee20 Revert changes that should have been sent as a patch
Revert changes that were meant to be sent as a single commit with
summary for the differential review, but were accidently sent directly.

This reverts commit 3bc5353fc6.
2021-12-03 03:42:37 +05:30
Groverkss c15724ab34 Address bondhugula's comments 2021-12-03 03:23:22 +05:30
Groverkss b912bf240e Fix doc comment for mergeLocalIds. 2021-12-03 03:23:21 +05:30
Groverkss 76ad74a4a9 Address more comments. 2021-12-03 03:23:21 +05:30
Groverkss 8a0967481f Address arjun's comments 2021-12-03 03:23:18 +05:30
Groverkss c9cea1909f Move division representation to a common function 2021-12-03 03:23:18 +05:30
Groverkss 985789ce0b Update mergeLocalIds docs 2021-12-03 03:23:17 +05:30
Groverkss 06a119a3bd Update docs for mergeLocalIds 2021-12-03 03:23:17 +05:30
Groverkss 3bc5353fc6 Implement division merging 2021-12-03 03:23:16 +05:30
Aart Bik 543924284f [mlir][bufferization] fixed typo in to_memref doc
Reviewed By: pifon2a

Differential Revision: https://reviews.llvm.org/D114824
2021-12-02 10:55:57 -08:00
Mogball ecaad4a876 [mlir][ods][nfc] fix gcc-5 build 2021-12-01 18:34:59 +00:00
Mogball ca6bd9cd43 [mlir][ods] AttrOrTypeGen uses Class
AttrOrType def generator uses `Class` code gen helper,
instead of naked raw_ostream.

Depends on D113714 and D114807

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D113715
2021-12-01 16:53:23 +00:00
Nicolas Vasilache c537a94334 [mlir][Vector] Thread 0-d vectors through vector.transfer ops
This revision adds 0-d vector support to vector.transfer ops.
In the process, numerous cleanups are applied, in particular around normalizing
and reducing the number of builders.

Reviewed By: ThomasRaoux, springerm

Differential Revision: https://reviews.llvm.org/D114803
2021-12-01 16:49:43 +00:00
Matthias Springer 2fd0ea960c [mlir][linalg][bufferize] CallOps do not bufferize to memory writes
However, since CallOps have no aliasing OpResults, their OpOperands always bufferize out-of-place.

This change removes `bufferizesToMemoryWrite` from `CallOpInterface`. This method was called, but its return value did not matter.

Differential Revision: https://reviews.llvm.org/D114616
2021-12-01 18:47:28 +09:00
Alexander Belyaev 3a6c4f307b [mlir] Add a helper for TiledLoopOp to get an operand tied to the bbArg.
Differential Revision: https://reviews.llvm.org/D114852
2021-12-01 09:32:00 +01:00
Jacques Pienaar 62fea88bc5 [mlir] Update accessors prefixed form (NFC) 2021-11-30 19:42:37 -08:00
Aart Bik 0e85232fa3 [mlir][sparse] refine simply dynamic sparse tensor outputs
Proper test for sparse tensor outputs is a single condition throughout
the whole tensor index expression (not a general conjunction, since this
may include other conditions that cause cancellation).

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D114810
2021-11-30 13:45:58 -08:00
Nicolas Vasilache a08b750ce9 [mlir][tensor] InsertSliceOp verification.
This revision reintroduces tensor.insert_slice verification which seems
to have vanished over time: a verifier was initially introduced in cf9503c1b7
but for some reason the invalid.mlir was not properly updated; as time passed the verifier was not called anymore and later the code was deleted.

As a consequence, a non-negligible portion of tests has run astray using invalid
tensor.insert_slice semantics and needed to be fixed.

Also, extract isRankReducedType from TensorOps for better reuse
Originally, this facility was used by both tensor and memref forms but
it got copied around as dialects were split.

Differential Revision: https://reviews.llvm.org/D114715
2021-11-30 20:37:06 +00:00
MaheshRavishankar 311dd55c9e [mlir][MemRef] Fix SubViewOp canonicalization when a subset of unit-dims are dropped.
The canonical type of the result of the `memref.subview` needs to make
sure that the previously dropped unit-dimensions are the ones dropped
for the canonicalized type as well. This means the generic
`inferRankReducedResultType` cannot be used. Instead the current
dropped dimensions need to be querried and the same need to be dropped.

Reviewed By: nicolasvasilache, ThomasRaoux

Differential Revision: https://reviews.llvm.org/D114751
2021-11-30 20:37:06 +00:00
not-jenni 13bdb7ab4a [mlir][tosa] Add tosa.conv2d as fully_connected canonicalization
For a 1x1 weight and stride of 1, the input/weight can be reshaped and passed into a fully connected op then reshaped back

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D114757
2021-11-30 12:01:14 -08:00
gysit c8f2139eb0 [mlir][linalg] Add decompose to CodegenStrategy.
Add the decompose patterns that lower higher dimensional convolutions to lower dimensional ones to CodegenStrategy and use CodegenStrategy to test the decompose patterns. Additionally, remove the assertion that checks the anchor op name is set in the CodegenStrategyTest pass. Removing the assertion allows us to simplify the pipelines used in the interchange and decompose tests.

Depends On D114797

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114798
2021-11-30 15:48:29 +00:00
gysit 98dbcff19c [mlir][linalg] Adapt the decompose patterns to use a filter (NFC).
The revision updates the convolution decomposition patterns to take a linalg transformation filter. The transformation filter in a later revision allows use the patterns from CodegenStrategy.

Depends On D114690

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114797
2021-11-30 15:46:10 +00:00
gysit 316e627c2b [mlir][linalg] Support the empty anchor op string when padding.
Add support for an empty anchor op string in vectorization. An empty anchor op string is useful after fusion when there are multiple different operations to vectorize.

Depends On D114689

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114690
2021-11-30 15:32:13 +00:00
Julian Gross ae1ea0bead [mlir] Decompose Bufferization Clone operation into Memref Alloc and Copy.
This patch introduces a new conversion to convert bufferization.clone operations
into a memref.alloc and a memref.copy operation. This transformation is needed to
transform all remaining clones which "survive" all previous transformations, before
a given program is lowered further (to LLVM e.g.). Otherwise, these operations
cannot be handled anymore and lead to compile errors.
See: https://llvm.discourse.group/t/bufferization-error-related-to-memref-clone/4665

Differential Revision: https://reviews.llvm.org/D114233
2021-11-30 10:15:56 +01:00
Alexander Belyaev f89bb3c012 [mlir] Move bufferization-related passes to `bufferization` dialect.
[RFC](https://llvm.discourse.group/t/rfc-dialect-for-bufferization-related-ops/4712)

Differential Revision: https://reviews.llvm.org/D114698
2021-11-30 09:58:47 +01:00
Stella Laurenzo bdc3183742 [mlir][python] Implement more SymbolTable methods.
* set_symbol_name, get_symbol_name, set_visibility, get_visibility, replace_all_symbol_uses, walk_symbol_tables
* In integrations I've been doing, I've been reaching for all of these to do both general IR manipulation and module merging.
* I don't love the replace_all_symbol_uses underlying APIs since they necessitate SYMBOL_COUNT walks and have various sharp edges. I'm hoping that whatever emerges eventually for this can still retain this simple API as a one-shot.

Differential Revision: https://reviews.llvm.org/D114687
2021-11-29 20:31:13 -08:00
Aart Bik 7d4da4e1ab [mlir][sparse] generalize sparse tensor output implementation
Moves sparse tensor output support forward by generalizing from injective
insertions only to include reductions. This revision accepts the case with all
parallel outer and all reduction inner loops, since that can be handled with
an injective insertion still. Next revision will allow the inner parallel loop
to move inward (but that will require "access pattern expansion" aka "workspace").

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D114399
2021-11-29 16:15:53 -08:00
Aart Bik 52668355f4 [mlir][sparse] some leftover cleanup from migration to bufferization dialect
Reviewed By: pifon2a

Differential Revision: https://reviews.llvm.org/D114730
2021-11-29 12:46:01 -08:00
Arnab Dutta c2280b5517 [MLIR] Avoid creation of buggy affine maps when incorrect values of number of dimensions and number of symbols are provided.
We check whether the maximum index of dimensional identifier present
in the result expressions is less than dimCount (number of dimensional
identifiers) argument passed in the AffineMap::get() and the maximum index
of symbolic identifier present in the result expressions is less than
symbolCount (number of symbolic identifiers) argument passed in AffineMap::get().

Reviewed By: nicolasvasilache, bondhugula

Differential Revision: https://reviews.llvm.org/D114238
2021-11-27 00:37:08 +05:30
Chris Jones 344eee6f38 [MLIR] Allow `Idempotent` trait to be applied to binary ops.
Add `Idempotent` trait to `arith.{andi,ori}`.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D114574
2021-11-26 18:22:49 +00:00
Michal Terepeta 7e65fc9a60 [mlir][Vector] Support 0-D vectors in `BroadcastOp`
This changes the op to produce `AnyVectorOfAnyRank` following mostly the code for 1-D vectors.

Depends On D114598

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114550
2021-11-26 17:17:18 +00:00
Michal Terepeta d0f927121e [mlir][Standard] Support 0-D vectors in `SplatOp`
This changes the op to produce `AnyVectorOfAnyRank` and implements this by just
inserting the element (skipping the shuffle that we do for the 1-D case).

Depends On D114549

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114598
2021-11-26 17:05:15 +00:00
Mats Petersson 30238c3676 [mlir][OpenMP] Add support for SIMD modifier
Add support for SIMD modifier in OpenMP worksharing loops.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D111051
2021-11-26 14:04:46 +00:00
Matthias Springer b62b21b980 [mlir][linalg][bufferize][NFC] InsertSliceOp no-copy detection as PostAnalysis
There is special logic for InsertSliceOp to check if a memcpy is needed. This change extracts that piece of code and makes it a PostAnalysisStep.

The purpose of this change is to untangle `bufferize` from BufferizationAliasInfo. (Not fully there yet.)

Differential Revision: https://reviews.llvm.org/D114513
2021-11-26 22:19:29 +09:00
Stanislav Funiak a76ee58f3c Multi-root PDL matching using upward traversals.
This is commit 4 of 4 for the multi-root matching in PDL, discussed in https://llvm.discourse.group/t/rfc-multi-root-pdl-patterns-for-kernel-matching/4148 (topic flagged for review).

This PR integrates the various components (root ordering algorithm, nondeterministic execution of PDL bytecode) to implement multi-root PDL matching. The main idea is for the pattern to specify mulitple candidate roots. The PDL-to-PDLInterp lowering selects one of these roots and "hangs" the pattern from this root, traversing the edges downwards (from operation to its operands) when possible and upwards (from values to its uses) when needed. The root is selected by invoking the optimal matching multiple times, once for each candidate root, and the connectors are determined form the optimal matching. The costs in the directed graph are equal to the number of upward edges that need to be traversed when connecting the given two candidate roots. It can be shown that, for this choice of the cost function, "hanging" the pattern an inner node is no better than from the optimal root.

The following three main additions were implemented as a part of this PR:
1. OperationPos predicate has been extended to allow tracing the operation accepting a value (the opposite of operation defining a value).
2. Predicate checking if two values are not equal - this is useful to ensure that we do not traverse the edge back downwards after we traversed it upwards.
3. Function for for building the cost graph among the candidate roots.
4. Updated buildPredicateList, building the predicates optimal branching has been determined.

Testing: unit tests (an integration test to follow once the stack of commits has landed)

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D108550
2021-11-26 18:11:37 +05:30
Stanislav Funiak 3eb1647af0 Introduced iterative bytecode execution.
This is commit 2 of 4 for the multi-root matching in PDL, discussed in https://llvm.discourse.group/t/rfc-multi-root-pdl-patterns-for-kernel-matching/4148 (topic flagged for review).

This commit implements the features needed for the execution of the new operations pdl_interp.get_accepting_ops, pdl_interp.choose_op:
1. The implementation of the generation and execution of the two ops.
2. The addition of Stack of bytecode positions within the ByteCodeExecutor. This is needed because in pdl_interp.choose_op, we iterate over the values returned by pdl_interp.get_accepting_ops until we reach finalize. When we reach finalize, we need to return back to the position marked in the stack.
3. The functionality to extend the lifetime of values that cross the nondeterministic choice. The existing bytecode generator allocates the values to memory positions by representing the liveness of values as a collection of disjoint intervals over the matcher positions. This is akin to register allocation, and substantially reduces the footprint of the bytecode executor. However, because with iterative operation pdl_interp.choose_op, execution "returns" back, so any values whose original liveness cross the nondeterminstic choice must have their lifetime executed until finalize.

Testing: pdl-bytecode.mlir test

Reviewed By: rriddle, Mogball

Differential Revision: https://reviews.llvm.org/D108547
2021-11-26 18:11:37 +05:30
Stanislav Funiak 842b6861c0 Defines new PDLInterp operations needed for multi-root matching in PDL.
This is commit 1 of 4 for the multi-root matching in PDL, discussed in https://llvm.discourse.group/t/rfc-multi-root-pdl-patterns-for-kernel-matching/4148 (topic flagged for review).

These operations are:
* pdl.get_accepting_ops: Returns a list of operations accepting the given value or a range of values at the specified position. Thus if there are two operations `%op1 = "foo"(%val)` and `%op2 = "bar"(%val)` accepting a value at position 0, `%ops = pdl_interp.get_accepting_ops of %val : !pdl.value at 0` will return both of them. This allows us to traverse upwards from a value to operations accepting the value.
* pdl.choose_op: Iteratively chooses one operation from a range of operations. Therefore, writing `%op = pdl_interp.choose_op from %ops` in the example above will select either `%op1`or `%op2`.

Testing: Added the corresponding test cases to mlir/test/Dialect/PDLInterp/ops.mlir.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D108543
2021-11-26 17:59:22 +05:30
Matthias Springer 8e2214aa60 [mlir][linalg][bufferize][NFC] Pass BufferizationState to PostAnalysisStep
Pass BufferizationStep instead of BufferizationAliasInfo. Note: BufferizationState contains BufferizationAliasInfo.

Differential Revision: https://reviews.llvm.org/D114512
2021-11-26 11:46:14 +09:00
Matthias Springer d62b4b08af [mlir][linalg][bufferize] Compose dialect-specific bufferization state
Use composition instead of inheritance for storing dialect-specific bufferization state. This is in preparation of adding "tensor dialect"-specific bufferization state.

Differential Revision: https://reviews.llvm.org/D114508
2021-11-26 11:35:45 +09:00
Matthias Springer c637e3ea9e [mlir][linalg][bufferize][NFC] Extract func boundary bufferization
Bufferization of function boundaries is extracted from ComprehensiveBufferize into a separate file. This will become its own build target in the future.

Differential Revision: https://reviews.llvm.org/D114226
2021-11-26 10:25:36 +09:00
Matthias Springer f32c3d9528 [mlir][linalg][bufferize][NFC] Move Affine interface impl to new build target
This makes ComprehensiveBufferize entirely independent of the Affine dialect.

Differential Revision: https://reviews.llvm.org/D114222
2021-11-26 09:27:47 +09:00
Mehdi Amini 850e8b4504 Fix link to the other docs from the Bufferization dialect 2021-11-26 00:13:32 +00:00
Tres Popp 6eca1957ee Don't store nullptrs in mlir::FuncOp::getAll*Attrs' result
These methods for results and arguments would create an ArrayRef full
of nullptrs when there were no argument attributes. This is problematic
because this result could not be passed to the FuncOp::build creator
without causing a segfault. Now the list will have empty attributes.

Differential Revision: https://reviews.llvm.org/D114358
2021-11-25 15:12:29 +01:00
seongwon bang 35c1e6ac1a [MLIR] [docs] Fix misguided examples in memref.subview operation.
The examples in `memref.subview` operation are misguided in that subview's strides operands mean "memref-rank number of strides that compose multiplicatively with the base memref strides in each dimension.".
So the below examples should be changed from `Strides: [64, 4, 1]` to `Strides: [1, 1, 1]`

Before changes
```
// Subview with constant offsets, sizes and strides.
%1 = memref.subview %0[0, 2, 0][4, 4, 4][64, 4, 1]
  : memref<8x16x4xf32, (d0, d1, d2) -> (d0 * 64 + d1 * 4 + d2)> to
    memref<4x4x4xf32, (d0, d1, d2) -> (d0 * 64 + d1 * 4 + d2 + 8)>
```

After changes
```
// Subview with constant offsets, sizes and strides.
%1 = memref.subview %0[0, 2, 0][4, 4, 4][1, 1, 1]
  : memref<8x16x4xf32, (d0, d1, d2) -> (d0 * 64 + d1 * 4 + d2)> to
    memref<4x4x4xf32, (d0, d1, d2) -> (d0 * 64 + d1 * 4 + d2 + 8)>
```

Also I fixed some syntax issues in docs related with memref layout map and added detailed explanation in subview rank reducing case.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D114500
2021-11-25 21:24:10 +09:00
Alexander Belyaev 57470abc41 [mlir] Move memref.[tensor_load|buffer_cast|clone] to "bufferization" dialect.
https://llvm.discourse.group/t/rfc-dialect-for-bufferization-related-ops/4712

Differential Revision: https://reviews.llvm.org/D114552
2021-11-25 11:50:39 +01:00
Matthias Springer 48107eaa07 [mlir][linalg][bufferize][NFC] Move SCF interface impl to new build target
This makes ComprehensiveBufferize entirely independent of the SCF dialect.

Differential Revision: https://reviews.llvm.org/D114221
2021-11-25 19:00:17 +09:00
Alexander Belyaev 3c228573bc Revert "[mlir][SCF] Further simplify affine maps during `for-loop-canonicalization`"
This reverts commit ee1bf18672.

It breaks IREE lowering. Reverting the commit for now while we
investigate what's going on.
2021-11-25 10:54:52 +01:00
Butygin 467acf3b6b [mlir][spirv] Float atomics should not imply Shader
Differential Revision: https://reviews.llvm.org/D114551
2021-11-25 12:07:28 +03:00
Matthias Springer a5c2f78287 [mlir][interfaces] Add insideMutuallyExclusiveRegions helper
Add a helper function to ControlFlowInterfaces for checking if two ops
are in mutually exclusive regions according to RegionBranchOpInterface.

Utilize this new helper in Linalg ComprehensiveBufferize. This makes the
analysis independent of the SCF dialect and generalizes it to other ops
that implement RegionBranchOpInterface.

Differential Revision: https://reviews.llvm.org/D114220
2021-11-25 17:44:39 +09:00
Matthias Springer ee1bf18672 [mlir][SCF] Further simplify affine maps during `for-loop-canonicalization`
* Implement `FlatAffineConstraints::getConstantBound(EQ)`.
* Inject a simpler constraint for loops that have at most 1 iteration.
* Taking into account constant EQ bounds of FlatAffineConstraint dims/symbols during canonicalization of the resulting affine map in `canonicalizeMinMaxOp`.

Differential Revision: https://reviews.llvm.org/D114138
2021-11-25 12:44:19 +09:00
Matthias Springer d3bb4fec2a [mlir][linalg][bufferize][NFC] Move arith interface impl to new build target
This makes ComprehensiveBufferize entirely independent of the arith dialect.

Differential Revision: https://reviews.llvm.org/D114219
2021-11-25 10:21:02 +09:00
Lei Zhang cb395f66ac [mlir][spirv] Change the return type for {Min|Max}VersionBase
For synthesizing an op's implementation of the generated interface
from {Min|Max}Version, we need to define an `initializer` and
`mergeAction`. The `initializer` specifies the initial version,
and `mergeAction` specifies how version specifications from
different parts of the op should be merged to generate a final
version requirements.

Previously we use the specified version enum as the type for both
the initializer and thus the final return type. This means we need
to perform `static_cast` over some hopefully not used number (`~0u`)
as the initializer. This is quite opaque and sort of not guaranteed
to work. Also, there are ops that have an enum attribute where some
values declare version requirements (e.g., enumerant `B` requires
v1.1+) but some not (e.g., enumerant `A` requires nothing). Then a
concrete op instance with `A` will still declare it implements the
version interface (because interface implementation is static for
an op) but actually theirs no requirements for version.

So this commit changes to use an more explicit `llvm::Optional`
to wrap around the returned version enum.  This should make it
more clear.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D108312
2021-11-24 17:33:01 -05:00
Tobias Gysi 86f186efea [mlir][linalg] Add makeComposedPadHighOp.
Add the makeComposedPadHighOp method which creates a new PadTensorOp if necessary. If the source to pad is actually the result of a sequence of padded LinalgOps, the method checks if padding is needed or if we can use the padded result of the padded LinalgOp sequence directly.

Example:
```
%0 = tensor.extract_slice %arg0 [%iv0, %iv1] [%sz0, %sz1]
%1 = linalg.pad_tensor %0 low[0, 0] high[...] { linalg.yield %cst }
%2 = linalg.matmul ins(...) outs(%1)
%3 = tensor.extract_slice %2 [0, 0] [%sz0, %sz1]
```
when padding %3 return %2 instead of introducing
```
%4 = linalg.pad_tensor %3 low[0, 0] high[...] { linalg.yield %cst }
```

Depends On D114161

Reviewed By: nicolasvasilache, pifon2a

Differential Revision: https://reviews.llvm.org/D114175
2021-11-24 19:18:59 +00:00
Florian Hahn fb46e64a01
Revert "[ThreadPool] Do not return shared futures."
This reverts commit a5fff58781.

The offending commit broke building with LLVM_ENABLE_THREADS=OFF.
2021-11-24 19:01:47 +00:00
MaheshRavishankar 0a58982b08 [mlir][Linalg] Remove alloc/dealloc pair as a callback.
The alloc dealloc pair generation callback is really central to the
bufferization algorithm, it modifies the state in a way that affects
correctness. This is not really a configurable option. Moving it to
BufferizationState removes what was probably the reason it was added
as a callback.

Differential Revision: https://reviews.llvm.org/D114417
2021-11-24 10:36:34 -08:00
Matthias Springer ca9d149e07 [mlir][linalg][bufferize][NFC] Move vector interface impl to new build target
This makes ComprehensiveBufferize entirely independent of the vector dialect.

Differential Revision: https://reviews.llvm.org/D114218
2021-11-24 19:36:12 +09:00
Matthias Springer bb273a35a0 [mlir][linalg][bufferize][NFC] Move tensor interface impl to new build target
This makes ComprehensiveBufferize entirely independent of the tensor dialect.

Differential Revision: https://reviews.llvm.org/D114217
2021-11-24 18:25:17 +09:00
Uday Bondhugula 8bd08a9fd7 [MLIR] Remove duplicate `Pass` suffix from ViewOpGraph class name
Remove duplicate `Pass` suffix from view-op-graph pass class name. The
extra suffix would lead to methods like registerViewOpGraphPassPass
being generated.

Differential Revision: https://reviews.llvm.org/D114459
2021-11-24 08:00:16 +05:30
Butygin 75a1bee05d [mlir][spirv] Add math to OpenCL conversion
Differential Revision: https://reviews.llvm.org/D113780
2021-11-24 02:31:21 +03:00
Rob Suderman 0f1e52afa9 [mlir][tosa] Materialize tosa.pad value and fold noop pads
Padding now can explicitly specify the padding value when non-zero is wanted.
This also includes bypassing pads when the pad does nothing.

Differential Revision: https://reviews.llvm.org/D113611
2021-11-23 12:23:42 -08:00
Rob Suderman 54eec7cafc [mlir][tosa] Separate tosa.transpose_conv decomposition and added stride support
Transpose convolution decomposition is now performed in a separate pass. This
allows padding / constant propagation to be performed at the TOSA level. It
also adds support for striding when there is no dilation.

Differential Revision: https://reviews.llvm.org/D114409
2021-11-23 12:16:44 -08:00
MaheshRavishankar b57e2f071a [mlir][Linalg] Add pad vectorization patterns into LinalgStrategyVectorize passes.
Add an option to control whether these patterns are added to the
pattern list or not.

Differential Revision: https://reviews.llvm.org/D114290
2021-11-23 11:47:54 -08:00
Nicolas Vasilache 3ff4e5f2a4 [mlir][Vector] Thread 0-d vectors through InsertElementOp.
This revision makes concrete use of 0-d vectors to extend the semantics of
InsertElementOp.

Reviewed By: dcaballe, pifon2a

Differential Revision: https://reviews.llvm.org/D114388
2021-11-23 12:55:11 +00:00
Nicolas Vasilache e7026aba00 [mlir][Vector] Thread 0-d vectors through ExtractElementOp.
This revision starts making concrete use of 0-d vectors to extend the semantics of
ExtractElementOp.
In the process a new VectorOfAnyRank Tablegen OpBase.td is added to allow progressive transition to supporting 0-d vectors by gradually opting in.

Differential Revision: https://reviews.llvm.org/D114387
2021-11-23 12:39:44 +00:00
Matthias Springer f24d9313cc [mlir][linalg][bufferize][NFC] Specify bufferize traversal in `bufferize`
The interface method `bufferize` controls how (and it what order) nested ops are traversed. This simplifies bufferization of scf::ForOps and scf::IfOps, which used to need special rules in scf::YieldOp.

Differential Revision: https://reviews.llvm.org/D114057
2021-11-23 21:33:19 +09:00
Florian Hahn a5fff58781
[ThreadPool] Do not return shared futures.
The only users of returned futures from ThreadPool is llvm-reduce after
D113857.

There should be no cases where multiple threads wait on the same future,
so there should be no need to return std::shared_future<>. Instead return
plain std::future<>.

If users need to share a future between multiple threads, they can share
the futures themselves.

Reviewed By: Meinersbur, mehdi_amini

Differential Revision: https://reviews.llvm.org/D114363
2021-11-23 10:06:08 +00:00
Alexander Belyaev c7cc70c8f8 Revert "Revert "[mlir] Move AllocationOpInterface to Bufferize/IR/AllocationOpInterface.td.""
This reverts and fixes commit de18b7dee6.
2021-11-23 10:49:26 +01:00
Nicolas Vasilache b2729fda60 [mlir][Vector] Add a vblendps-based impl for transpose8x8 (both intrin and inline_asm)
This revision follows up on the conversation titled:

```[llvm-dev] Understanding and controlling some of the AVX shuffle emission paths```

The revision adds a vblendps-based implementation for transpose8x8 and further distinguishes between and intrinsics and an inline_asm implementation.

This results in roughly 20% fewer cycles as reported by llvm-mca:

After this revision (intrinsic version, resolves to virtually identical assembly as per the llvm-dev discussion, no vblendps instruction is emitted):
```
Iterations:        100
Instructions:      5900
Total Cycles:      2415
Total uOps:        7300

Dispatch Width:    6
uOps Per Cycle:    3.02
IPC:               2.44
Block RThroughput: 24.0

Cycles with backend pressure increase [ 89.90% ]
Throughput Bottlenecks:
  Resource Pressure       [ 89.65% ]
  - SKXPort1  [ 0.04% ]
  - SKXPort2  [ 12.42% ]
  - SKXPort3  [ 12.42% ]
  - SKXPort5  [ 89.52% ]
  Data Dependencies:      [ 37.06% ]
  - Register Dependencies [ 37.06% ]
  - Memory Dependencies   [ 0.00% ]
```

After this revision (inline_asm version, vblendps instructions are indeed emitted):
```
Iterations:        100
Instructions:      6300
Total Cycles:      2015
Total uOps:        7700

Dispatch Width:    6
uOps Per Cycle:    3.82
IPC:               3.13
Block RThroughput: 20.0

Cycles with backend pressure increase [ 83.47% ]
Throughput Bottlenecks:
  Resource Pressure       [ 83.18% ]
  - SKXPort0  [ 14.49% ]
  - SKXPort1  [ 14.54% ]
  - SKXPort2  [ 19.70% ]
  - SKXPort3  [ 19.70% ]
  - SKXPort5  [ 83.03% ]
  - SKXPort6  [ 14.49% ]
  Data Dependencies:      [ 39.75% ]
  - Register Dependencies [ 39.75% ]
  - Memory Dependencies   [ 0.00% ]
```

An accessible copy of the conversation is available [here](https://gist.github.com/nicolasvasilache/68c7f34012584b0e00f335bcb374ede0).

Differential Revision: https://reviews.llvm.org/D114393
2021-11-23 07:31:22 +00:00
Sandeep Dasgupta e5a8c8c883 [mlir] Refactoring a few Parser APIs
Refactored two new parser APIs parseGenericOperationAfterOperands and
 parseCustomOperationName out of parseGenericOperation and parseCustomOperation.

Motivation: Sometimes an op can be printed in a special way if certain criteria
is met. While parsing, we need to handle all the versions.
`parseGenericOperationAfterOperands` is handy in situation where we already
parsed the operands and decide to fall back to default parsing.

`parseCustomOperationName` is useful when we need to know details (dialect,
operation name etc.) about a parsed token meant to be an mlir operation.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D113719
2021-11-23 06:11:01 +00:00
Matthias Springer 26c0dd83ab [mlir][linalg][bufferize][NFC] Move helper function to op interface
This is in preparation of changing the op traversal during bufferization.

Differential Revision: https://reviews.llvm.org/D114040
2021-11-23 11:59:47 +09:00
Matthias Springer 8d0994ed21 [mlir][linalg][bufferize][NFC] Remove special casing of CallOps
Differential Revision: https://reviews.llvm.org/D113966
2021-11-23 11:14:10 +09:00
Matthias Springer b1083830d6 [mlir][linalg][bufferize][NFC] Clean up headers and function visibility
Differential Revision: https://reviews.llvm.org/D113964
2021-11-23 10:29:26 +09:00
Christian Ulmann f6718fc6d3 [mlir] FlatAffineConstraint parsing for unit tests
This patch adds functionality to parse FlatAffineConstraints from a
StringRef with the intention to be used for unit tests. This should
make the construction of FlatAffineConstraints easier for testing
purposes.

The patch contains an example usage of the functionality in a unit test that
uses FlatAffineConstraints.

Reviewed By: bondhugula, grosser

Differential Revision: https://reviews.llvm.org/D113275
2021-11-23 03:04:30 +05:30
Alexander Belyaev de18b7dee6 Revert "[mlir] Move AllocationOpInterface to Bufferize/IR/AllocationOpInterface.td."
This reverts commit 3028bca6a9.
For some reason using FallbackModel works with CMake and does not work
with bazel. Using `ExternalModel` works. I will check what's going on
and resubmit tomorrow.
2021-11-22 21:35:20 +01:00
Alexander Belyaev 3028bca6a9 [mlir] Move AllocationOpInterface to Bufferize/IR/AllocationOpInterface.td.
Remove the interface from op defs in MemRefOps.td and make it an external model.

This is the first PR of many that will move bufferization-related ops, interfaces, passes to Dialect/Bufferize.
RFC: https://llvm.discourse.group/t/rfc-dialect-for-bufferization-related-ops/4712
It is still debated if the comprehensive bufferization has to be moved there as well, so for now I am just moving the "gradual" bufferization.

Differential Revision: https://reviews.llvm.org/D114147
2021-11-22 21:00:59 +01:00
Mehdi Amini e0b7bee7cf Revert "[mlir][Vector] Add a vblendps-based impl for transpose8x8 (both intrin and inline_asm)"
This reverts commit a9e236bed8.
This broke the Windows build:

mlir\include\mlir/Dialect/X86Vector/Transforms.h(28): error C2061: syntax error: identifier 'uint'
2021-11-22 19:23:18 +00:00
Alex Zinenko 9c5982ef8e [mlir] support recursive types in type conversion infra
MLIR supports recursive types but they could not be handled by the conversion
infrastructure directly as it would result in infinite recursion in
`convertType` for elemental types. Support this case by keeping the "call
stack" of nested type conversions in the TypeConverter class and by passing it
as an optional argument to the individual conversion callback. The callback can
then check if a specific type is present on the stack more than once to detect
and handle the recursive case.

This approach is preferred to the alternative approach of having a separate
callback dedicated to handling only the recursive case as the latter was
observed to introduce ~3% time overhead on a 50MB IR file even if it did not
contain recursive types.

This approach is also preferred to keeping a local stack in type converters
that need to handle recursive types as that would compose poorly in case of
out-of-tree or cross-project extensions.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D113579
2021-11-22 18:16:02 +01:00
Tres Popp 106f307499 Rename MlirExecutionEngine lookup to lookupPacked
The purpose of the change is to make clear whether the user is
retrieving the original function or the wrapper function, in line with
the invoke commands. This new functionality is useful for users that
already have defined their own packed interface, so they do not want the
extra layer of indirection, or for users wanting to the look at the
resulting primary function rather than the wrapper function.

All locations, except the python bindings now have a `lookupPacked`
method that matches the original `lookup` functionality. `lookup`
still exists, but with new semantics.

- `lookup` returns the function with a given name. If `bool f(int,int)`
is compiled, `lookup` will return a reference to `bool(*f)(int,int)`.
- `lookupPacked` returns the packed wrapper of the function with the
given name. If `bool f(int,int)` is compiled, `lookupPacked` will return
`void(*mlir_f)(void**)`.

Differential Revision: https://reviews.llvm.org/D114352
2021-11-22 14:12:09 +01:00
Tobias Gysi f7751a3a42 [mlir][linalg] Remove tile and fuse test pass (NFC).
Remove the tile and fuse test pass that has been replaced by codegen strategy.

Depends On D114067

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114068
2021-11-22 12:33:31 +00:00
Tobias Gysi e3d386ea27 [mlir][linalg] Add a tile and fuse on tensors pattern.
Add a pattern to apply the new tile and fuse on tensors method. Integrate the pattern into the CodegenStrategy and use the CodegenStrategy to implement the tests.

Depends On D114012

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D114067
2021-11-22 11:13:21 +00:00