Commit Graph

6003 Commits

Author SHA1 Message Date
River Riddle 5de12bb703 [mlir][Tablegen-LSP] Add support for a basic TableGen language server
This follows the same general structure of the MLIR and PDLL language
servers. This commits adds the basic functionality for setting up the server,
and initially only supports providing diagnostics. Followon commits will
build out more comprehensive behavior.

Realistically this should eventually live in llvm/, but building in MLIR is an easier
initial step given that:
* All of the necessary LSP functionality is already here
* It allows for proving out useful language features (e.g. compilation databases)
  without affecting wider scale tablegen users
* MLIR has a vscode extension that can immediately take advantage of it

Differential Revision: https://reviews.llvm.org/D125440
2022-05-16 16:03:51 -07:00
wren romano 8cb332406c [mlir][sparse] Enhancing sparse=>sparse conversion.
Fixes: https://github.com/llvm/llvm-project/issues/51652

Depends On D122060

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D122061
2022-05-16 15:42:19 -07:00
River Riddle e0c3b94c80 [mlir] Restrict dialect doc gen to a single dialect
In the overwhelmingly majority of cases only one dialect is generated at a time
anyways, and this restriction more easily catches user error when multiple
dialects might be generated. We hit this semi-recently with the PDL dialect,
and circt+other downstream users are also actively hitting this as well.

Differential Revision: https://reviews.llvm.org/D125651
2022-05-16 15:35:07 -07:00
Matthias Springer 0b293bf045 [mlir][bufferize] Better propagation of errors
Return immediately when an op bufferization patterns fails.

Differential Revision: https://reviews.llvm.org/D125087
2022-05-16 23:17:01 +02:00
Mogball 67f0e8eec3 [mlir][ods] Fix verification of attribute + colon type ambiguity
An attribute without a type builder followed by a colon in an assembly format is potentially ambiguous because the parser will read ahead to parse the colon-type and pass this as the type argument to the attribute's constructor.

However, the previous verifier that checks for this ambiguity erroneously produces an error in the case of

```
let assemblyFormat = "( `(` $attr `)` )? `:`";
```

This patch fixes the bug by implementing a checker that correctly handles all edge cases, including very strange assembly formats like:

```
let assemblyFormat = "( `(` $attr ) : (`>`)? attr-dict (`>` $a^) : (`<`)? `:`";
```

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D125445
2022-05-16 21:15:27 +00:00
River Riddle a6cef03f66 [mlir] Remove the `type` keyword from type alias definitions
This was carry over from LLVM IR where the alias definition can
be ambiguous, but MLIR type aliases have no such problems.
Having the `type` keyword is superfluous and doesn't add anything.
This commit drops it, which also nicely aligns with the syntax for
attribute aliases (which doesn't have a keyword).

Differential Revision: https://reviews.llvm.org/D125501
2022-05-16 13:54:02 -07:00
Mogball c8457eb532 [mlir][transforms] Add a topological sort utility and pass
This patch adds a topological sort utility and pass. A topological sort reorders
the operations in a block without SSA dominance such that, as much as possible,
users of values come after their producers.

The utility function sorts topologically the operation range in a given block
with an optional user-provided callback that can be used to virtually break cycles.
The toposort pass itself recursively sorts graph regions under the target op.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D125063
2022-05-16 20:47:30 +00:00
Mogball 0533253d81 [mlir][ods] Ignore AttributeSelfTypeParameter in assembly formats
The attribute self type parameter is currently treated like any other attribute parameter in the assembly format. The self type parameter should be handled by the operation parser and printer and play no role in the generated parsers and printers of attributes.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D125724
2022-05-16 20:23:54 +00:00
Aart Bik 736c1b66ef [mlir][sparse] introduce complex type to sparse tensor support
This is the first implementation of complex (f64 and f32) support
in the sparse compiler, with complex add/mul as first operations.
Note that various features are still TBD, such as other ops, and
reading in complex values from file. Also, note that the
std::complex<float> had a bit of an ABI issue when passed as
single argument. It is still TBD if better solutions are possible.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D125596
2022-05-16 13:17:36 -07:00
Robert Suderman cb4a5eae1e [mlir][tosa] Use math.ctlz intrinsic for tosa.clz
We were custom counting per bit for the clz instruction. Math dialect
now has an intrinsic to do this in one instruction. Migrated to this
instruction and fixed a minor bug math-to-llvm for the intrinsic.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D125592
2022-05-16 11:31:35 -07:00
Matthias Springer f287da8a15 [mlir][bufferize] Better user control of layout maps
This changes replaces the `fully-dynamic-layout-maps` options (which was badly named) with two new options:

* `unknown-type-conversion` controls the layout maps on buffer types for which no layout map can be inferred.
* `function-boundary-type-conversion` controls the layout maps on buffer types inside of function signatures.

Differential Revision: https://reviews.llvm.org/D125615
2022-05-16 18:06:13 +02:00
Min-Yih Hsu 3da65c4c0b [mlir][LLVMIR] Add support for translating shufflevector
Add support for translating llvm::ShuffleVectorInst

Differential Revision: https://reviews.llvm.org/D125030
2022-05-14 15:14:40 -07:00
Min-Yih Hsu b8f52c08f8 [mlir][LLVMIR] Add support for translating insert/extractvalue
Add support for translating llvm::InsertValue and llvm::ExtractValue.

Differential Revision: https://reviews.llvm.org/D125028
2022-05-14 15:14:40 -07:00
Arnab Dutta 16219f8c94 [MLIR][GPU] Add canonicalizer for gpu.memcpy
Erase gpu.memcpy op when only uses of dest are
the memcpy op in question, its allocation and deallocation
ops.

Reviewed By: bondhugula, csigg

Differential Revision: https://reviews.llvm.org/D124257
2022-05-14 19:01:04 +05:30
Chris Lattner 5ac9d66209 [DenseElementsAttr] Teach isValidRawBuffer that 1-elt values are splats.
We want getRaw() on tensors with i1 element type with a zero or 1 value
to be treated as a splat.  This fixes:
https://github.com/llvm/llvm-project/issues/55440
2022-05-14 11:49:43 +01:00
Mogball 70b69c54fa [mlir] Rename Zero* traits to Zero*s
Rename
ZeroResult -> ZeroResults
ZeroSuccessor -> ZeroSuccessors
ZeroRegion -> ZeroRegions

to be in line with ZeroOperands and grammatically correct.
2022-05-14 00:20:28 +00:00
Chris Lattner 27478872fd [ParseResult] Fix warning in flang build, incorporate feedback from River.
The warning caused build errors on a couple flang testers that are
building with -Werror.  The diagnostic change makes the generated
error correct.

This is a followup to https://reviews.llvm.org/D125549

Differential Revision: https://reviews.llvm.org/D125587
2022-05-13 23:30:27 +01:00
Chris Lattner 1d7b5cd5bf [ParseResult] Mark this as LLVM_NODISCARD (like LogicalResult) and fix issues.
There are a lot of cases where we accidentally ignored the result of some
parsing hook.  Mark ParseResult as LLVM_NODISCARD just like ParseResult is.
This exposed some stuff to clean up, so do.

Differential Revision: https://reviews.llvm.org/D125549
2022-05-13 16:28:53 +01:00
Tres Popp 1dce51b888 [mlir] Add TensorToLinalgPass
This pass is to handle computationally complex operations like
tensor.pad which are not simply lowered to the exact same operation in
the memref dialect.

Differential Revision: https://reviews.llvm.org/D125384
2022-05-13 12:17:22 +02:00
Aart Bik 6f3c7dfb77 [mlir][sparse] add sparse sign integration test
Implements a floating-point sign operator (using the new semi-ring ops)
that accomodates +/-Inf and +/-NaN in consistent way.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D125494
2022-05-12 15:56:36 -07:00
River Riddle 86e1c2f097 [mlir] Fix pipeline-parsing.mlir on windows
We shouldn't be making assumptions about the result of llvm::getTypeName,
which may have different results for anonymous namespaces depending
on the platform.
2022-05-12 13:40:16 -07:00
River Riddle c2fb9c29b4 [mlir:Pass] Add support for op-agnostic pass managers
This commit refactors the current pass manager support to allow for
operation agnostic pass managers. This allows for a series of passes
to be executed on any viable pass manager root operation, instead
of one specific operation type. Op-agnostic/generic pass managers
only allow for adding op-agnostic passes.

These types of pass managers are extremely useful when constructing
pass pipelines that can apply to many different types of operations,
e.g., the default inliner simplification pipeline. With the advent of
interface/trait passes, this support can be used to define FunctionOpInterface
pass managers, or other pass managers that effectively operate on
specific interfaces/traits/etc (see #52916 for an example).

Differential Revision: https://reviews.llvm.org/D123536
2022-05-12 13:12:59 -07:00
Ashay Rane 5380e30e04
[mlir] translate memref.reshape ops that have static shapes
This patch references code for translating memref.reinterpret_cast ops
to add translation rules for memref.reshape ops that have a static shape
argument.  Since reshape ops don't have offsets, sizes, or strides, this
patch simply sets the allocated and aligned pointers of the MemRef
descriptor.

Reviewed By: ftynse, cathyzhyi

Differential Revision: https://reviews.llvm.org/D125039
2022-05-12 11:57:20 -07:00
Thomas Raoux d02f10d96d [mlir][vector] Add lowering pattern for vector.warp_execute_on_lane_0 op
Add lowering of the vector.warp_execute_on_lane_0 into scf.if plus memory
transfer for the operands and yield values.

This also add an integration test running on GPU warp. The same tests can be
later re-used with different comment lines to tests distribution
transformations.

This is mostly from @springerm contribution.

Differential Revision: https://reviews.llvm.org/D125430
2022-05-12 13:27:43 +00:00
Benjamin Kramer 303638248a [mlir][linalg] Add lowering of named ops on complex numbers
This lets linalg.dot and friends lower to a complex muladd using ops
from the complex dialect.

Differential Revision: https://reviews.llvm.org/D125461
2022-05-12 13:37:34 +02:00
Benjamin Kramer 27dad99622 [mlir][LLVM] Make the nested type restriction on complex constants less aggressive
Complex nested in other types is perfectly fine, just nested structs
aren't supported. Instead of checking whether there's nesting just check
whether the struct we're dealing with is a complex number.

Differential Revision: https://reviews.llvm.org/D125381
2022-05-12 11:47:01 +02:00
Matthias Springer 82ea0d8b82 [mlir][bufferize] Support alloc hoisting across function boundaries
This change integrates the BufferResultsToOutParamsPass into One-Shot Module Bufferization. This improves memory management (deallocation) when buffers are returned from a function.

Note: This currently only works with statically-sized tensors. The generated code is not very efficient yet and there are opportunities for improvment (fewer copies). By default, this new functionality is deactivated.

Differential Revision: https://reviews.llvm.org/D125376
2022-05-12 09:44:07 +02:00
Matthias Springer 2fe40c34ea [mlir][bufferize] Fix op filter
Bufferization has an optional filter to exclude certain ops from analysis+bufferization. There were a few remaining places in the codebase where the filter was not checked.

Differential Revision: https://reviews.llvm.org/D125356
2022-05-12 09:33:07 +02:00
River Riddle 1155c1fe65 [mlir:Parser] Emit a better diagnostic when a custom operation is unknown
When a custom operation is unknown and does not have a dialect prefix, we currently
emit an error using the name of the operation with the default dialect prefix. This
leads to a confusing error message, especially when operations get moved between dialects.
For example, `func` was recently moved out of `builtin` and to the `func` dialect. The current
error message we get is:

```
func @foo()
^ custom op 'builtin.func' is unknown
```

This could lead users to believe that there is supposed to be a `builtin.func`,
because there used to be. This commit adds a better error message that does
not assume that the operation is supposed to be in the default dialect:

```
func @foo()
^ custom op 'func' is unknown (tried 'builtin.func' as well)
```

Differential Revision: https://reviews.llvm.org/D125351
2022-05-11 22:54:44 -07:00
Mahesh Ravishankar 8be7e6f56a [mlir][Linalg] Combine canonicalizers that deal with removing dead/redundant args.
`linalg.generic` ops have canonicalizers that either remove arguments
not used in the payload, or redundant arguments. Combine these and
enhance the canonicalization to also remove results that have no use.
This is effectively dead code elimination for Linalg ops.

Differential Revision: https://reviews.llvm.org/D123632
2022-05-12 05:22:30 +00:00
Mogball 19906262c9 [mlir] (NFC) Use assembly format for test.graph_region 2022-05-12 04:19:25 +00:00
grosul1 a4b227c28a [mlir] Fix loop unrolling: properly replace the arguments of the epilogue loop.
Using "replaceUsesOfWith" is incorrect because the same initializer value may appear multiple times.

For example, if the epilogue is needed when this loop is unrolled
```
%x:2 = scf.for ... iter_args(%arg1 = %c1, %arg2 = %c1) {
  ...
}
```
then both epilogue's arguments will be incorrectly renamed to use the same result index (note #1 in both cases):
```
%x_unrolled:2 = scf.for ... iter_args(%arg1 = %c1, %arg2 = %c1) {
  ...
}
%x_epilogue:2 = scf.for ... iter_args(%arg1 = %x_unrolled#1, %arg2 = %x_unrolled#1) {
  ...
}
```
2022-05-12 01:54:39 +00:00
Chris Lattner 86445e8c63 [AsmParser] Adopt emitWrongTokenError more, improving QoI
This is a full audit of emitError calls, I took the opportunity
to remove extranous parens and fix a couple cases where we'd
generate multiple diagnostics for the same error.

Differential Revision: https://reviews.llvm.org/D125355
2022-05-11 20:41:12 +01:00
Chris Lattner 34b6f206cb [AsmParser] Improve error recovery again.
Change the parsing logic to use StringRef instead of lower level
char* logic.  Also, if emitting a diagnostic on the first token
in the file, we make sure to use that position instead of the
very start of the file.

Differential Revision: https://reviews.llvm.org/D125353
2022-05-11 08:25:36 +01:00
Thomas Raoux 15bcc36eed [mlir][gpu] Move async copy ops to NVGPU and add caching hints
Move async copy operations to NVGPU as they only exist on NV target and are
designed to match ptx semantic. This allows us to also add more fine grain
caching hint attribute to the op.
Add hint to bypass L1 and hook it up to NVVM op.

Differential Revision: https://reviews.llvm.org/D125244
2022-05-10 22:30:24 +00:00
Nicolas Vasilache 1f23211cb1 [mlir][SCF] Retire `cloneWithNewYields` helper function.
This is now subsumed by `replaceLoopWithNewYields`.

Differential Revision: https://reviews.llvm.org/D125309
2022-05-10 18:44:11 +00:00
Mahesh Ravishankar 567fd523bf [mlir][SCF] Add utility method to add new yield values to a loop.
The current implementation of `cloneWithNewYields` has a few issues
- It clones the loop body of the original loop to create a new
  loop. This is very expensive.
- It performs `erase` operations which are incompatible when this
  method is called from within a pattern rewrite. All erases need to
  go through `PatternRewriter`.

To address these a new utility method `replaceLoopWithNewYields` is added
which
- moves the operations from the original loop into the new loop.
- replaces all uses of the original loop with the corresponding
  results of the new loop
- use a call back to allow caller to generate the new yield values.
- the original loop is modified to just yield the basic block
  arguments corresponding to the iter_args of the loop. This
  represents a no-op loop. The loop itself is dead (since all its uses
  are replaced), but is not removed. The caller is expected to erase
  the op. Consequently, this method can be called from within a
  `matchAndRewrite` method of a `PatternRewriter`.

The `cloneWithNewYields` could be replaces with
`replaceLoopWithNewYields`, but that seems to trigger a failure during
walks, potentially due to the operations being moved. That is left as
a TODO.

Differential Revision: https://reviews.llvm.org/D125147
2022-05-10 18:44:11 +00:00
Krzysztof Drewniak f1f05a91ca [MLIR][AMDGPU] Add AMDGPU dialect, wrappers around raw buffer intrinsics
By analogy with the NVGPU dialect, introduce an AMDGPU dialect for
AMD-specific intrinsic wrappers.

The dialect initially includes wrappers around the raw buffer intrinsics.

On AMD GPUs, a memref can be converted to a "buffer descriptor" that
allows more precise control of memory access, such as by allowing for
out of bounds loads/stores to be replaced by 0/ignored without adding
additional conditional logic, which is important for performance.

The repository currently contains a limited conversion from
transfer_read/transfer_write to Mubuf intrinsics, which are an older,
deprecated intrinsic for the same functionality.

The new amdgpu.raw_buffer_* ops allow these operations to be used
explicitly and for including metadata such as whether the target
chipset is an RDNA chip or not (which impacts the interpretation of
some bits in the buffer descriptor), while still maintaining an
MLIR-like interface.

(This change also exposes the floating-point atomic add intrinsic.)

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D122765
2022-05-10 14:59:58 +00:00
Chris Lattner ad3b358180 [MLIR Parser] Improve QoI for "expected token" errors
A typical problem with missing a token is that the missing
token is at the end of a line.  The problem with this is that
the error message gets reported on the start of the following
line (which is where the next / invalid token is) which can
be confusing.

Handle this by noticing this case and backing up to the end of
the previous line.

Differential Revision: https://reviews.llvm.org/D125295
2022-05-10 15:44:17 +01:00
Thomas Raoux 09fc685ce6 [mlir][nvvm] Add attribute to nvvm.cpAsyncOp to control l1 bypass
Add attribute to be able to generate the intrinsic version of async copy
generating a copy with l1 bypass. This correspond to
cp.async.cg.shared.global in ptx.

Differential Revision: https://reviews.llvm.org/D125241
2022-05-09 19:34:48 +00:00
Stella Stamenova 057863a9bc [mlir] Fix build & test of mlir python bindings on Windows
There are a couple of issues with the python bindings on Windows:
- `create_symlink` requires special permissions on Windows - using `copy_if_different` instead allows the build to complete and then be usable
- the path to the `python_executable` is likely to contain spaces if python is installed in Program Files. llvm's python substitution adds extra quotes in order to account for this case, but mlir's own python substitution does not
- the location of the shared libraries is different on windows
- if the type is not specified for numpy arrays, they appear to be treated as strings

I've implemented the smallest possible changes for each of these in the patch, but I would actually prefer a slightly more comprehensive fix for the python_executable and the shared libraries.

For the python substitution, I think it makes sense to leverage the existing %python instead of adding %PYTHON and instead add a new variable for the case when preloading is needed. This would also make it clearer which tests are which and should be skipped on platforms where the preloading won't work.

For the shared libraries, I think it would make sense to pass the correct path and extension (possibly even the names) to the python script since these are known by lit and don't have to be hardcoded in the test at all.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D125122
2022-05-09 11:10:20 -07:00
Jerry Wu ad7c49bef7 [mlir][linalg] Fix padding size calculation for Conv2d ops.
This patch fixed the padding size calculation for Conv2d ops when the stride > 1. It contains the changes below:

- Use addBound to add constraint for AffineApplyOp in getUpperBoundForIndex. So the result value can be mapped and retrieved later.

- Fixed the bound from AffineMinOp by adding as a closed bound. Originally the bound was added as an open upper bound, which results in the incorrect bounds when we multiply the values. For example:

```
%0 = affine.min affine_map<()[s0] -> (4, -s0 + 11)>()[iv0]
%1 = affine.apply affine_map<()[s0] -> (s0 * 2)>()[%0]

If we add the affine.min as an open bound, addBound will internally transform it into the close bound "%0 <= 3". The following sliceBounds will derive the bound of %1 as "%1 <= 6" and return the open bound "%1 < 7", while the correct bound should be "%1 <= 8".
```

- In addition to addBound, I also changed sliceBounds to support returning closed upper bound, since for the size computation, we usually care about the closed bounds.

- Change the getUpperBoundForIndex to favor constant bounds when required. The sliceBounds will return a tighter but non-constant bounds, which can't be used for padding. The constantRequired option requires getUpperBoundForIndex to get the constant bounds when possible.

Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D124821
2022-05-09 08:45:37 -07:00
Ashay Rane e287d647c6 [mlir] Add translation from tensor.reshape to memref.reshape
This patch augments the `tensor-bufferize` pass by adding a conversion
rule to translate ReshapeOp from the `tensor` dialect to the `memref`
dialect, in addition to adding a unit test to validate the translation.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D125031
2022-05-09 17:45:07 +02:00
Benjamin Kramer a48adc5658 [mlir][math] Promote (b)f16 to f32 when lowering to libm calls
libm doesn't have overloads for the small types, so promote them to a
bigger type and use the f32 function.

Differential Revision: https://reviews.llvm.org/D125093
2022-05-09 11:59:55 +02:00
Christopher Bate 9879807393 [mlir][NvGpu] Fix nvgpu.mma.sync lowering to NVVM for f32, tf32 types
Adds missing logic in the lowering from NvGPU to NVVM to support fp32
(in an accumulator operand) and tf32 (in multiplicand operand) types.
Fixes logic in one of the helper functions for converting the result
of a mma.sync operation with multiple 8x256bit output tiles, which is
the case for f32 outputs.

Differential Revision: https://reviews.llvm.org/D124533
2022-05-08 21:49:42 -06:00
Mehdi Amini 25cd6fba98 Fix MLIR integration test after a8308020 (`func.` prefix is required bythe parser now) 2022-05-07 09:09:24 +00:00
River Riddle a8308020ac [mlir] Remove special case parsing/printing of `func` operations
This was leftover from when the standard dialect was destroyed, and
when FuncOp moved to the func dialect. Now that these transitions
have settled a bit we can drop these.

Most updates were handled using a simple regex: replace `^( *)func` with `$1func.func`

Differential Revision: https://reviews.llvm.org/D124146
2022-05-06 13:36:15 -07:00
Mehdi Amini b37d158f71 Apply clang-tidy fixes for bugprone-copy-constructor-init in TestPassManager.cpp (NFC) 2022-05-06 20:19:19 +00:00
Aart Bik 5b122a7310 [mlir][sparse] integration test for zero preserving math op
Also fixes omission in lowering math ops that require lib support

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D125104
2022-05-06 10:42:33 -07:00
Matthias Springer 988748c077 [mlir][bufferize] Do not copy buffers with undefined contents
Buffers with undefined contents (e.g., the result of an init_tensor) are no longer copied.

Differential Revision: https://reviews.llvm.org/D125015
2022-05-06 17:31:01 +09:00
Matthias Springer a5d09c6372 [mlir][scf] Implement BufferizableOpInterface for scf::WhileOp
This follows the same implementation strategy as scf::ForOp and common functionality is extracted into helper functions.

This implementation works well in cases where each yielded value (from either body/condition region) is equivalent to the corresponding bbArg of the parent block. In that case, each OpResult of the loop may be aliasing with the corresponding OpOperand of the loop (and with no other OpOperand).

In the absence of said equivalence relationship, new buffer copies must be inserted, so that the aliasing OpOperand/OpResult contract of scf::WhileOp is honored. In essence, by yielding a newly allocated buffer, we can enforce the specified may-alias relationship. (Newly allocated buffers cannot alias with any OpOperands of the loop.)

Differential Revision: https://reviews.llvm.org/D124929
2022-05-06 17:24:33 +09:00
Chengji Yao e5a4cf6743 [mlir] Fix printer when it is a DenseElementsAttr of i1
A large DenseElementsAttr of i1could trigger a bug in printer/parser roundtrip.

Ex. A DenseElementsAttr of i1 with 200 elements will print as Hex format of length 400 before the fix. However, when parsing the printed text, an error will be triggered. After fix, the printed length will be 50.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D122925
2022-05-05 16:56:43 -07:00
Aart Bik 952fa3018e [mlir][sparse] add more zero-preserving unary ops to sparse compiler
Although we now have semi-rings to deal with arbitrary ops,
it is still good to convey zero-preserving semantics of
ops to the sparse compiler.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D125043
2022-05-05 15:35:19 -07:00
River Riddle 6609c1cc59 [mlir] Add a better error message when failing to parse an attribute
The fallback attribute parse path is parsing a Type attribute, but this results
in a really unintuitive error message: `expected non-function type`, which
doesn't really hint at tall that we were trying to parse an attribute. This
commit fixes this by trying to optionally parse a type, and on failure
emitting an error that we were expecting an attribute.

Differential Revision: https://reviews.llvm.org/D124870
2022-05-05 15:06:11 -07:00
Stella Stamenova d4555698f8 [mlir] Fix the names of exported functions
The names of the functions that are supposed to be exported do not match the implementations. This is due in part to cac7aabbd8.

This change makes the implementations and declarations match and adds a couple missing declarations.

The new names follow the pattern of the existing `verify` functions where the prefix is maintained as `_mlir_ciface_` but the suffix follows the new naming convention.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D124891
2022-05-05 13:46:15 -07:00
Christopher Bate 22c6e7b277 [mlir][nvvm] Fix support for tf32 data type in mma.sync
The NVVM dialect test coverage for all possible type/shape combinations
in the `nvvm.mma.sync` op is mostly complete. However, there were tests
missing for TF32 datatype support. This change adds tests for the one
relevant shape/type combination. This uncovered a small bug in the op
verifier, which this change also fixes.

Differential Revision: https://reviews.llvm.org/D124975
2022-05-05 11:02:03 -06:00
Matthias Springer e300682597 [mlir][scf][bufferize] Update verifyAnalysis error message
The previous error message was technically incorrect. We do not compare equivalence of YieldOp operands and ForOp operands.

Differential Revision: https://reviews.llvm.org/D124934
2022-05-05 16:56:50 +09:00
Min-Yih Hsu 794c4218a6 [mlir][LLVMIR] Do not update instMap via assignments to entry references
Inside processInstruction, we assign the translated mlir::Value to a
reference previously taken from the corresponding entry in instMap.
However, instMap (a DenseMap) might resize after the entry reference was
taken, rendering the assignment useless since it's assigning to a
dangling reference. Here is a (pseudo) snippet that shows the concept:
```
// inst has type llvm::Instruction *
Value &v = instMap[inst];
...
// op is one of the operands of inst, has type llvm::Value *
processValue(op);
// instMap resizes inside processValue
...
translatedValue = b.createOp<Foo>(...);
// v is already a dangling reference at this point!
// The following assignment is bogus.
v = translatedValue;
```

Nevertheless, after we stop caching llvm::Constant into instMap, there
is only one case that can cause processValue to resize instMap: If the
operand is a llvm::ConstantExpr. In which case we will insert the
derived llvm::Instruction into instMap.
To trigger instMap to resize, which is a DenseMap, the threshold depends
on the ratio between # of map entries and # of (hash) buckets. More specifically,
it resizes if (# of map entries / # of buckets) >= 0.75.
In this case # of map entries is equal to # of LLVM instructions, and # of
buckets is the power-of-two upperbound of # of map entries. Thus, eventually
in the attaching test case (test/Target/LLVMIR/Import/incorrect-instmap-assignment.ll),
we picked 96 and 128 for the # of map entries and # of buckets, respectively.
(We can't pick numbers that are too small since DenseMap used inlined
storage for small number of entries). Therefore, the ConstantExpr in the
said test case (i.e. a GEP) is the 96-th llvm::Value cached into the
instMap, triggering the issue we're discussing here on its enclosing
instruction (i.e. a load).

This patch fixes this issue by calling `operator[]` everytime we need to
update an entry.

Differential Revision: https://reviews.llvm.org/D124627
2022-05-04 13:16:51 -07:00
Bixia Zheng 1cd13e6e98 [mlir][sparse][taco] Support more data types.
Support int8, int16, int32 and int32. Also fix source code format in mlir_pytaco_utils.py.

Add tests.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D124925
2022-05-04 10:05:20 -07:00
Alexander Belyaev e8f7d019fc [mlir] Add a flag to allow equivalent results.
Differential Revision: https://reviews.llvm.org/D124931
2022-05-04 17:48:18 +02:00
Matthias Springer b34ea97f55 [mlir][linalg][bufferize][NFC] Remove remaining Comprehensive Bufferize code
This commit removes the Linalg Comprehensive Bufferize pass.

Differential Revision: https://reviews.llvm.org/D124854
2022-05-04 17:19:44 +09:00
Matthias Springer 5f60c4825b [mlir][linalg][bufferize][NFC] Make init_tensor elimination a separate pre-processing pass
This commit decouples init_tensor elimination from the rest of the bufferization.

Differential Revision: https://reviews.llvm.org/D124853
2022-05-04 17:17:27 +09:00
Matthias Springer 37a1473524 [mlir][bufferize] Allow in-place bufferization for writes to init_tensors in loops
This commit relaxes the rules around ops that define a value but do not specify the tensor's contents. (The only such op at the moment is init_tensor.)

When such a tensor is written in a loop, it should not cause out-of-place bufferization.

Differential Revision: https://reviews.llvm.org/D124849
2022-05-04 16:43:43 +09:00
Jim Kitchen 2c33266084 [mlir][sparse] Add lowering for unary and binary ops
Adding lowering for Unary and Binary required several changes due to
their unique nature of containing custom code for different "regions"
of the sparse structure being operated on. Along with a Kind, a pointer
to the Operation is passed along to be merged once the lattice
structure is figured out.

The original operation is maintained, as it is required for subsequent
lattice decisions. However, sparse_tensor.binary has some branches
are considered as fully handled and therefore are marked with as
kBinaryBranch to distinguish them.

A unique aspect of the custom code is that sometimes the desired result
is no result at all -- i.e. a user wants overlapping sparse entries to
become empty in the output. The solution to this is to return an
uninitialized Value(), which is checked and handled elsewhere in the
code and results in nothing being written to the output tensor for that
case.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D123057
2022-05-03 15:50:26 -05:00
Goran Flegar 672b908bca [mlir] Add sin & cos ops to complex dialect
Also adds conversions for those ops to math + arith.

Differential Revision: https://reviews.llvm.org/D124773
2022-05-03 19:36:12 +02:00
Min-Yih Hsu 857eb4a152 [mlir][LLVMIR] Add support for translating Switch instruction
Add support for translating llvm::SwitchInst.

Differential Revision: https://reviews.llvm.org/D124628
2022-05-03 09:41:40 -07:00
Alex Zinenko 6c57b0debe [mlir] improve and test TransformState::Extension
Add the mechanism for TransformState extensions to update the mapping between
Transform IR values and Payload IR operations held by the state. The mechanism
is intentionally restrictive, similarly to how results of the transform op are
handled.

Introduce test ops that exercise a simple extension that maintains information
across the application of multiple transform ops.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D124778
2022-05-03 11:33:00 +02:00
Benjamin Kramer dccb73318a [mlir][MemRef] Return `0` for the canonical strided layout expr of a 0d memref
There can't be any strides, and the offset for the canonical expr is
always 0. Fixes #55229.

Differential Revision: https://reviews.llvm.org/D124795
2022-05-03 10:58:44 +02:00
Min-Yih Hsu e927a336a5 [mlir][LLVMIR] Add support for translating FCmp & FP constants
This patch add supports for translating FCmp and more kinds of FP
constants in addition to 32 & 64-bit ones. However, we can't express
ppc_fp128 constants right now because the semantics for its underlying
APFloat is `S_PPCDoubleDouble` but mlir::FloatType doesn't support such
semantics right now.

Differential Revision: https://reviews.llvm.org/D124630
2022-05-02 16:22:35 -07:00
Raghu Maddhipatla c685f82126 [mlir][OpenMP] Add omp.cancel and omp.cancellationpoint.
Reviewed By: kiranchandramohan, peixin, shraiysh

Differential Revision: https://reviews.llvm.org/D123828
2022-05-02 12:23:38 -05:00
Shraiysh Vaishay a60fda59dc [mlir][OpenMP] Restrict types for omp.parallel args
This patch restricts the value of `if` clause expression to an I1 value.
It also restricts the value of `num_threads` clause expression to an I32
value.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D124142
2022-05-02 14:17:34 +05:30
River Riddle 3c75228991 [mlir:PDLInterp] Refactor the implementation of result type inferrence
The current implementation uses a discrete "pdl_interp.inferred_types"
operation, which acts as a "fake" handle to a type range. This op is
used as a signal to pdl_interp.create_operation that types should be
inferred. This is terribly awkward and clunky though:

* This op doesn't have a byte code representation, and its conversion
  to bytecode kind of assumes that it is only used in a certain way. The
  current lowering is also broken and seemingly untested.

* Given that this is a different operation, it gives off the assumption
  that it can be used multiple times, or that after the first use
  the value contains the inferred types. This isn't the case though,
  the resultant type range can never actually be used as a type range.

This commit refactors the representation by removing the discrete
InferredTypesOp, and instead adds a UnitAttr to
pdl_interp.CreateOperation that signals when the created operations
should infer their types. This leads to a much much cleaner abstraction,
a more optimal bytecode lowering, and also allows for better error
handling and diagnostics when a created operation doesn't actually
support type inferrence.

Differential Revision: https://reviews.llvm.org/D124587
2022-05-01 12:25:05 -07:00
Chris Lattner d85eb4e2d6 [AsmParser] Introduce a new "Argument" abstraction + supporting logic
MLIR has a common pattern for "arguments" that uses syntax
like `%x : i32 {attrs} loc("sourceloc")` which is implemented
in adhoc ways throughout the codebase.  The approach this uses
is verbose (because it is implemented with parallel arrays) and
inconsistent (e.g. lots of things drop source location info).

Solve this by introducing OpAsmParser::Argument and make addRegion
(which sets up BlockArguments for the region) take it.  Convert the
world to propagating this down.  This means that we correctly
capture and propagate source location information in a lot more
cases (e.g. see the affine.for testcase example), and it also
simplifies much code.

Differential Revision: https://reviews.llvm.org/D124649
2022-04-29 12:19:34 -07:00
Matthias Springer 3c2a74a3ae [mlir][linalg][transform] Add TileOp to transform dialect
This commit adds a tiling op to the transform dialect as an external op.

Differential Revision: https://reviews.llvm.org/D124661
2022-04-29 21:35:31 +09:00
River Riddle 651d9f70ed [mlir:PDLL] Fix the import of native constraints from ODS
We weren't properly returning the result of the constraint,
which leads to errors when actually trying to use the generated
C++.

Differential Revision: https://reviews.llvm.org/D124586
2022-04-28 12:58:00 -07:00
River Riddle ebb1e900d3 [mlir:PDLL] Fix error handling of eof within a string literal
We currently aren't handling this properly, and in the case
of a string block just crash. This commit adds proper error handling
and detection for eof.

Differential Revision: https://reviews.llvm.org/D124585
2022-04-28 12:58:00 -07:00
River Riddle 32bf1f1d57 [mlir:LSP] Improve conversion between SourceMgr and LSP locations
SourceMgr generally uses 1-based locations, whereas the LSP is zero based.
This commit corrects this conversion and also enhances the conversion from SMLoc
to SMRange to support string tokens.

Differential Revision: https://reviews.llvm.org/D124584
2022-04-28 12:58:00 -07:00
River Riddle 9613a850b6 [mlir:PDL] Rework errors for pdl.operations with non-inferrable results
We currently emit an error during verification if a pdl.operation with non-inferrable
results is used within a rewrite. This allows for catching some errors during compile
time, but is slightly broken. For one, the verification at the PDL level assumes that
all dialects have been loaded, which is true at run time, but may not be true when
the PDL is generated (such as via PDLL). This commit fixes this by not emitting the
error if the operation isn't registered, i.e. it uses the `mightHave` variant of trait/interface
methods.

Secondly, we currently don't verify when a pdl.operation has no explicit results, but the
operation being created is known to expect at least one. This commit adds a heuristic
error to detect these cases when possible and fail. We can't always capture when the user
made an error, but we can capture the most common case where the user expected an
operation to infer its result types (when it actually isn't possible).

Differential Revision: https://reviews.llvm.org/D124583
2022-04-28 12:58:00 -07:00
River Riddle d4381b3f93 [mlir:PDL] Fix a syntax ambiguity in pdl.attribute
pdl.attribute currently has a syntax ambiguity that leads to the incorrect parsing
of pdl.attribute operations with locations that don't also have a constant value. For example:

```
pdl.attribute loc("foo")
```

The above IR is treated as being a pdl.attribute with a constant value containing the location,
`loc("foo")`, which is incorrect. This commit changes the syntax to use `= <constant-value>` to
clearly distinguish when the constant value is present, as opposed to just trying to parse an attribute.

Differential Revision: https://reviews.llvm.org/D124582
2022-04-28 12:57:59 -07:00
River Riddle 92a836da07 [mlir] Attach InferTypeOpInterface on SameOperandsAndResultType operations when possible
This allows for inferring the result types of operations in certain situations by using the type of
an operand. This commit allowed for automatically supporting type inference for many more
operations with no additional effort, e.g. nearly all Arithmetic operations now support
result type inferrence with no additional changes.

Differential Revision: https://reviews.llvm.org/D124581
2022-04-28 12:57:59 -07:00
River Riddle 1bd1edaf40 [mlir:ODS] Support using attributes in AllTypesMatch to automatically add InferTypeOpInterface
This allows for using attribute types in result type inference for use with
InferTypeOpInterface. This was a TODO before, but it isn't much
additional work to properly support this. After this commit,
arith::ConstantOp can now have its InferTypeOpInterface implementation automatically
generated.

Differential Revision: https://reviews.llvm.org/D124580
2022-04-28 12:57:59 -07:00
Chris Lattner 5dedf911de [AsmParser] Rework logic around "region argument parsing"
The asm parser had a notional distinction between parsing an
operand (like "%foo" or "%4#3") and parsing a region argument
(which isn't supposed to allow a result number like #3).

Unfortunately the implementation has two problems:

1) It didn't actually check for the result number and reject
   it.  parseRegionArgument and parseOperand were identical.
2) It had a lot of machinery built up around it that paralleled
   operand parsing.  This also was functionally identical, but
   also had some subtle differences (e.g. the parseOptional
   stuff had a different result type).

I thought about just removing all of this, but decided that the
missing error checking was important, so I reimplemented it with
a `allowResultNumber` flag on parseOperand.  This keeps the
codepaths unified and adds the missing error checks.

Differential Revision: https://reviews.llvm.org/D124470
2022-04-28 11:12:44 -07:00
Marius Brehler 84fe39a45b [mlir][emitc] Add a cast op
This adds a cast operation that allows to perform an explicit type
conversion. The cast op is emitted as a C-style cast. It can be applied
to integer, float, index and EmitC types.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D123514
2022-04-28 15:50:59 +00:00
Vitaly Buka 0d70bc990b [mlir][msan][test] Disable jit tests
I am going to enable MLIR test on msan bot
https://lab.llvm.org/buildbot/#/builders/sanitizer-x86_64-linux-bootstrap-msan

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D124574
2022-04-28 08:50:13 -07:00
Marius Brehler 50d648b40e [mlir][emitc] Replace !emitc.opaque pointers
Replaces using !emitc.opaque pointers which using !emitc.ptr types.
2022-04-28 15:20:39 +00:00
Marius Brehler 39dd29736f [mlir][emitc] Disallow !emitc.opaque pointers
Fordbids to express pointer via the `!emitc.opaque` type. Point the user
to use the `!emitc.ptr` type instead.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D124002
2022-04-28 15:08:21 +00:00
Lei Zhang bbffece383 [mlir][spirv] Remove layout decoration on unneeded storage classes
Per SPIR-V validation rules, explict layout decorations are only
needed for StorageBuffer, PhysicalStorageBuffer, Uniform, and
PushConstant storage classes. (And even that is for Shader
capabilities). So we don't need such decorations on the rest.

Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D124543
2022-04-28 08:18:23 -04:00
Lei Zhang 8854b73606 [mlir][spirv] Convert memref.alloca to spv.Variable
Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D124542
2022-04-28 08:13:40 -04:00
Min-Yih Hsu a75657d66a [mlir][LLVMIR] Do not cache llvm::Constant into instMap
Constants in MLIR are not globally unique, unlike that in LLVM IR.
Therefore, reusing previous-translated constants might cause the user
operations not being dominated by the constant (because the
previous-translated ones can be placed in arbitrary place)

This indeed misses some opportunities where we actually can reuse a
previous-translated constants, but verbosity is not our first priority
here.

Differential Revision: https://reviews.llvm.org/D124404
2022-04-27 09:43:49 -07:00
Min-Yih Hsu ea9bcb8b27 [mlir][LLVMIR] Do not cache Instruction generated on-the-fly
More specifically, the llvm::Instruction generated by
llvm::ConstantExpr::getAsInstruction. Such Instruction will be deleted
right away, but it's possible that when getAsInstruction is called
again, it will create a new Instruction that has the same address with
the one we just deleted. Thus, we shouldn't keep it in the `instMap` to
avoid a conflicting index that triggers an assertion in
processInstruction.

Differential Revision: https://reviews.llvm.org/D124402
2022-04-27 09:42:59 -07:00
Min-Yih Hsu 00fcf9e95a [mlir][LLVMIR] Add support for importing struct-type ConstantAggregate(Zero)
And move importer test files from `test/Target/LLVMIR` into
`test/Target/LLVMIR/Import`.

We simply translate struct-type ConstantAggregate(Zero) into a
serious of `llvm.insertvalue` operations against a `llvm.undef` root.
Note that this doesn't affect the original logics on translating
vector/array-type ConstantAggregate values.

Differential Revision: https://reviews.llvm.org/D124399
2022-04-27 09:42:26 -07:00
Mathieu Fehr 88bc24a7e3 [mlir] Allow setting operation legality with an OperationName
This is necessary to handle conversions of operations defined at runtime in extensible dialects.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D124353
2022-04-27 08:54:51 -07:00
Lei Zhang d137c05fc9 [mlir][spirv] Add conversion from vector.reduction
Only supports addition and multiplication for now; other cases
to be implemented.

Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D124380
2022-04-27 10:29:46 -04:00
Lei Zhang 38e802a09d [mlir][spirv] Allow converting from index type in unsigned ops
`index` type is converted to `i32` in SPIR-V. This is fine to
support for all signed/unsigned ops.

Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D124451
2022-04-27 10:13:50 -04:00
Mathieu Fehr 9e0b553359 [mlir] Add extensible dialects
Depends on D104534
Add support for extensible dialects, which are dialects that can be
extended at runtime with new operations and types.

These operations and types cannot at the moment implement traits
or interfaces.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D104554
2022-04-26 19:48:22 -07:00
River Riddle 71aad31c0b [mlir:PDLL] Use normalized paths in compilation database test
This fixes issues with the compilation database when the file path
isn't in the correct form.
2022-04-26 19:45:30 -07:00
River Riddle 021b254547 [mlir:PDLL] Fix build on windows related to different file paths
This fixes issues with the compilation database when the file path
isn't in the correct form.
2022-04-26 19:40:41 -07:00
River Riddle 41d2c6df5c [mlir][PDLL-LSP] Add code completion for include file paths
This allows for providing completion results for include directive
file paths by searching the set of include directories for the current
file.

Differential Revision: https://reviews.llvm.org/D124112
2022-04-26 18:33:17 -07:00
River Riddle 09af7fefc8 [mlir][PDLL] Add document link and hover support to mlir-pdll-lsp-server
This allows for navigating to included files on click, and also provides hover
information about the include file (similarly to clangd).

Differential Revision: https://reviews.llvm.org/D124077
2022-04-26 18:33:17 -07:00
River Riddle fb5a59f6e1 [mlir][PDLL] Add initial support for a PDLL compilation database
The compilation database acts in a similar way to the compilation database
(compile_commands.json) used by clang-tidy, i.e. it provides additional
information about the compilation of project files to help the language
server. The main piece of information provided by the PDLL compilation
database in this commit is the set of include directories used when processing
the input .pdll file. This allows for the server to properly process .pdll files
that use includes anchored by the include directories set up in the build system.

The structure of the textual form of a compilation database is a yaml file
containing documents of the following form:

```
--- !FileInfo:
  filepath: <string> - Absolute file path of the file.
  includes: <string> - Semi-colon delimited list of include directories.
```

This commit also adds support to cmake for automatically generating
a `pdll_compile_commands.yml` file at the top-level of the build
directory.

Differential Revision: https://reviews.llvm.org/D124076
2022-04-26 18:33:17 -07:00