Commit Graph

2303 Commits

Author SHA1 Message Date
Rahul Joshi a0dd5e876f [MLIR] Print function name when ReturnOp verification fails
Summary:
- Print function name when ReturnOp verification fails
- This helps easily finding the invalid ReturnOp in an IR dump.

Differential Revision: https://reviews.llvm.org/D81513
2020-06-10 17:22:49 -07:00
Rob Suderman 3d56f166bd [mlir][StandardOps] Updated IndexCastOp to support tensor<index> cast
Summary:
We now support index casting for tensor<index> to tensor<int>. This
better supports compatibility with the Shape dialect.

Differential Revision: https://reviews.llvm.org/D81611
2020-06-10 17:19:08 -07:00
HazemAbdelhafez 4b7aa6c8c1 [mlir][spirv] Enhance structure type member decoration handling
Modify structure type in SPIR-V dialect to support:
1) Multiple decorations per structure member
2) Key-value based decorations (e.g., MatrixStride)

This commit kept the Offset decoration separate from members'
decorations container for easier implementation and logical clarity.
As such, all references to Structure layoutinfo are now offsetinfo,
and any member layout defining decoration (e.g., RowMajor for Matrix)
will be add to the members' decorations container along with its
value if any.

Differential Revision: https://reviews.llvm.org/D81426
2020-06-10 19:25:03 -04:00
George Mitenkov d93d8fcdec [MLIR][SPIRVToLLVM] Implemented conversion for arithmetic ops and 3 bitwise ops.
Following the previous revision `D81100`, this commit implements a templated class
that would provide conversion patterns for “straightforward” SPIR-V ops into
LLVM dialect. Templating allows to abstract away from concrete implementation
for each specific op. Those are mainly binary operations. Currently supported
and tested ops are:
- Arithmetic ops: `IAdd`, `ISub`, `IMul`, `FAdd`, `FSub`, `FMul`, `FDiv`,  `FNegate`,
  `SDiv`, `SRem` and  `UDiv`
- Bitwise ops: `BitwiseAnd`, `BitwiseOr`, `BitwiseXor`

The implementation relies on `SPIRVToLLVMConversion` class that makes use of
`OpConversionPattern`.

Differential Revision: https://reviews.llvm.org/D81305
2020-06-10 19:10:31 -04:00
Mehdi Amini 83d920c72a Fix MLIR test: -dump-input-on-failure is no longer a valid option 2020-06-10 15:58:58 +00:00
Frederik Gossen 904f91db5f [MLIR][Standard] Make the `dim` operation index an operand.
Allow for dynamic indices in the `dim` operation.
Rather than an attribute, the index is now an operand of type `index`.
This allows to apply the operation to dynamically ranked tensors.
The correct lowering of dynamic indices remains to be implemented.

Differential Revision: https://reviews.llvm.org/D81551
2020-06-10 13:54:47 +00:00
Frederik Gossen e4184c84ca [MLIR][Shape] Make dimension an operand of `get_extent`
The operation `get_extent` now accepts the dimension as an operand and is no
longer limited to constant dimensions.
A helper function facilitates the common constant use case.

Differential Revision: https://reviews.llvm.org/D81248
2020-06-10 11:47:18 +00:00
Stephen Neuendorffer d3ead060be [JitRunner] add support for i32 and i64 output
Differential Revision: https://reviews.llvm.org/D80675
2020-06-09 22:25:03 -07:00
aartbik 1e45b55dcc [mlir] [VectorOps] Handle 'vector.shape_cast' lowering for all cases
Summary:
Even though this operation is intended for 1d/2d conversions currently,
leaving a semantic hole in the lowering prohibits proper testing of this
operation. This CL adds a straightforward reference implementation for the
missing cases.

Reviewers: nicolasvasilache, mehdi_amini, ftynse, reidtatge

Reviewed By: reidtatge

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, msifontes

Tags: #mlir

Differential Revision: https://reviews.llvm.org/D81503
2020-06-09 16:08:45 -07:00
Mehdi Amini d31c9e5a46 Change filecheck default to dump input on failure
Having the input dumped on failure seems like a better
default: I debugged FileCheck tests for a while without knowing
about this option, which really helps to understand failures.

Remove `-dump-input-on-failure` and the environment variable
FILECHECK_DUMP_INPUT_ON_FAILURE which are now obsolete.

Differential Revision: https://reviews.llvm.org/D81422
2020-06-09 18:57:46 +00:00
Stephan Herhut 2c8afe1298 [mlir][gpu] Add support for f16 when lowering to nvvm intrinsics
Summary:
The NVVM target only provides implementations for tanh etc. on f32 and
f64 operands. To also support f16, we now insert operations to extend to f32
and truncate back to f16 around the intrinsic call.

Differential Revision: https://reviews.llvm.org/D81473
2020-06-09 19:33:45 +02:00
msifontes 1c189d71db [mlir] Add number of operands verification for shape.assuming_all operation
Implemented a verification to ensure that the shape.assuming_all
operation always has at least one operand.
2020-06-09 09:59:04 -07:00
George Mitenkov fda5192d4f [MLIR][SPIRVToLLVM] Add skeleton for SPIR-V to LLVM dialect conversion
These commits set up the skeleton for SPIR-V to LLVM dialect conversion.
I created SPIR-V to LLVM pass, registered it in Passes.td, InitAllPasses.h.
Added a pattern for `spv.BitwiseAndOp` and tests for it. Integer, float
and vector types are converted through LLVMTypeConverter.

Differential Revision: https://reviews.llvm.org/D81100
2020-06-08 18:22:42 -04:00
Alexander Belyaev 80be54c08f [mlir] Lower Shape binary ops (AddOp, MulOp) to Standard.
Differential Revision: https://reviews.llvm.org/D81344
2020-06-08 17:48:01 +02:00
Wen-Heng (Jack) Chung 603b974cf7 [mlir][gpu] Fix logic error in D79508 computing number of private attributions.
Fix logic error in D79508. The old logic would make the first check in
`GPUFuncOp::verifyBody` always pass.
2020-06-08 07:40:34 -05:00
Frederik Gossen 215914151e [MLIR][Shape] Add support for `OpAsmInterface` in `shape.const_size`
The SSA values created with `shape.const_size` are now named depending on the
value.
A constant size of 3, e.g., is now automatically named `%c3`.

Differential Revision: https://reviews.llvm.org/D81249
2020-06-08 10:27:28 +00:00
Alexander Belyaev 250dcf61ae Revert "Revert "[MLIR] Lower shape.num_elements -> shape.reduce.""
This reverts commit a25f5cd70c.

Now the build with `-DBUILD_SHARED_LIBS=ON` is fixed.
2020-06-08 12:19:54 +02:00
Frederik Gossen 970bb4a291 [MLIR] Add `to/from_extent_tensor` lowering to the standard dialect
The operations `to_extent_tensor` and `from_extent_tensor` become no-ops when
lowered to the standard dialect.
This is possible with a lowering from `shape.shape` to `tensor<?xindex>`.

Differential Revision: https://reviews.llvm.org/D81162
2020-06-08 09:38:18 +00:00
Frederik Gossen 867bc41e85 [MLIR] Add type conversion for `shape.shape`
Convert `shape.shape` to `tensor<?xindex>` when lowering the `shape` to the
`std` dialect.

Differential Revision: https://reviews.llvm.org/D81161
2020-06-08 09:34:03 +00:00
Tres Popp 68a8336bf2 Revert "Revert "[mlir] Folding and canonicalization of shape.cstr_eq""
This reverts commit 12e31f6e40.
2020-06-08 10:06:55 +02:00
Tres Popp d216f983e6 Revert "Revert "[mlir] Canonicalization and folding of shape.cstr_broadcastable""
This reverts commit 4261b026ad.
2020-06-08 10:06:55 +02:00
Ehsan Toosi 4214031d43 [mlir] Introduce allowMemrefFunctionResults for the helper operation converters of buffer placement
This parameter gives the developers the freedom to choose their desired function
signature conversion for preparing their functions for buffer placement. It is
introduced for BufferAssignmentFuncOpConverter, and also for
BufferAssignmentReturnOpConverter, and BufferAssignmentCallOpConverter to adapt
the return and call operations with the selected function signature conversion.
If the parameter is set, buffer placement won't also deallocate the returned
buffers.

Differential Revision: https://reviews.llvm.org/D81137
2020-06-08 09:25:41 +02:00
Mehdi Amini a25f5cd70c Revert "[MLIR] Lower shape.num_elements -> shape.reduce."
This reverts commit e80617df89.

This broke the build with `-DBUILD_SHARED_LIBS=ON`
2020-06-07 19:32:36 +00:00
Alexander Belyaev e80617df89 [MLIR] Lower shape.num_elements -> shape.reduce.
Differential Revision: https://reviews.llvm.org/D81279
2020-06-07 16:39:21 +02:00
Alexander Belyaev 50f68c1e33 [mlir] Add verifier for `shape.yield`.
Differential Revision: https://reviews.llvm.org/D81262
2020-06-07 15:40:11 +02:00
Tres Popp 4261b026ad Revert "[mlir] Canonicalization and folding of shape.cstr_broadcastable"
This reverts commit 6aab709459.

Some users have failing builds with ShapeCanonicalization.td, so revert
for now.
2020-06-06 11:17:44 +02:00
Tres Popp 12e31f6e40 Revert "[mlir] Folding and canonicalization of shape.cstr_eq"
This reverts commit 0a554e607f.

Some users have build failures when building ShapeCanonicalization.td,
so revert changes that created and rely on it.
2020-06-06 11:08:41 +02:00
Diego Caballero 7d59f49bda [mlir] Fix representation of BF16 constants
This patch is a follow-up on https://reviews.llvm.org/D81127

BF16 constants were represented as 64-bit floating point values due to the lack
of support for BF16 in APFloat. APFloat was recently extended to support
BF16 so this patch is fixing the BF16 constant representation to be 16-bit.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D81218
2020-06-05 17:43:06 -07:00
Nicolas Vasilache b54a4d0f8f [mlir][Linalg] NFC - Make useFullTileBuffersByDefault option take a boolean. 2020-06-05 17:44:29 -04:00
Nicolas Vasilache b6c88549bc [mlir] Fix spurious f64 -> f16 change in CPU runner test 2020-06-05 17:23:21 -04:00
Nicolas Vasilache eb7db879af [mlir][test][CPU] Reduce the size of mlir-cpu-runner-tests
Two tests regularly show up on the long tail when testing MLIR.
This revision reduces their size.
2020-06-05 13:47:29 -04:00
Nicolas Vasilache b56bf30d3c [mlir][Vector] Add folding of memref_cast into vector_transfer ops
Summary:
This revision adds a common folding pattern that starts appearing on
vector_transfer ops.

Differential Revision: https://reviews.llvm.org/D81281
2020-06-05 13:27:00 -04:00
Jacques Pienaar b0921f68e1 [mlir] Add verify method to adaptor
This allows verifying op-indepent attributes (e.g., attributes that do not require the op to have been created) before constructing an operation. These include checking whether required attributes are defined or constraints on attributes (such as I32 attribute). This is not perfect (e.g., if one had a disjunctive constraint where one part relied on the op and the other doesn't, then this would not try and extract the op independent from the op dependent).

The next step is to move these out to a trait that could be verified earlier than in the generated method. The first use case is for inferring the return type while constructing the op. At that point you don't have an Operation yet and that ends up in one having to duplicate the same checks, e.g., verify that attribute A is defined before querying A in shape function which requires that duplication. Instead this allows one to invoke a method to verify all the traits and, if this is checked first during verification, then all other traits could use attributes knowing they have been verified.

It is a little bit funny to have these on the adaptor, but I see the adaptor as a place to collect information about the op before the op is constructed (e.g., avoiding stringly typed accessors, verifying what is possible to verify before the op is constructed) while being cheap to use even with constructed op (so layer of indirection between the op constructed/being constructed). And from that point of view it made sense to me.

Differential Revision: https://reviews.llvm.org/D80842
2020-06-05 09:47:37 -07:00
Julian Lettner 99d6e05e71 [lit] Improve naming of test result categories
Improve consistency when printing test results:
Previously we were using different labels for group names (the header
for the list of, e.g., failing tests) and summary count lines.  For
example, "Failing Tests"/"Unexpected Failures".  This commit changes lit
to label things consistently.

Improve wording of labels:
When talking about individual test results, the first word in
"Unexpected Failures", "Expected Passes", and "Individual Timeouts" is
superfluous.  Some labels contain the word "Tests" and some don't.
Let's simplify the names.

Before:
```
Failing Tests (1):
  ...

Expected Passes    : 3
Unexpected Failures: 1
```

After:
```
Failed Tests (1):
  ...

Passed: 3
Failed: 1
```

Reviewed By: ldionne

Differential Revision: https://reviews.llvm.org/D77708
2020-06-05 08:14:42 -07:00
Wen-Heng (Jack) Chung 2fd6403a6d [mlir][gpu] Introduce mlir-rocm-runner.
Summary:
`mlir-rocm-runner` is introduced in this commit to execute GPU modules on ROCm
platform. A small wrapper to encapsulate ROCm's HIP runtime API is also inside
the commit.

Due to behavior of ROCm, raw pointers inside memrefs passed to `gpu.launch`
must be modified on the host side to properly capture the pointer values
addressable on the GPU.

LLVM MC is used to assemble AMD GCN ISA coming out from
`ConvertGPUKernelToBlobPass` to binary form, and LLD is used to produce a shared
ELF object which could be loaded by ROCm HIP runtime.

gfx900 is the default target be used right now, although it could be altered via
an option in `mlir-rocm-runner`. Future revisions may consider using ROCm Agent
Enumerator to detect the right target on the system.

Notice AMDGPU Code Object V2 is used in this revision. Future enhancements may
upgrade to AMDGPU Code Object V3.

Bitcode libraries in ROCm-Device-Libs, which implements math routines exposed in
`rocdl` dialect are not yet linked, and is left as a TODO in the logic.

Reviewers: herhut

Subscribers: mgorny, tpr, dexonsmith, mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, csigg, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #mlir, #llvm

Differential Revision: https://reviews.llvm.org/D80676
2020-06-05 09:46:39 -05:00
HazemAbdelhafez cc2349e3cf [MLIR][SPIRV] Support flat, location, and noperspective decorations
Add support for flat, location, and noperspective decorations in the
serializer and deserializer to be able to process basic shader files
for graphics applications.

Differential Revision: https://reviews.llvm.org/D80837
2020-06-05 08:55:22 -04:00
Nicolas Vasilache 247e185dd5 [mlir][Vector] Move temporary alloc to top of the function alloca when lowering vector_transfers
Recently introduced allocation hoisting is quite conservative on the cases when it triggers.
This revision makes it such that the allocations for vector transfer lowerings are hoisted
to the top of the function.
This should be revisited in the context of parallelism and is a temporary workaround.

Differential Revision: https://reviews.llvm.org/D81253
2020-06-05 08:45:52 -04:00
Nicolas Vasilache 6953cf6502 [mlir][Linalg] Add a hoistRedundantVectorTransfers helper function
This revision adds a helper function to hoist vector.transfer_read /
vector.transfer_write pairs out of immediately enclosing scf::ForOp
iteratively, if the following conditions are true:
   1. The 2 ops access the same memref with the same indices.
   2. All operands are invariant under the enclosing scf::ForOp.
   3. No uses of the memref either dominate the transfer_read or are
   dominated by the transfer_write (i.e. no aliasing between the write and
   the read across the loop)

To improve hoisting opportunities, call the `moveLoopInvariantCode` helper
function on the candidate loop above which to hoist. Hoisting the transfers
results in scf::ForOp yielding the value that originally transited through
memory.

This revision additionally exposes `moveLoopInvariantCode` as a helper in
LoopUtils.h and updates SliceAnalysis to support return scf::For values and
allow hoisting across multiple scf::ForOps.

Differential Revision: https://reviews.llvm.org/D81199
2020-06-05 06:50:24 -04:00
Alexander Belyaev 04fb2b6123 [Mlir] Implement printer, parser, verifier and builder for shape.reduce.
Differential Revision: https://reviews.llvm.org/D81186
2020-06-05 11:25:32 +02:00
Tres Popp 655e08ceeb [mlir] Canonicalization of shape.assuming
Summary:
This will inline the region to a shape.assuming in the case that the
input witness is found to be statically true.

Differential Revision: https://reviews.llvm.org/D80302
2020-06-05 11:00:20 +02:00
Tres Popp 0a554e607f [mlir] Folding and canonicalization of shape.cstr_eq
In the case of all inputs being constant and equal, cstr_eq will be
replaced with a true_witness.

Differential Revision: https://reviews.llvm.org/D80303
2020-06-05 11:00:20 +02:00
Tres Popp 6aab709459 [mlir] Canonicalization and folding of shape.cstr_broadcastable
This allows replacing of this op with a true witness in the case of both
inputs being const_shapes and being found to be broadcastable.

Differential Revision: https://reviews.llvm.org/D80304
2020-06-05 11:00:19 +02:00
Tres Popp 4a255bbd29 [mlir] Add folding for shape.any
If any input to shape.any is a const_shape, shape.any can be replaced
with that input.

Differential Revision: https://reviews.llvm.org/D80305
2020-06-05 11:00:19 +02:00
Tres Popp 6b3a5bff93 [mlir] Folding of shape.assuming_all
This allows assuming_all to be replaced when all inputs are known to be
statically passing witnesses.

Differential Revision: https://reviews.llvm.org/D80306
2020-06-05 11:00:19 +02:00
Tres Popp 1c3e38d98c [mlir] Add a shape op that returns a constant witness
This will later be used during canonicalization and folding steps to replace
statically known passing constraints.

Differential Revision: https://reviews.llvm.org/D80307
2020-06-05 11:00:19 +02:00
Alexander Belyaev 5a675f0552 [Mlir] Add assembly format for `shape.mul`.
Differential Revision: https://reviews.llvm.org/D81194
2020-06-05 10:55:54 +02:00
Uday Bondhugula 0f6999af88 [MLIR] Update linalg.conv lowering to use affine load in the absence of padding
Update linalg to affine lowering for convop to use affine load for input
whenever there is no padding. It had always been using std.loads because
max in index functions (needed for non-zero padding if not materializing
zeros) couldn't be represented in the non-zero padding cases.

In the future, the non-zero padding case could also be made to use
affine - either by materializing or using affine.execute_region. The
latter approach will not impact the scf/std output obtained after
lowering out affine.

Differential Revision: https://reviews.llvm.org/D81191
2020-06-05 12:28:30 +05:30
River Riddle c0cd1f1c5c [mlir] Refactor BoolAttr to be a special case of IntegerAttr
This simplifies a lot of handling of BoolAttr/IntegerAttr. For example, a lot of places currently have to handle both IntegerAttr and BoolAttr. In other places, a decision is made to pick one which can lead to surprising results for users. For example, DenseElementsAttr currently uses BoolAttr for i1 even if the user initialized it with an Array of i1 IntegerAttrs.

Differential Revision: https://reviews.llvm.org/D81047
2020-06-04 16:41:24 -07:00
Nicolas Vasilache 3463d9835b [mlir][Linalg] Add a hoistViewAllocOps helper function
This revision adds a helper function to hoist alloc/dealloc pairs and
alloca op out of immediately enclosing scf::ForOp if both conditions are true:
   1. all operands are defined outside the loop.
   2. all uses are ViewLikeOp or DeallocOp.

This is now considered Linalg-specific and will be generalized on a per-need basis.

Differential Revision: https://reviews.llvm.org/D81152
2020-06-04 18:59:03 -04:00
Diego Caballero 5c990d6994 [mlir] Add support for bf16 to StandardToLLVM conversion
Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D81127
2020-06-04 14:36:36 -07:00
Thomas Raoux 661235e126 [mlir][gpu] Add subgroup Id/Size/Num to GPU dialect
Add SubgroupId, SubgroupSize and NumSubgroups to GPU dialect ops and add the
lowering of those ops to SPIRV.

Differential Revision: https://reviews.llvm.org/D81042
2020-06-04 10:52:40 -07:00
Hanhan Wang 0b025d2733 [mlir][StandardToSPIRV] Handle i1 case for lowering std.zexti to SPIR-V.
Differential Revision: https://reviews.llvm.org/D80965
2020-06-03 15:01:18 -07:00
Hanhan Wang 27fca57546 [mlir][Linalg] Add support for fusion between indexed_generic ops and tensor_reshape ops
Summary:
The fusion for tensor_reshape is embedding the information to indexing maps,
thus the exising pattenr also works for indexed_generic ops.

Depends On D80347

Differential Revision: https://reviews.llvm.org/D80348
2020-06-03 14:59:47 -07:00
Hanhan Wang cc11ceda16 [mlir][Linalg] Add support for fusion between indexed_generic ops and generic ops on tensors.
Summary:
Different from the fusion between generic ops, indices are involved. In this
context, we need to re-map the indices for producer since the fused op is built
on consumer's perspective. This patch supports all combination of the fusion
between indexed_generic ops and generic ops, which includes tests case:
  1) generic op as producer and indexed_generic op as consumer.
  2) indexed_generic op as producer and generic op as consumer.
  3) indexed_generic op as producer and indexed_generic op as consumer.

Differential Revision: https://reviews.llvm.org/D80347
2020-06-03 14:58:43 -07:00
aartbik 6391da98f4 [mlir] [VectorOps] Use 'vector.flat_transpose' for 2-D 'vector.tranpose'
Summary:
Progressive lowering of vector.transpose into an operation that
is closer to an intrinsic, and thus the hardware ISA. Currently
under the common vector transform testing flag, as we prepare
deploying this transformation in the LLVM lowering pipeline.

Reviewers: nicolasvasilache, reidtatge, andydavis1, ftynse

Reviewed By: nicolasvasilache, ftynse

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #llvm, #mlir

Differential Revision: https://reviews.llvm.org/D80772
2020-06-03 14:55:50 -07:00
Frederik Gossen 3713314bfa [MLIR] Shape to standard dialect lowering
Add a new pass to lower operations from the `shape` to the `std` dialect.
The conversion applies only to the `size_to_index` and `index_to_size`
operations and affected types.
Other patterns will be added as needed.

Differential Revision: https://reviews.llvm.org/D81091
2020-06-03 16:17:03 +00:00
Nicolas Vasilache e349fb70a2 [mlir][Linalg] NFC - Make markers use Identifier instead of StringRef
Summary: This removes string ownership worries by putting everything into the context and allows more constructing identifiers programmatically.

Reviewers: ftynse

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul

Tags: #mlir

Differential Revision: https://reviews.llvm.org/D81027
2020-06-03 05:52:32 -04:00
Diego Caballero 8a418e5f8e [mlir][Affine] Enable fusion of loops with vector loads/stores
This patch enables affine loop fusion for loops with affine vector loads
and stores. For that, we only had to use affine memory op interfaces in
LoopFusionUtils.cpp and Utils.cpp so that vector loads and stores are
also taken into account.

Reviewed By: andydavis1, ftynse

Differential Revision: https://reviews.llvm.org/D80971
2020-06-03 01:26:22 +03:00
HazemAbdelhafez 915e55c910 [mlir][spirv] Add support for matrix type
This commit adds basic matrix type support to the SPIR-V dialect
including type definition, IR assembly, parsing, printing, and
(de)serialization.

Differential Revision: https://reviews.llvm.org/D80594
2020-06-02 16:30:58 -04:00
Alex Zinenko 5c5dafc534 [mlir] support materialization for 1-1 type conversions
Dialect conversion infrastructure supports 1->N type conversions by requiring
individual conversions to provide facilities to generate operations
retrofitting N values into 1 of the original type when N > 1. This
functionality can also be used to materialize explicit "cast"-like operations,
but it did not support 1->1 type conversions until now. Modify TypeConverter to
support materialization of cast operations for 1-1 conversions.

This also makes materialization specification more extensible following the
same pattern as type conversions. Instead of overloading a virtual function,
users or subclasses of TypeConversion can now register type-specific
materialization callbacks that will be called in order for the given type.

Differential Revision: https://reviews.llvm.org/D79729
2020-06-02 13:48:33 +02:00
Ehsan Toosi 3f6a35e3ff [mlir] Introduce CallOp converter for buffer placement
Add BufferAssignmentCallOpConverter as a pattern rewriter for Buffer
Placement. It matches the signature of the caller operation with the callee
after rewriting the callee with FunctionAndBlockSignatureConverter.

Differential Revision: https://reviews.llvm.org/D80785
2020-06-02 11:35:24 +02:00
MaheshRavishankar 2bcd1927dd [mlir][SCFToGPU] Remove conversions from scf.for to gpu.launch.
Keeping in the affine.for to gpu.launch conversions, which should
probably be the affine.parallel to gpu.launch conversion as well.

Differential Revision: https://reviews.llvm.org/D80747
2020-06-01 23:06:20 -07:00
Thomas Raoux c652c306a6 [mlir][spirv] Clean up coop matrix assembly declaration.
Address code review feedback and use declarative assembly format.

Differential Revision: https://reviews.llvm.org/D80687
2020-05-29 16:37:35 -07:00
Nicolas Vasilache 9534192c3b [mlir][Linalg] Make contraction vectorization use vector transfers
This revision replaces the load + vector.type_cast by appropriate vector transfer
operations. These play more nicely with other vector abstractions and canonicalization
patterns and lower to load/store with or without masks when appropriate.

Differential Revision: https://reviews.llvm.org/D80809
2020-05-29 15:03:46 -04:00
Anchu Rajendran dbb5979d15 [MLIR][OpenMP] Defined master operation in OpenMP Dialect
Summary:
Implemented the basic changes for defining master operation in OpenMP.
It uses the generic parser and printer.

Reviewed By: kiranchandramohan, ftynse

Differential Revision: https://reviews.llvm.org/D80689
2020-05-29 22:46:02 +05:30
Nicolas Vasilache 1ee114322c [mlir][Linalg][Vector] Add forwarding patterns between linalg.copy and vector.transfer
This revision adds custom rewrites for patterns that arise during linalg structured
ops vectorization. These patterns allow the composition of linalg promotion,
vectorization and removal of redundant copies.

The patterns are voluntarily limited and restrictive atm.
More robust behavior will be implemented once more powerful side effect modeling and analyses are available on view/subview.

On the transfer_read side, the following pattern is rewritten:
```
   %alloc = ...
   [optional] %view = std.view %alloc ...
   %subView = subview %allocOrView ...
   [optional] linalg.fill(%allocOrView, %cst) ...
   ...
   linalg.copy(%in, %subView) ...
   vector.transfer_read %allocOrView[...], %cst ...
```
into
```
   [unchanged] %alloc = ...
   [unchanged] [optional] %view = std.view %alloc ...
   [unchanged] [unchanged] %subView = subview %allocOrView ...
   ...
   vector.transfer_read %in[...], %cst ...
```

On the transfer_write side, the following pattern is rewriten:
```
   %alloc = ...
   [optional] %view = std.view %alloc ...
   %subView = subview %allocOrView...
   ...
   vector.transfer_write %..., %allocOrView[...]
   linalg.copy(%subView, %out)
```

Differential Revision: https://reviews.llvm.org/D80728
2020-05-29 08:08:34 -04:00
Nicolas Vasilache aa93659c9f [mlir][SCF] Add utility to clone an scf.ForOp while appending new yield values.
This utility factors out the machinery required to add iterArgs and yield values to an scf.ForOp.

Differential Revision: https://reviews.llvm.org/D80656
2020-05-29 07:28:17 -04:00
Ehsan Toosi 7a3a253585 [MLIR][BufferPlacement] Support functions that return Memref typed results
Buffer placement can now operates on functions that return buffers. These
buffers escape from the deallocation phase of buffer placement.

Differential Revision: https://reviews.llvm.org/D80696
2020-05-29 11:03:22 +02:00
Marius Brehler b0b2507717 [mlir] Add test to check if standalone dialect is registered
Summary: Add a test to check if the standalone dialect is registered within standalone-opt. Similar to the mlir-opt commandline.mlir test.

Reviewers: Kayjukh, stephenneuendorffer

Reviewed By: Kayjukh

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, grosul1, frgossen, jurahul, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80764
2020-05-29 00:34:34 +02:00
Nicolas Vasilache 5f9e0466f2 [mlir][Vector] Fix vector.transfer alignment calculation
https://reviews.llvm.org/D79246 introduces alignment propagation for vector transfer operations. Unfortunately, the alignment calculation is incorrect and can result in crashes.

This revision fixes the calculation by using the natural alignment of the memref elemental type, instead of the resulting vector type.

If more alignment is desired, it can be done in 2 ways:
1. use a proper vector.type_cast to transform a memref<axbxcxdxf32> into a memref<axbxvector<cxdxf32>> giving a natural alignment of vector<cxdxf32>
2. add an alignment attribute to vector transfer operations and propagate it.

With this change the alignment in the relevant tests goes down from 128 to 4.

Lastly, a few minor cleanups are performed and the custom `isMinorIdentityMap` is deprecated.

Differential Revision: https://reviews.llvm.org/D80734
2020-05-28 17:58:51 -04:00
Marius Brehler 3bff62d45f [mlir] Extend standalone example by standalone-translate
Extend the standalone by standalone-translate, based on mlir-translate.

Differential Revision: https://reviews.llvm.org/D80737
2020-05-28 14:07:55 -07:00
MaheshRavishankar 2b0c8546ac [mlir][Linalg] Add pass to remove unit-extent dims from tensor
operands of Generic ops.

Unit-extent dimensions are typically used for achieving broadcasting
behavior. The pattern added (along with canonicalization patterns
added previously) removes the use of unit-extent dimensions, and
instead uses a more canonical representation of the computation.  This
new pattern is not added as a canonicalization for now since it
entails adding additional reshape operations. A pass is added to
exercise these patterns, along with an API entry to populate a
patterns list with these patterns.

Differential Revision: https://reviews.llvm.org/D79766
2020-05-28 11:06:47 -07:00
Alex Zinenko 72ede60b75 [mlir][GPU] Link relevant LLVM components in GPUCommon instead of test
D80142 restructured MLIR-to-GPU-binary conversion to support multiple
targets. It also modified cmake files to link relevant LLVM components
in test/lib, which broke shared-library builds, and likely made the
conversions unusable outside mlir-opt (or other tools that link in test
library targets). Link these components to GPUCommon instead.

Differential Revision: https://reviews.llvm.org/D80739
2020-05-28 20:01:54 +02:00
Jacques Pienaar fefe4366c3 [mlir] Use ValueRange instead of ArrayRef<Value>
This allows constructing operand adaptor from existing op (useful for commonalizing verification as I want to do in a follow up).

I also add ability to use member initializers for the generated adaptor constructors for convenience.

Differential Revision: https://reviews.llvm.org/D80667
2020-05-28 09:05:24 -07:00
Wen-Heng (Jack) Chung 061fb8eb2d [mlir][gpu][mlir-cuda-runner] Refactor ConvertKernelFuncToCubin to be generic.
Make ConvertKernelFuncToCubin pass to be generic:

- Rename to ConvertKernelFuncToBlob.
- Allow specifying triple, target chip, target features.
- Initializing LLVM backend is supplied by a callback function.
- Lowering process from MLIR module to LLVM module is via another callback.
- Change mlir-cuda-runner to adopt the revised pass.
- Add new tests for lowering to ROCm HSA code object (HSACO).
- Tests for CUDA and ROCm are kept in separate directories.

Differential Revision: https://reviews.llvm.org/D80142
2020-05-28 09:08:28 -05:00
Frederik Gossen fdaa391e3d [MLIR] Add `num_elements` to the shape dialect
The operation `num_elements` determines the number of elements for a given
shape.
That is the product of its dimensions.

Differential Revision: https://reviews.llvm.org/D80281
2020-05-28 14:05:58 +00:00
Frederik Gossen 6594d54571 [MLIR] Add `index_to_size` and `size_to_index` to the shape dialect
Add the two conversion operations `index_to_size` and `size_to_index` to the
shape dialect.
This facilitates the conversion of index types between the shape and the
standard dialect.

Differential Revision: https://reviews.llvm.org/D80280
2020-05-28 13:57:20 +00:00
Alexander Belyaev c3098e4f40 [MLIR] Add TensorFromElementsOp to Standard ops.
Differential Revision: https://reviews.llvm.org/D80705
2020-05-28 15:48:10 +02:00
Sean Silva 25132b36a8 [mlir][shape] Use IndexElementsAttr in Shape dialect.
Summary:
Index is the proper type for storing shapes when constant folding, so
this fixes the previous code (which was using i64).

Differential Revision: https://reviews.llvm.org/D80600
2020-05-27 13:39:49 -07:00
Sean Silva 9546d8b108 [mlir][core] Add IndexElementsAttr helpers.
Summary:
In a follow-up, I'll update the Shape dialect to use this instead of
I64ElementsAttr.

Differential Revision: https://reviews.llvm.org/D80601
2020-05-27 13:39:48 -07:00
aartbik c295a65da4 [mlir] [VectorOps] Add 'vector.flat_transpose' operation
Summary:
Provides a representation of the linearized LLVM instrinsic.
With tests and lowering implementation to LLVM IR dialect.
Prepares better lowering for 2-D vector.transpose.

Reviewers: nicolasvasilache, ftynse, reidtatge, bkramer, dcaballe

Reviewed By: ftynse, dcaballe

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80419
2020-05-27 11:09:48 -07:00
MaheshRavishankar 4d6f44f5f0 [mlir][spirv] Lower allocation/deallocations of workgroup memory.
This allocation of a workgroup memory is lowered to a
spv.globalVariable. Only static size allocation with element type
being int or float is handled. The lowering does account for the
element type that are not supported in the lowered spv.module based on
the extensions/capabilities and adjusts the number of elements to get
the same byte length.

Differential Revision: https://reviews.llvm.org/D80411
2020-05-27 09:53:16 -07:00
David Truby 5ba874e472 [MLIR] [OpenMP] Add basic OpenMP parallel operation
Summary:
This includes a basic implementation for the OpenMP parallel
operation without a custom pretty-printer and parser.
The if, num_threads, private, shared, first_private, last_private,
proc_bind and default clauses are included in this implementation.

Currently the reduction clause is omitted as it is more complex and
requires analysis to see if we can share implementation with the loop
dialect. The allocate clause is also omitted.

A discussion about the design of this operation can be found here:
https://llvm.discourse.group/t/openmp-parallel-operation-design-issues/686

The current OpenMP Specification can be found here:
https://www.openmp.org/wp-content/uploads/OpenMP-API-Specification-5.0.pdf

Co-authored-by: Kiran Chandramohan <kiran.chandramohan@arm.com>

Reviewers: jdoerfert

Subscribers: mgorny, yaxunl, kristof.beyls, guansong, mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, Joonsoo, grosul1, frgossen, Kayjukh, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79410
2020-05-27 17:16:44 +01:00
Jacques Pienaar 31f40f603d [mlir] Add simple generator for return types
Take advantage of equality constrains to generate the type inference interface.
This is used for equality and trivially built types. The type inference method
is only generated when no type inference trait is specified already.

This reorders verification that changes some test error messages.

Differential Revision: https://reviews.llvm.org/D80484
2020-05-27 08:45:55 -07:00
MaheshRavishankar 0ed2d4c7cb [mlir][linalg] Allow promotion to use callbacks for
alloc/dealloc/copies.

Add options to LinalgPromotion to use callbacks for implementating the
allocation, deallocation of buffers used for the promoted subviews,
and to copy data into and from the original subviews to the allocated
buffers.
Also some misc. cleanup of the code.

Differential Revision: https://reviews.llvm.org/D80365
2020-05-26 21:33:57 -07:00
MaheshRavishankar 5759e47316 [mlir][Linalg] Avoid using scf.parallel for non-parallel loops in Linalg ops.
Modifying the loop nest builder for generating scf.parallel loops to
not generate scf.parallel loops for non-parallel iterator types in
Linalg operations. The existing implementation incorrectly generated
scf.parallel for all tiled loops. It is rectified by refactoring logic
used while lowering to loops that accounted for this.

Differential Revision: https://reviews.llvm.org/D80188
2020-05-26 21:33:57 -07:00
Sean Silva cf42b70439 [mlir][shape] Add `shape.get_extent`.
Summary:
This op extracts an extent from a shape.

This also is the first op which constant folds to shape.const_size,
which revealed that shape.const_size needs a folder (ConstantLike ops
seem to always need folders for the constant folding infra to work).

Differential Revision: https://reviews.llvm.org/D80394
2020-05-26 17:03:40 -07:00
Nicolas Vasilache ba10daa820 [mlir][Vector] Add more vector.contract -> outerproduct lowerings and fix vector.contract type inference.
This revision expands the types of vector contractions that can be lowered to vector.outerproduct.
All 8 permutation cases are support.
The idiomatic manipulation of AffineMap written declaratively makes this straightforward.

In the process a bug with the vector.contract verifier was uncovered.
The vector shape verification part of the contract op is rewritten to use AffineMap composition.
One bug in the vector `ops.mlir` test is fixed and a new case not yet captured is added
to the vector`invalid.mlir` test.

Differential Revision: https://reviews.llvm.org/D80393
2020-05-26 15:40:55 -04:00
Christian Sigg 222e0e58a8 [MLIR] Helper class referencing MemRefType to unify runner implementations.
Summary:
Add DynamicMemRefType which can reference one of the statically ranked StridedMemRefType or a UnrankedMemRefType so that runner utils only need to be implemented once.

There is definitely room for more clean up and unification, but I will keep that for follow-ups.

Reviewers: nicolasvasilache

Reviewed By: nicolasvasilache

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80513
2020-05-26 16:32:36 +02:00
Nicolas Vasilache 9578a54f50 [mlir][Vector] Add vector contraction to outerproduct lowering
This revision adds the additional lowering and exposes the patterns at a finer granularity for better programmatic reuse. The unit test makes use of the finer grained pattern for simpler checks.

As the ContractionOpLowering is exposed programmatically, cleanup opportunities appear and static class methods are turned into free functions with static visibility.

Differential Revision: https://reviews.llvm.org/D80375
2020-05-26 09:31:26 -04:00
George Mitenkov 7293dd5b40 Added pow intrinsic to LLVMIR dialect
Added pow intrinsic to LLVMIR dialect. Added a roundrip test for it.

Differential Revision: https://reviews.llvm.org/D80248
2020-05-25 07:57:33 -04:00
Jacques Pienaar 4b8632e174 [mlir] Expand operand adapter to take attributes
* Enables using with more variadic sized operands;
* Generate convenience accessors for attributes;
  - The accessor are named the same as their name in ODS and returns attribute
    type (not convenience type) and no derived attributes.

This is first step to changing adapter to support verifying argument
constraints before the op is even created. This does not change the name of
adaptor nor does it require it except for ops with variadic operands to keep this change smaller.

Considered creating separate adapter but decided against that given operands also require attributes in general (and definitely for verification of operands and attributes).

Differential Revision: https://reviews.llvm.org/D80420
2020-05-24 21:06:47 -07:00
Thomas Raoux 0712eac766 [mlir][spirv] Enable composite instructions for cooperative matrix type.
Enable inset/extract/construct composite ops as well as access chain for
cooperative matrix. ConstantComposite requires more change and will be done in
a separate patch. Also fix the getNumElements function for coopMatrix per
feedback from Jeff Bolz. The number of element is implementation dependent so
it cannot be known at compile time.

Differential Revision: https://reviews.llvm.org/D80321
2020-05-21 12:19:55 -07:00
Thomas Raoux 15389cdc5b [mlir][spirv] Add remaining cooperative matrix instructions
Adds support for cooperative matrix support for arithmetic and cast
instructions. It also adds cooperative matrix store, muladd and matrixlength
instructions which are part of the extension.

Differential Revision: https://reviews.llvm.org/D80181
2020-05-21 11:55:33 -07:00
jerryyin 9c53ac08de [mlir][rocdl] Exposing buffer load/store intrinsic
Summary:
* Updated ROCDLOps tablegen
* Added parsing and printing function for new intrinsic
* Added unit tests

Reviewers: ftynse

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80233
2020-05-21 14:14:35 +00:00
Wen-Heng (Jack) Chung 2cbbc266ec [mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass.
Due to similar APIs between CUDA and ROCm (HIP),
ConvertGpuLaunchFuncToCudaCalls pass could be used on both platforms with some
refactoring.

In this commit:

- Migrate ConvertLaunchFuncToCudaCalls from GPUToCUDA to GPUCommon, and rename.
- Rename runtime wrapper APIs be platform-neutral.
- Let GPU binary annotation attribute be specifiable as a PassOption.
- Naming changes within the implementation and tests.

Subsequent patches would introduce ROCm-specific tests and runtime wrapper
APIs.

Differential Revision: https://reviews.llvm.org/D80167
2020-05-21 08:53:47 -05:00
Mehdi Amini 5c3ebd7725 Revert "[mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass."
This reverts commit cdb6f05e2d.

The build is broken with:

  You have called ADD_LIBRARY for library obj.MLIRGPUtoCUDATransforms without any source files. This typically indicates a problem with your CMakeLists.txt file
2020-05-21 03:44:35 +00:00
Wen-Heng (Jack) Chung cdb6f05e2d [mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass.
Due to similar APIs between CUDA and ROCm (HIP),
ConvertGpuLaunchFuncToCudaCalls pass could be used on both platforms with some
refactoring.

In this commit:

- Migrate ConvertLaunchFuncToCudaCalls from GPUToCUDA to GPUCommon, and rename.
- Rename runtime wrapper APIs be platform-neutral.
- Let GPU binary annotation attribute be specifiable as a PassOption.
- Naming changes within the implementation and tests.

Subsequent patches would introduce ROCm-specific tests and runtime wrapper
APIs.

Differential Revision: https://reviews.llvm.org/D80167
2020-05-20 16:11:48 -05:00
MaheshRavishankar 0e88eb5c51 [mlir][spirv] Adapt subview legalization to the updated op semantics.
The subview semantics changes recently to allow for more natural
representation of constant offsets and strides. The legalization of
subview op for lowering to SPIR-V needs to account for this.
Also change the linearization to use the strides from the affine map
of a memref.

Differential Revision: https://reviews.llvm.org/D80270
2020-05-20 12:00:21 -07:00
MaheshRavishankar 071358e082 [mlir][Linalg] Add producer-consumer fusion when producer is a ConstantOp
and Consumer is a GenericOp.

Differential Revision: https://reviews.llvm.org/D79838
2020-05-20 09:16:19 -07:00