Commit Graph

679 Commits

Author SHA1 Message Date
Wen-Heng (Jack) Chung 2fd6403a6d [mlir][gpu] Introduce mlir-rocm-runner.
Summary:
`mlir-rocm-runner` is introduced in this commit to execute GPU modules on ROCm
platform. A small wrapper to encapsulate ROCm's HIP runtime API is also inside
the commit.

Due to behavior of ROCm, raw pointers inside memrefs passed to `gpu.launch`
must be modified on the host side to properly capture the pointer values
addressable on the GPU.

LLVM MC is used to assemble AMD GCN ISA coming out from
`ConvertGPUKernelToBlobPass` to binary form, and LLD is used to produce a shared
ELF object which could be loaded by ROCm HIP runtime.

gfx900 is the default target be used right now, although it could be altered via
an option in `mlir-rocm-runner`. Future revisions may consider using ROCm Agent
Enumerator to detect the right target on the system.

Notice AMDGPU Code Object V2 is used in this revision. Future enhancements may
upgrade to AMDGPU Code Object V3.

Bitcode libraries in ROCm-Device-Libs, which implements math routines exposed in
`rocdl` dialect are not yet linked, and is left as a TODO in the logic.

Reviewers: herhut

Subscribers: mgorny, tpr, dexonsmith, mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, csigg, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #mlir, #llvm

Differential Revision: https://reviews.llvm.org/D80676
2020-06-05 09:46:39 -05:00
Nicolas Vasilache 247e185dd5 [mlir][Vector] Move temporary alloc to top of the function alloca when lowering vector_transfers
Recently introduced allocation hoisting is quite conservative on the cases when it triggers.
This revision makes it such that the allocations for vector transfer lowerings are hoisted
to the top of the function.
This should be revisited in the context of parallelism and is a temporary workaround.

Differential Revision: https://reviews.llvm.org/D81253
2020-06-05 08:45:52 -04:00
Diego Caballero 5c990d6994 [mlir] Add support for bf16 to StandardToLLVM conversion
Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D81127
2020-06-04 14:36:36 -07:00
Thomas Raoux 661235e126 [mlir][gpu] Add subgroup Id/Size/Num to GPU dialect
Add SubgroupId, SubgroupSize and NumSubgroups to GPU dialect ops and add the
lowering of those ops to SPIRV.

Differential Revision: https://reviews.llvm.org/D81042
2020-06-04 10:52:40 -07:00
Hanhan Wang 0b025d2733 [mlir][StandardToSPIRV] Handle i1 case for lowering std.zexti to SPIR-V.
Differential Revision: https://reviews.llvm.org/D80965
2020-06-03 15:01:18 -07:00
Frederik Gossen 3713314bfa [MLIR] Shape to standard dialect lowering
Add a new pass to lower operations from the `shape` to the `std` dialect.
The conversion applies only to the `size_to_index` and `index_to_size`
operations and affected types.
Other patterns will be added as needed.

Differential Revision: https://reviews.llvm.org/D81091
2020-06-03 16:17:03 +00:00
Alex Zinenko 5c5dafc534 [mlir] support materialization for 1-1 type conversions
Dialect conversion infrastructure supports 1->N type conversions by requiring
individual conversions to provide facilities to generate operations
retrofitting N values into 1 of the original type when N > 1. This
functionality can also be used to materialize explicit "cast"-like operations,
but it did not support 1->1 type conversions until now. Modify TypeConverter to
support materialization of cast operations for 1-1 conversions.

This also makes materialization specification more extensible following the
same pattern as type conversions. Instead of overloading a virtual function,
users or subclasses of TypeConversion can now register type-specific
materialization callbacks that will be called in order for the given type.

Differential Revision: https://reviews.llvm.org/D79729
2020-06-02 13:48:33 +02:00
Alex Zinenko eb8edd8526 [mlir] SCFToGPUPass: fix macros referring to LOOPS to use SCF instead
One header guard was overlooked when renaming LoopOps to SCF, rename it.
Also drop two unused macros, one of which referred to LoopOp (not "Ops",
hence the overlook).
2020-06-02 13:03:13 +02:00
MaheshRavishankar 2bcd1927dd [mlir][SCFToGPU] Remove conversions from scf.for to gpu.launch.
Keeping in the affine.for to gpu.launch conversions, which should
probably be the affine.parallel to gpu.launch conversion as well.

Differential Revision: https://reviews.llvm.org/D80747
2020-06-01 23:06:20 -07:00
Nicolas Vasilache 5f9e0466f2 [mlir][Vector] Fix vector.transfer alignment calculation
https://reviews.llvm.org/D79246 introduces alignment propagation for vector transfer operations. Unfortunately, the alignment calculation is incorrect and can result in crashes.

This revision fixes the calculation by using the natural alignment of the memref elemental type, instead of the resulting vector type.

If more alignment is desired, it can be done in 2 ways:
1. use a proper vector.type_cast to transform a memref<axbxcxdxf32> into a memref<axbxvector<cxdxf32>> giving a natural alignment of vector<cxdxf32>
2. add an alignment attribute to vector transfer operations and propagate it.

With this change the alignment in the relevant tests goes down from 128 to 4.

Lastly, a few minor cleanups are performed and the custom `isMinorIdentityMap` is deprecated.

Differential Revision: https://reviews.llvm.org/D80734
2020-05-28 17:58:51 -04:00
Stephen Neuendorffer 8b3155829a [MLIR] Fix build when NVPTX is not enabled
In this case, neither target is selected, but there is still a dependence
on the MC library (through the TargetOptions.h include)
2020-05-28 14:07:55 -07:00
Alex Zinenko 72ede60b75 [mlir][GPU] Link relevant LLVM components in GPUCommon instead of test
D80142 restructured MLIR-to-GPU-binary conversion to support multiple
targets. It also modified cmake files to link relevant LLVM components
in test/lib, which broke shared-library builds, and likely made the
conversions unusable outside mlir-opt (or other tools that link in test
library targets). Link these components to GPUCommon instead.

Differential Revision: https://reviews.llvm.org/D80739
2020-05-28 20:01:54 +02:00
Jacques Pienaar fefe4366c3 [mlir] Use ValueRange instead of ArrayRef<Value>
This allows constructing operand adaptor from existing op (useful for commonalizing verification as I want to do in a follow up).

I also add ability to use member initializers for the generated adaptor constructors for convenience.

Differential Revision: https://reviews.llvm.org/D80667
2020-05-28 09:05:24 -07:00
Wen-Heng (Jack) Chung 061fb8eb2d [mlir][gpu][mlir-cuda-runner] Refactor ConvertKernelFuncToCubin to be generic.
Make ConvertKernelFuncToCubin pass to be generic:

- Rename to ConvertKernelFuncToBlob.
- Allow specifying triple, target chip, target features.
- Initializing LLVM backend is supplied by a callback function.
- Lowering process from MLIR module to LLVM module is via another callback.
- Change mlir-cuda-runner to adopt the revised pass.
- Add new tests for lowering to ROCm HSA code object (HSACO).
- Tests for CUDA and ROCm are kept in separate directories.

Differential Revision: https://reviews.llvm.org/D80142
2020-05-28 09:08:28 -05:00
aartbik c295a65da4 [mlir] [VectorOps] Add 'vector.flat_transpose' operation
Summary:
Provides a representation of the linearized LLVM instrinsic.
With tests and lowering implementation to LLVM IR dialect.
Prepares better lowering for 2-D vector.transpose.

Reviewers: nicolasvasilache, ftynse, reidtatge, bkramer, dcaballe

Reviewed By: ftynse, dcaballe

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80419
2020-05-27 11:09:48 -07:00
MaheshRavishankar 4d6f44f5f0 [mlir][spirv] Lower allocation/deallocations of workgroup memory.
This allocation of a workgroup memory is lowered to a
spv.globalVariable. Only static size allocation with element type
being int or float is handled. The lowering does account for the
element type that are not supported in the lowered spv.module based on
the extensions/capabilities and adjusts the number of elements to get
the same byte length.

Differential Revision: https://reviews.llvm.org/D80411
2020-05-27 09:53:16 -07:00
Alex Zinenko cadb7ccf2c [mlir] SCF: provide function_ref builders for IfOp
Now that OpBuilder is available in `build` functions, it becomes possible to
populate the "then" and "else" regions directly when building the "if"
operation. This is desirable in more structured forms of builders, especially
in when conditionals are mixed with loops. Provide new `build` APIs taking
callbacks for body constructors, similarly to scf::ForOp, and replace more
clunky edsc::BlockBuilder uses with these. The original APIs remain available
and go through the new implementation.

Differential Revision: https://reviews.llvm.org/D80527
2020-05-27 16:12:58 +02:00
Benjamin Kramer a9b5edc5e2 Make mlir::Value's bool conversion operator explicit
This still allows `if (value)` while requiring an explicit cast when not
in a boolean context. This means things like `std::set<Value>` will no
longer compile.

Differential Revision: https://reviews.llvm.org/D80497
2020-05-25 18:22:00 +02:00
Wen-Heng (Jack) Chung 2cbbc266ec [mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass.
Due to similar APIs between CUDA and ROCm (HIP),
ConvertGpuLaunchFuncToCudaCalls pass could be used on both platforms with some
refactoring.

In this commit:

- Migrate ConvertLaunchFuncToCudaCalls from GPUToCUDA to GPUCommon, and rename.
- Rename runtime wrapper APIs be platform-neutral.
- Let GPU binary annotation attribute be specifiable as a PassOption.
- Naming changes within the implementation and tests.

Subsequent patches would introduce ROCm-specific tests and runtime wrapper
APIs.

Differential Revision: https://reviews.llvm.org/D80167
2020-05-21 08:53:47 -05:00
Mehdi Amini 5c3ebd7725 Revert "[mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass."
This reverts commit cdb6f05e2d.

The build is broken with:

  You have called ADD_LIBRARY for library obj.MLIRGPUtoCUDATransforms without any source files. This typically indicates a problem with your CMakeLists.txt file
2020-05-21 03:44:35 +00:00
Nicolas Vasilache 3393cc4ceb [mlir] NFC - Appease GCC 5 again.. 2020-05-20 17:58:18 -04:00
Wen-Heng (Jack) Chung cdb6f05e2d [mlir][gpu] Refactor ConvertGpuLaunchFuncToCudaCalls pass.
Due to similar APIs between CUDA and ROCm (HIP),
ConvertGpuLaunchFuncToCudaCalls pass could be used on both platforms with some
refactoring.

In this commit:

- Migrate ConvertLaunchFuncToCudaCalls from GPUToCUDA to GPUCommon, and rename.
- Rename runtime wrapper APIs be platform-neutral.
- Let GPU binary annotation attribute be specifiable as a PassOption.
- Naming changes within the implementation and tests.

Subsequent patches would introduce ROCm-specific tests and runtime wrapper
APIs.

Differential Revision: https://reviews.llvm.org/D80167
2020-05-20 16:11:48 -05:00
Nicolas Vasilache ebf14d9b6d [mlir] NFC - Appease GCC 5 again.. 2020-05-20 16:47:45 -04:00
MaheshRavishankar 0e88eb5c51 [mlir][spirv] Adapt subview legalization to the updated op semantics.
The subview semantics changes recently to allow for more natural
representation of constant offsets and strides. The legalization of
subview op for lowering to SPIR-V needs to account for this.
Also change the linearization to use the strides from the affine map
of a memref.

Differential Revision: https://reviews.llvm.org/D80270
2020-05-20 12:00:21 -07:00
Nicolas Vasilache 7c3c5b11b1 [mlir][Vector] Add option to fully unroll for VectorTransfer to SCF lowering
Summary:
Previously, the only support partial lowering from vector transfers to SCF was
going through loops. This requires a dedicated allocation and extra memory
roundtrips because LLVM aggregates cannot be indexed dynamically (for more
details see the [deep-dive](https://mlir.llvm.org/docs/Dialects/Vector/#deeperdive)).

This revision allows specifying full unrolling which removes this additional roundtrip.
This should be used carefully though because full unrolling will spill, negating the
benefits of removing the interim alloc in the first place.

Proper heuristics are left for a later time.

Differential Revision: https://reviews.llvm.org/D80100
2020-05-20 11:02:13 -04:00
Alex Zinenko a7d88a9038 [mlir] SCFToStandard: support any ops in and around the control flow ops
Originally, the SCFToStandard conversion only declared Ops from the Standard
dialect as legal after conversion. This is undesirable as it would fail the
conversion if the SCF ops contained ops from any other dialect. Furthermore,
this would be problematic for progressive lowering of `scf.parallel` to
`scf.for` after `ensureRegionTerminator` is made aware of the pattern rewriting
infrastructure because it creates temporary `scf.yield` operations declared
illegal. Change the legalization target to declare any op other than `scf.for`,
`scf.if` and `scf.parallel` legal.

Differential Revision: https://reviews.llvm.org/D80137
2020-05-20 16:12:05 +02:00
Alex Zinenko 57cbeaa8b5 [mlir] Erase or clear blocks through ConversionPatternRewriter when applicable
Multiple places in the code base were erasing Blocks or operations in them
using in-place modifications (`Block::erase` or `Block::clear`) unknown to
ConversionPatternRewriter. These operations could not be undone if the pattern
failed and could lead to inconsistent in-memory state of the IR with dangling
pointers. Use `ConversionPatternRewriter::eraseOp` and `::eraseBlock` instead.

Differential Revision: https://reviews.llvm.org/D80136
2020-05-20 16:12:05 +02:00
Nicolas Vasilache da95a0d8cc [mlir] NFC - Appease gcc 5
This should fix the error ```
VectorToSCF.cpp:238:62: error: specialization of 'template<class
ConcreteOp> mlir::LogicalResult
{anonymous}::NDTransferOpHelper<ConcreteOp>::doReplace()' in different
namespace
```
2020-05-19 22:47:51 -04:00
Benjamin Kramer 350dadaa8a Give helpers internal linkage. NFC. 2020-05-19 22:16:37 +02:00
Hanhan Wang 520a570268 [mlir][StandardToSPIRV] Fix signedness issue in bitwidth emulation.
Summary:
Previously, after applying the mask, a negative number would convert to a
positive number because the sign flag was forgotten. This patch adds two more
shift operations to do the sign extension. This assumes that we're using two's
complement.

This patch applies sign extension unconditionally when loading a unspported integer width, and it relies the pattern to do the casting because the signedness semantic is carried by operator itself.

Differential Revision: https://reviews.llvm.org/D79753
2020-05-19 11:00:01 -07:00
Alex Zinenko d1560f3956 [mlir] scf::ForOp: provide builders with callbacks for loop body
Thanks to a recent change that made `::build` functions take an instance of
`OpBuilder`, it is now possible to build operations within a region attached to
the operation about to be created. Exercise this on `scf::ForOp` by taking a
callback that populates the loop body while the loop is being created.

Additionally, provide helper functions to build perfect nests of `ForOp`s,
with support for iteration arguments. These functions provide the same
functionality as EDSC LoopNestBuilder with simpler implementation, without
relying on edsc::ScopedContext, and using `OpBuilder` in an unambiguous way.
Compatibility functions for EDSC are provided, but may be removed in the
future.

Differential Revision: https://reviews.llvm.org/D79688
2020-05-19 16:26:29 +02:00
George 3c4ef74555 Fixed a typo in the comment for allocateBuffer()
Differential Revision: https://reviews.llvm.org/D80087
2020-05-18 14:41:57 -04:00
Nicolas Vasilache 1870e787af [mlir][Vector] Add an optional "masked" boolean array attribute to vector transfer operations
Summary:
Vector transfer ops semantic is extended to allow specifying a per-dimension `masked`
attribute. When the attribute is false on a particular dimension, lowering to LLVM emits
unmasked load and store operations.

Differential Revision: https://reviews.llvm.org/D80098
2020-05-18 11:52:08 -04:00
Nicolas Vasilache 36cdc17f8c [mlir][Vector] Make minor identity permutation map optional in transfer op printing and parsing
Summary:
This revision makes the use of vector transfer operatons more idiomatic by
allowing to omit and inferring the permutation_map.

Differential Revision: https://reviews.llvm.org/D80092
2020-05-18 11:41:27 -04:00
Stephen Neuendorffer 37ce8d6ade [MLIR] Fix linkage for libMLIR.so
Generally:
1) don't use target_link_libraries() and add_mlir_library() on the same target, use LINK_LIBS PUBLIC instead.
2) don't use LINK_LIBS to specify LLVM libraries.  Use LINK_COMPONENTS instead
3) no need to link against LLVMSupport.  We pull it in by default.

Differential Revision: https://reviews.llvm.org/D80076
2020-05-17 13:46:52 -07:00
Stephen Neuendorffer 1cff8e8de7 [MLIR] LinalgToStandard: use LINK_LIBS rather than target_link_libraries.
Also, missing MLIRTransforms as a dependency.  This breaks BUILD_SHARED_LIBS=on

Differential Revision: https://reviews.llvm.org/D80035
2020-05-15 14:25:28 -07:00
aartbik b1c688dbae [mlir] [VectorOps] Implement vector.create_mask lowering to LLVM IR
Summary:
First, compact implementation of lowering to LLVM IR. A bit more
challenging than the constant mask due to the dynamic indices, of course.
I like to hear if there are more efficient ways of doing this in LLVM,
but this for now at least gives us a functional reference implementation.

Reviewers: nicolasvasilache, ftynse, bkramer, reidtatge, andydavis1, mehdi_amini

Reviewed By: nicolasvasilache

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, stephenneuendorffer, Joonsoo, grosul1, frgossen, Kayjukh, jurahul, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79954
2020-05-15 11:02:30 -07:00
Alex Zinenko 4ead2cf76c [mlir] Rename conversions involving ex-Loop dialect to mention SCF
The following Conversions are affected: LoopToStandard -> SCFToStandard,
LoopsToGPU -> SCFToGPU, VectorToLoops -> VectorToSCF. Full file paths are
affected. Additionally, drop the 'Convert' prefix from filenames living under
lib/Conversion where applicable.

API names and CLI options for pass testing are also renamed when applicable. In
particular, LoopsToGPU contains several passes that apply to different kinds of
loops (`for` or `parallel`), for which the original names are preserved.

Differential Revision: https://reviews.llvm.org/D79940
2020-05-15 10:45:11 +02:00
Alex Zinenko 7fc5f28068 [mlir] LinalgToStandard: add build dependency on MLIRPass
This is supposed to resolve the build problem with shared libraries.
2020-05-15 10:20:04 +02:00
MaheshRavishankar 0b3e478b10 [mlir][GPUToSPIRV] Use default ABI only when none of the arguments
have abi attributes.

To ensure there is no conflict, use the default ABI only when none of
the arguments have the spv.interface_var_abi attribute. This also
implies that if one of the arguments has a spv.interface_var_abi
attribute, all of them should have it as well.

Differential Revision: https://reviews.llvm.org/D77232
2020-05-14 21:48:51 -07:00
Nicolas Vasilache f1b972041a [mlir][Linalg] Start a LinalgToStandard pass and move conversion to library calls.
This revision starts decoupling the include the kitchen sink behavior of Linalg to LLVM lowering by inserting a -convert-linalg-to-std pass.

The lowering of linalg ops to function calls was previously lowering to memref descriptors by having both linalg -> std and std -> LLVM patterns in the same rewrite.

When separating this step, a new issue occurred: the layout is automatically type-erased by this process. This revision therefore introduces memref casts to perform these type erasures explicitly. To connect everything end-to-end, the LLVM lowering of MemRefCastOp is relaxed because it is artificially more restricted than the op semantics. The op semantics already guarantee that source and target MemRefTypes are cast-compatible. An invalid lowering test now becomes valid and is removed.

Differential Revision: https://reviews.llvm.org/D79468
2020-05-15 00:24:03 -04:00
Diego Caballero bc5565f9ea [mlir][Affine] Introduce affine.vector_load and affine.vector_store
This patch adds `affine.vector_load` and `affine.vector_store` ops to
the Affine dialect and lowers them to `vector.transfer_read` and
`vector.transfer_write`, respectively, in the Vector dialect.

Reviewed By: bondhugula, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D79658
2020-05-14 13:17:58 -07:00
Alex Zinenko 60f443bb3b [mlir] Change dialect namespace loop->scf
All ops of the SCF dialect now use the `scf.` prefix instead of `loop.`. This
is a part of dialect renaming.

Differential Revision: https://reviews.llvm.org/D79844
2020-05-13 19:20:21 +02:00
MaheshRavishankar 49e6c19100 [mlir][StandardToLLVM] Add SinOp to LLVM dialect and lowering of std.sin to this op.
Differential Revision: https://reviews.llvm.org/D79505
2020-05-12 23:15:25 -07:00
Nicolas Vasilache 63c0e72b2f [mlir] Revisit std.subview handling of static information.
The main objective of this revision is to change the way static information is represented, propagated and canonicalized in the SubViewOp.

In the current implementation the issue is that canonicalization may strictly lose information because static offsets are combined in irrecoverable ways into the result type, in order to fit the strided memref representation.

The core semantics of the op do not change but the parser and printer do: the op always requires `rank` offsets, sizes and strides. These quantities can now be either SSA values or static integer attributes.

The result type is automatically deduced from the static information and more powerful canonicalizations (as powerful as the representation with sentinel `?` values allows). Previously static information was inferred on a best-effort basis from looking at the source and destination type.

Relevant tests are rewritten to use the idiomatic `offset: x, strides : [...]`-form. Bugs are corrected along the way that were not trivially visible in flattened strided memref form.

Lowering to LLVM is updated, simplified and now supports all cases.
A mixed static-dynamic mode test that wouldn't previously lower is added.

It is an open question, and a longer discussion, whether a better result type representation would be a nicer alternative. For now, the subview op carries the required semantic.

Differential Revision: https://reviews.llvm.org/D79662
2020-05-12 20:04:44 -04:00
Alex Zinenko 473bdaf2e8 [mlir] Move Conversion/StandardToStandard to Dialect/StandardOps/Transforms/FuncConversions
Conversion/ folders were originally intended to store patterns for
DialectA->DialectB conversions that depend on both dialects and do not
conceptually belong to either of the dialects. As such, DialectA->DialectA
conversion does not make sense under Conversion/ and should rather live with
the dialect it operates on.

Differential Revision: https://reviews.llvm.org/D79569
2020-05-13 00:33:25 +02:00
Hanhan Wang 756d6959d7 [mlir][StandardToSPIRV] Add support for lowering index_cast to SPIR-V.
Differential Revision: https://reviews.llvm.org/D79644
2020-05-11 15:41:25 -07:00
Reid Tatge 334a4159ec [mlir][Vector] NFC - Rename vector.strided_slice into vector.extract_strided_slice
Differential Revision: https://reviews.llvm.org/D79734
2020-05-11 14:21:10 -07:00
Sean Silva 98eead8186 [mlir][Value] Add v.getDefiningOp<OpTy>()
Summary:
This makes a common pattern of
`dyn_cast_or_null<OpTy>(v.getDefiningOp())` more concise.

Differential Revision: https://reviews.llvm.org/D79681
2020-05-11 12:55:27 -07:00
Nicolas Vasilache 6ed61a26c2 [mlir] Simplify and better document std.view semantics
This [discussion](https://llvm.discourse.group/t/viewop-isnt-expressive-enough/991/2) raised some concerns with ViewOp.

In particular, the handling of offsets is incorrect and does not match the op description.
Note that with an elemental type change, offsets cannot be part of the type in general because sizeof(srcType) != sizeof(dstType).

Howerver, offset is a poorly chosen term for this purpose and is renamed to byte_shift.

Additionally, for all intended purposes, trying to support non-identity layouts for this op does not bring expressive power but rather increases code complexity.

This revision simplifies the existing semantics and implementation.
This simplification effort is voluntarily restrictive and acts as a stepping stone towards supporting richer semantics: treat the non-common cases as YAGNI for now and reevaluate based on concrete use cases once a round of simplification occurred.

Differential revision: https://reviews.llvm.org/D79541
2020-05-11 12:29:23 -04:00
Alex Zinenko c25b20c0f6 [mlir] NFC: Rename LoopOps dialect to SCF (Structured Control Flow)
This dialect contains various structured control flow operaitons, not only
loops, reflect this in the name. Drop the Ops suffix for consistency with other
dialects.

Note that this only moves the files and changes the C++ namespace from 'loop'
to 'scf'. The visible IR prefix remains the same and will be updated
separately. The conversions will also be updated separately.

Differential Revision: https://reviews.llvm.org/D79578
2020-05-11 15:04:27 +02:00
Hanhan Wang 3f07cab312 [mlir][StandardToLLVM] Add support for lowering FPToSIOp to LLVM.
Summary: Depends On D79374

Differential Revision: https://reviews.llvm.org/D79455
2020-05-11 01:29:24 -07:00
Hanhan Wang ac691c4fe7 [mlir][StandardToSPIRV] Add support for lowering FPToSIOp to SPIR-V.
Summary: Depends On D79373

Differential Revision: https://reviews.llvm.org/D79374
2020-05-11 01:27:54 -07:00
Frederik Gossen 5d5f61fc89 [MLIR] Add complex addition and substraction to the standard dialect
Complex addition and substraction are the first two binary operations on complex
numbers.
Remaining operations will follow the same pattern.

Differential Revision: https://reviews.llvm.org/D79479
2020-05-08 09:54:18 +00:00
Alex Zinenko a99f62c40a [mlir] VectorToLLVM: propagate up from replaceTransferOp
In the Vector to LLVM conversion, the `replaceTransferOp` function calls
into a type converter that may fail and suppresses the status. Change
the function to return the failure status instead, Since it is called
from a pattern, the failure can be readily propagated to the rest of
infrastructure.
2020-05-07 11:53:48 +02:00
Wen-Heng (Jack) Chung a23f190213 [mlir][vector] set alignment when lowering transfer_read and transfer_write.
When emitting masked load / store, set alignment from data layout.

Differential Revision: https://reviews.llvm.org/D79246
2020-05-07 11:44:25 +02:00
Stephen Neuendorffer e78ef9385c [MLIR] GPUToCUDA conversion: MC is only needed if NVPTX is enabled.
This patch conditionally links with MC
2020-05-05 08:55:17 -07:00
Alexander Belyaev b79751e83d [MLIR] Add conversion from AtomicRMWOp -> GenericAtomicRMWOp.
Adding this pattern reduces code duplication. There is no need to have a
custom implementation for lowering to llvm.cmpxchg.

Differential Revision: https://reviews.llvm.org/D78753
2020-05-05 10:32:13 +02:00
Stephen Neuendorffer 5469f434bb [MLIR] Reapply: Adjust libMLIR building to more closely follow libClang
This reverts commit ab1ca6e60f.
2020-05-04 20:47:57 -07:00
Stephen Neuendorffer 146192ade4 [MLIR] Normalize usage of intrinsics_gen
Portions of MLIR which depend on LLVMIR generally need to depend on
intrinsics_gen, to ensure that tablegen'd header files from LLVM are built
first.  Without this, we get errors, typically about llvm/IR/Attributes.inc
not being found.

Note that previously the Linalg Dialect depended on intrinsics_gen, but it
doesn't need to, since it doesn't use LLVMIR.

Differential Revision: https://reviews.llvm.org/D79389
2020-05-04 20:47:57 -07:00
River Riddle 1e4faf23ff [mlir][IR] Add a Region::getOps method that returns a range of immediately nested operations
This allows for walking the operations nested directly within a region, without traversing nested regions.

Differential Revision: https://reviews.llvm.org/D79056
2020-05-04 17:46:25 -07:00
Hanhan Wang 5d10613b6e [mlir][StandardToSPIRV] Emulate bitwidths not supported for store op.
Summary:
As D78974, this patch implements the emulation for store op. The emulation is
done with atomic operations. E.g., if the storing value is i8, rewrite the
StoreOp to:

 1) load a 32-bit integer
 2) clear 8 bits in the loading value
 3) store 32-bit value back
 4) load a 32-bit integer
 5) modify 8 bits in the loading value
 6) store 32-bit value back

The step 1 to step 3 are done by AtomicAnd as one atomic step, and the step 4
to step 6 are done by AtomicOr as another atomic step.

Differential Revision: https://reviews.llvm.org/D79272
2020-05-04 15:18:44 -07:00
Stephen Neuendorffer ab1ca6e60f Revert "[MLIR] Adjust libMLIR building to more closely follow libClang"
This reverts commit 4f0f436749.

This seems to show some compile dependence problems, and also breaks flang.
2020-05-04 12:40:12 -07:00
Valentin Churavy 4f0f436749 [MLIR] Adjust libMLIR building to more closely follow libClang
- Exports MLIR targets to be used out-of-tree.
- mimicks `add_clang_library` and `add_flang_library`.
- Fixes libMLIR.so

After https://reviews.llvm.org/D77515 libMLIR.so was no longer containing
any object files. We originally had a cludge there that made it work with
the static initalizers and when switchting away from that to the way the
clang shlib does it, I noticed that MLIR doesn't create a `obj.{name}` target,
and doesn't export it's targets to `lib/cmake/mlir`.

This is due to MLIR using `add_llvm_library` under the hood, which adds
the target to `llvmexports`.

Differential Revision: https://reviews.llvm.org/D78773

[MLIR] Fix libMLIR.so and LLVM_LINK_LLVM_DYLIB

Primarily, this patch moves all mlir references to LLVM libraries into
either LLVM_LINK_COMPONENTS or LINK_COMPONENTS.  This enables magic in
the llvm cmake files to automatically replace reference to LLVM components
with references to libLLVM.so when necessary.  Among other things, this
completes fixing libMLIR.so, which has been broken for some configurations
since D77515.

Unlike previously, the pattern is now that mlir libraries should almost
always use add_mlir_library.  Previously, some libraries still used
add_llvm_library.  However, this confuses the export of targets for use
out of tree because libraries specified with add_llvm_library are exported
by LLVM.  Instead users which don't need/can't be linked into libMLIR.so
can specify EXCLUDE_FROM_LIBMLIR

A common error mode is linking with LLVM libraries outside of LINK_COMPONENTS.
This almost always results in symbol confusion or multiply defined options
in LLVM when the same object file is included as a static library and
as part of libLLVM.so.  To catch these errors more directly, there's now
mlir_check_all_link_libraries.

To simplify usage of add_mlir_library, we assume that all mlir
libraries depend on LLVMSupport, so it's not necessary to separately specify
it.

tested with:
BUILD_SHARED_LIBS=on,
BUILD_SHARED_LIBS=off + LLVM_BUILD_LLVM_DYLIB,
BUILD_SHARED_LIBS=off + LLVM_BUILD_LLVM_DYLIB + LLVM_LINK_LLVM_DYLIB.

By: Stephen Neuendorffer <stephen.neuendorffer@xilinx.com>
Differential Revision: https://reviews.llvm.org/D79067

[MLIR] Move from using target_link_libraries to LINK_LIBS

This allows us to correctly generate dependencies for derived targets,
such as targets which are created for object libraries.

By: Stephen Neuendorffer <stephen.neuendorffer@xilinx.com>
Differential Revision: https://reviews.llvm.org/D79243

Three commits have been squashed to avoid intermediate build breakage.
2020-05-04 11:40:46 -07:00
Frederik Gossen 031265ad8a [MLIR] Add complex numbers to standard dialect
Add `CreateComplexOp`, `ReOp`, and `ImOp` to the standard dialect.
This is the first step to support complex numbers.

Differential Revision: https://reviews.llvm.org/D79159
2020-05-04 14:04:28 +00:00
Wen-Heng (Jack) Chung bc23c1d85e [mlir][rocdl] add rocdl.barier op.
- Add rocdl.barrier op.
- Lower gpu.barier to rocdl.barrier in -convert-gpu-to-rocdl.

Differential Revision: https://reviews.llvm.org/D79126
2020-05-04 10:35:01 +02:00
River Riddle 2265009fbe [mlir][GPUOpsLowering] Add missing include for FormatVariadic 2020-05-01 15:58:20 -07:00
Benjamin Kramer f9223d47e4 Remove unused variable. NFC. 2020-05-01 14:29:58 +02:00
MaheshRavishankar 43b89ecdb9 [mlir] Add sine operation to Standard dialect.
Also add lowering of sine operation to SPIR-V dialect.

Differential Revision: https://reviews.llvm.org/D79102
2020-04-30 22:15:42 -07:00
Hanhan Wang be0ad5b034 [mlir][StandardToSPIRV] Add support for lowering integer casting.
Summary:
Maps ZeroExtendIOp and TruncateIOp to spirv::UConvertOp and spirv::SConvertOp.

Depends On D78974

Differential Revision: https://reviews.llvm.org/D79143
2020-04-30 19:29:31 -07:00
Hanhan Wang 6601b65aed [mlir][StandardToSPIRV] Emulate bitwidths not supported for load op.
Summary:
The current implementation in SPIRVTypeConverter just unconditionally turns
everything into 32-bit if it doesn't meet the requirements of extensions or
capabilities. In this case, we can load a 32-bit value and then do bit
extraction to get the value.

Differential Revision: https://reviews.llvm.org/D78974
2020-04-30 19:27:45 -07:00
Wen-Heng (Jack) Chung 9ad5e57316 [mlir][nvvm][rocdl] refactor NVVM and ROCDL dialect. NFC.
- Extract common logic between -convert-gpu-to-nvvm and -convert-gpu-to-rocdl.
- Cope with the fact that alloca operates on different addrspaces between NVVM
  and ROCDL.
- Modernize unit tests for ROCDL dialect.

Differential Revision: https://reviews.llvm.org/D79021
2020-05-01 00:13:26 +02:00
Nicolas Vasilache 0d61dcf606 [mlir][EDSC] Make use of InsertGuard
Summary:
This revision cleans up a layer of complexity in ScopedContext and uses InsertGuard instead of previously manual bookkeeping.
The method `getBuilder` is renamed to `getBuilderRef` and spurious copies of OpBuilder are tracked.

This results in some canonicalizations not happening anymore in the Linalg matmul to vector test. This test is retired because relying on DRRs for this has been shaky at best. The solution will be better support to write fused passes in C++ with more idiomatic pattern composition and application.

Differential Revision: https://reviews.llvm.org/D79208
2020-04-30 18:04:31 -04:00
aartbik 6937251f01 [mlir] [VectorOps] Included i1 support for vector.print
Summary:
Added boolean support to vector.print.
Useful for upcoming "mask" tests.

Reviewers: ftynse, nicolasvasilache, andydavis1

Reviewed By: andydavis1

Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, grosul1, frgossen, Kayjukh, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79198
2020-04-30 14:56:26 -07:00
Nicolas Vasilache 7a80139059 [mlir][Vector] Provide progressive lowering of masked n-D vector transfers
This revision allows masked vector transfers with m-D buffers and n-D vectors to
progressively lower to m-D buffer and 1-D vector transfers.

For a vector.transfer_read, assuming a `memref<(leading_dims) x (major_dims) x (minor_dims) x type>` and a `vector<(minor_dims) x type>` are involved in the transfer, this generates pseudo-IR resembling:
```
     if (any_of(%ivs_major + %offsets, <, major_dims)) {
       %v = vector_transfer_read(
         {%offsets_leading, %ivs_major + %offsets_major, %offsets_minor},
          %ivs_minor):
         memref<(leading_dims) x (major_dims) x (minor_dims) x type>,
         vector<(minor_dims) x type>;
     } else {
       %v = splat(vector<(minor_dims) x type>, %fill)
     }
```

Differential Revision: https://reviews.llvm.org/D79062
2020-04-29 21:28:27 -04:00
MaheshRavishankar 1c12a95d9c [mlir][StandardToSPIRV] Handle conversion of cmpi operation with i1
type operands.

The instructions used to convert std.cmpi cannot have i1 types
according to SPIR-V specification. A different set of operations are
specified in the SPIR-V spec for comparing boolean types. Enhance the
StandardToSPIRV lowering to target these instructions when operands to
std.cmpi operation are of i1 type.

Differential Revision: https://reviews.llvm.org/D79049
2020-04-29 10:09:03 -07:00
Wen-Heng (Jack) Chung f2b505a459 [mlir][std] allow subview take memrefs from non-zero addrspaces.
On certain targets std.subview should be able to take memrefs from non-zero
addrspaces. Improve lowering logic to llvm dialect and amend the tests.

Differential Revision: https://reviews.llvm.org/D79024
2020-04-29 17:19:27 +02:00
Wen-Heng (Jack) Chung be16075bfc [mlir][vector] let transfer_read and transfer_write take non-zero addrspace.
Enhance lowering logic and tests so vector.transfer_read and
vector.transfer_write take memrefs on non-zero addrspaces.

Differential Revision: https://reviews.llvm.org/D79023
2020-04-29 17:11:48 +02:00
Nicolas Vasilache 0c02106058 [mlir][EDSC] Retire OperationHandle
OperationHandle mostly existed to mirror the behavior of ValueHandle.
This has become unnecessary and can be retired.

Differential Revision: https://reviews.llvm.org/D78692
2020-04-29 00:32:44 -04:00
Alex Zinenko bb1d976feb [mlir][flang] use OpBuilder& instead of Builder* in <Op>::build methods
As we start defining more complex Ops, we increasingly see the need for
Ops-with-regions to be able to construct Ops within their regions in
their ::build methods. However, these methods only have access to
Builder, and not OpBuilder. Creating a local instance of OpBuilder
inside ::build and using it fails to trigger the operation creation
hooks in derived builders (e.g., ConversionPatternRewriter). In this
case, we risk breaking the logic of the derived builder. At the same
time, OpBuilder::create, which is by far the largest user of ::build
already passes "this" as the first argument, so an OpBuilder instance is
already available.

Update all ::build methods in all Ops in MLIR and Flang to take
"OpBuilder &" instead of "Builder *". Note the change from pointer and
to reference to comply with the common style in MLIR, this also ensures
all other users must change their ::build methods.

Differential Revision: https://reviews.llvm.org/D78713
2020-04-28 10:42:08 +02:00
Nicolas Vasilache b2c79c50ed [mlir][VectorOps] Extend VectorTransfer lowering to n-D memref with minor identity map
Summary: This revision extends the lowering of vector transfers to work with n-D memref and 1-D vector where the permutation map is an identity on the most minor dimensions (1 for now).

Differential Revision: https://reviews.llvm.org/D78925
2020-04-27 11:20:55 -04:00
Alexander Belyaev f31db760b3 [MLIR] Replace splitBlock() with createBlock in GenericAtomicRMWOp lowering.
`addArgument()` is not undoable and should not be used in
ConversionPattern, therefore replacing `splitBlock()` with
`createBlock()`, that creates a block with specified args.

Differential Revision: https://reviews.llvm.org/D78731
2020-04-25 17:42:45 +02:00
Nicolas Vasilache 367229e100 [mlir][EDSC] Retire ValueHandle
The EDSC discussion [thread](https://llvm.discourse.group/t/evolving-builder-apis-based-on-lessons-learned-from-edsc/879) points out that ValueHandle has become an unnecessary level of abstraction since MLIR switch from `Value *` to `Value` everywhere.

This revision removes this level of indirection.
2020-04-23 11:01:16 -04:00
Alexander Belyaev 21caba599e [MLIR] Lower GenericAtomicRMWOp to llvm.cmpxchg.
Summary:
Lowering is pretty much a copy of AtomicRMWOp -> llvm.cmpxchg
pattern.

Differential Revision: https://reviews.llvm.org/D78647
2020-04-23 09:29:34 +02:00
Denis Khalikov 1009177d49 [mlir][vulkan-runner] Add support for integer types.
Summary:
Add support for memrefs with element type as integer type
and simple test.

Differential Revision: https://reviews.llvm.org/D78560
2020-04-22 19:42:39 +03:00
Frederik Gossen 0372db05bb [MLIR] Use nested symbol to identify kernel in `LaunchFuncOp`.
Summary:
Use a nested symbol to identify the kernel to be invoked by a `LaunchFuncOp` in the GPU dialect.
This replaces the two attributes that were used to identify the kernel module and the kernel within seperately.

Differential Revision: https://reviews.llvm.org/D78551
2020-04-22 07:44:29 +00:00
Stephan Herhut c22876b550 [MLIR] Add extra locking during cubin generation.
We also need to lock the LLVMDialect mutex when initializing
LLVM targets or destroying llvm modules concurrently. Added another
scoped lock to that effect.

Differential Revision: https://reviews.llvm.org/D78580
2020-04-22 08:57:45 +02:00
Denis Khalikov a48f0a3c7e [mlir][vulkan-runner] Simplify vulkan launch call op.
Summary:
Workgroup size is written into the kernel. So to properly modelling
vulkan launch, we have to skip local workgroup size for vulkan launch
call op.

Differential Revision: https://reviews.llvm.org/D78307
2020-04-18 16:49:47 +03:00
aartbik 186709c6e0 [mlir] [VectorOps] Progressive lowering of vector.broadcast
Summary:
Rather than having a full, recursive, lowering of vector.broadcast
to LLVM IR, it is much more elegant to have a progressive lowering
of each vector.broadcast into a lower dimensional vector.broadcast,
until only elementary vector operations remain. This results
in more elegant, step-wise code, that is easier to understand.
Also makes some optimizations in the generated code.

Reviewers: nicolasvasilache, mehdi_amini, andydavis1, grosul1

Reviewed By: nicolasvasilache

Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, grosul1, frgossen, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78071
2020-04-16 21:02:27 -07:00
Stephen Neuendorffer 314f00a034 [MLIR][cmake] Remove redundant add_dependencies()
Libraries declared as target_link_libraries() do not also need
to be declared as dependencies using add_dependencies().

Differential Revision: https://reviews.llvm.org/D78320
2020-04-16 14:41:54 -07:00
Stephen Neuendorffer f061295732 [MLIR] Complete refactoring of Affine dialect into sub-libraries.
There were some unused CMakeFiles for Affine/IR and Affine/EDSC.
This change builds separate MLIRAffineOps and MLIRAffineEDSC libraries
using those CMakeFiles.  This combination replaces the old MLIRAffine
library.

Differential Revision: https://reviews.llvm.org/D78317
2020-04-16 13:41:17 -07:00
Stephan Herhut 69040d5b0b [MLIR] Allow for multiple gpu modules during translation.
This change makes the ModuleTranslation threadsafe by locking on the
LLVMContext. Furthermore, we now clone the llvm module into a new
context when compiling to PTX similar to what the OrcJit does.

Differential Revision: https://reviews.llvm.org/D78207
2020-04-16 14:18:31 +02:00
Jeremy Bruestle 9f3ab92ec8 [MLIR] Improve support for 0-dimensional Affine Maps.
Summary:
Modified AffineMap::get to remove support for the overload which allowed
an ArrayRef of AffineExpr but no context (and gathered the context from a
presumed first entry, resulting in bugs when there were 0 results).

Instead, we support only a ArrayRef and a context, and a version which
takes a single AffineExpr.

Additionally, removed some now needless case logic which previously
special cased which call to AffineMap::get to use.

Reviewers: flaub, bondhugula, rriddle!, nicolasvasilache, ftynse, ulysseB, mravishankar, antiagainst, aartbik

Subscribers: mehdi_amini, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, bader, grosul1, frgossen, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78226
2020-04-15 14:15:02 -07:00
River Riddle ebf190fcda [llvm][ADT] Move TypeSwitch class from MLIR to LLVM
This class implements a switch-like dispatch statement for a value of 'T' using dyn_cast functionality. Each `Case<T>` takes a callable to be invoked if the root value isa<T>, the callable is invoked with the result of dyn_cast<T>() as a parameter.

Differential Revision: https://reviews.llvm.org/D78070
2020-04-14 15:14:41 -07:00
Tres Popp 58516718fc [MLIR] Constant fold multiplies in deriveStaticUpperBound.
Summary:
This operation occurs during collapseParallelLoops, so we constant fold
them also to allow more situations of determining a loop invariant upper
bound when lowering to the GPU dialect from the Loop dialect.

Differential Revision: https://reviews.llvm.org/D77723
2020-04-14 13:04:16 +02:00
Christopher Tetreault eab73dfed9 [SVE] Change return type of getNumElements to unsigned
Reviewers: efriedma, sdesmalen, craig.topper, dexonsmith

Reviewed By: efriedma, sdesmalen

Subscribers: tschuett, hiraditya, rkruppe, psnobl, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, Joonsoo, grosul1, frgossen, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77763
2020-04-13 16:24:18 -07:00
River Riddle d3588d0814 [mlir][NFC] Replace mlir/Support/Functional.h with llvm equivalents.
Summary: Functional.h contains many different methods that have a direct, and more efficient, equivalent in LLVM. This revision replaces all usages with the LLVM equivalent, and removes the header. This is part of larger cleanup, pr45513, merging MLIR support facilities into LLVM.

Differential Revision: https://reviews.llvm.org/D78053
2020-04-13 14:22:12 -07:00
Lei Zhang a9cb529a84 [mlir][spirv] NFC: use Optional to replace SPV_Optional
Differential Revision: https://reviews.llvm.org/D78046
2020-04-13 15:44:06 -04:00
Chris Lattner 74e6a5b2a3 Eliminate all uses of Identifier::is() in the source tree, this doesn't remove the definition of it (yet). NFC.
Reviewers: mravishankar, antiagainst, herhut, rriddle!

Subscribers: jholewinski, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, csigg, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, bader, grosul1, frgossen, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78042
2020-04-13 11:49:31 -07:00
Christopher Tetreault 92dde8a657 Clean up usages of asserting vector getters in Type
Summary:
Remove usages of asserting vector getters in Type in preparation for the
VectorType refactor. The existence of these functions complicates the
refactor while adding little value.

Reviewers: rriddle, efriedma, sdesmalen

Reviewed By: sdesmalen

Subscribers: frgossen, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, Joonsoo, grosul1, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77258
2020-04-10 13:46:18 -07:00
Uday Bondhugula a5b9316b24 [MLIR][NFC] applyPatternsGreedily -> applyPatternsAndFoldGreedily
Rename mlir::applyPatternsGreedily -> applyPatternsAndFoldGreedily. The
new name is a more accurate description of the method - it performs
both, application of the specified patterns and folding of all ops in
the op's region irrespective of whether any patterns have been supplied.

Differential Revision: https://reviews.llvm.org/D77478
2020-04-10 12:55:21 +05:30
Nicolas Vasilache 8345b86d9a [mlir][Vector] Add lowering of 1-D vector transfer_read/write to masked load/store
Summary:
This revision adds support to lower 1-D vector transfers to LLVM.
A mask of the vector length is created that compares the base offset + linear index to the dim of the vector.
In each position where this does not overflow (i.e. offset + vector index < dim), the mask is set to 1.

A notable fact is that the lowering uses llvm.dialect_cast to allow writing code in the simplest form by targeting the simplest mix of vector and LLVM dialects and
letting other conversions kick in.

Differential Revision: https://reviews.llvm.org/D77703
2020-04-09 16:17:05 -04:00
River Riddle bd1ccfe6df [mlir] Add a new RewritePattern::hasBoundedRewriteRecursion hook.
Summary: Some pattern rewriters, like dialect conversion, prohibit the unbounded recursion(or reapplication) of patterns on generated IR. Most patterns are not written with recursive application in mind, so will generally explode the stack if uncaught. This revision adds a hook to RewritePattern, `hasBoundedRewriteRecursion`, to signal that the pattern can safely be applied to the generated IR of a previous application of the same pattern. This allows for establishing a contract between the pattern and rewriter that the pattern knows and can handle the potential recursive application.

Differential Revision: https://reviews.llvm.org/D77782
2020-04-09 12:42:28 -07:00
Uday Bondhugula d314b7d5ca [MLIR] ShapedType accessor minor fixes + add isDynamicDim accessor
Minor fixes and cleanup for ShapedType accessors, use
ShapedType::kDynamicSize, add ShapedType::isDynamicDim.

Differential Revision: https://reviews.llvm.org/D77710
2020-04-09 08:47:50 +05:30
River Riddle 400ad6f95d [mlir] Eliminate the remaining usages of cl::opt instead of PassOption.
Summary: Pass options are a better choice for various reasons and avoid the need for static constructors.

Differential Revision: https://reviews.llvm.org/D77707
2020-04-08 13:05:08 -07:00
Uday Bondhugula 3156b5422e [MLIR] Fix more gcc-5 build issues from D77528
820c420d4e1c630b5ead285917c6ecdd2f5092ad did not really fix all build
issues by D77528. This gets rid of two unnecessary 'using' declarations.

Differential Revision: https://reviews.llvm.org/D77726
2020-04-08 19:21:50 +05:30
Uday Bondhugula a59008a3a5 [MLIR] Fix gcc-5 build failure cause by D77528
Fix gcc-5 build failure cause by D77528

Differential Revision: https://reviews.llvm.org/D77719
2020-04-08 17:27:12 +05:30
Uday Bondhugula 01d97a3549 [MLIR] Add support to use aligned_alloc to lower AllocOp from std to llvm
Support to recognize and deal with aligned_alloc was recently added to
LLVM's TLI/MemoryBuiltins and its various optimization passes. This
revision adds support for generation of aligned_alloc's when lowering
AllocOp from std to LLVM. Setting 'use-aligned_alloc=1' will lead to
aligned_alloc being used for all heap allocations. An alignment and size
that works with the constraints of aligned_alloc is chosen.

Using aligned_alloc is preferable to "using malloc and adjusting the
allocated pointer to align for indexing" because the pointer access
arithmetic done for the latter only makes it harder for LLVM passes to
deal with for analysis, optimization, attribute deduction, and rewrites.

Differential Revision: https://reviews.llvm.org/D77528
2020-04-08 15:10:19 +05:30
River Riddle 1834ad4a69 [mlir][Pass] Update the PassGen to generate base classes instead of utilities
Summary:
This is much cleaner, and fits the same structure as many other tablegen backends. This was not done originally as the CRTP in the pass classes made it overly verbose/complex.

Differential Revision: https://reviews.llvm.org/D77367
2020-04-07 14:08:52 -07:00
River Riddle 80aca1eaf7 [mlir][Pass] Remove the use of CRTP from the Pass classes
This revision removes all of the CRTP from the pass hierarchy in preparation for using the tablegen backend instead. This creates a much cleaner interface in the C++ code, and naturally fits with the rest of the infrastructure. A new utility class, PassWrapper, is added to replicate the existing behavior for passes not suitable for using the tablegen backend.

Differential Revision: https://reviews.llvm.org/D77350
2020-04-07 14:08:52 -07:00
River Riddle 722f909f7a [mlir][Pass][NFC] Replace usages of ModulePass with OperationPass<ModuleOp>
ModulePass doesn't provide any special utilities and thus doesn't give enough benefit to warrant a special pass class. This revision replaces all usages with the more general OperationPass.

Differential Revision: https://reviews.llvm.org/D77339
2020-04-07 14:08:52 -07:00
Uday Bondhugula 7023f4b4cb [MLIR] Introduce std.alloca op
Introduce the alloca op for stack memory allocation. When converting to the
LLVM dialect, this is lowered to an llvm.alloca. Refactor the std to
llvm conversion for alloc op to reuse with alloca. Drop useAlloca option
with alloc op lowering.

Differential Revision: https://reviews.llvm.org/D76602
2020-04-07 15:45:07 +05:30
Nicolas Vasilache 8f229989d5 [mlir][Linalg] Add a linalg.tensor_reshape to operate on tensors
Summary:
This revision adds a tensor_reshape operation that operates on tensors.
In the tensor world the constraints are less stringent and we can allow more
arbitrary dynamic reshapes, as long as they are contractions.

The expansion of a dynamic dimension into multiple dynamic dimensions is under-specified and is punted on for now.

Differential Revision: https://reviews.llvm.org/D77360
2020-04-06 11:19:17 -04:00
Alexander Belyaev 51e3709c2b [MLIR] Don't insert YieldOp for non-void loop.for by default.
The ForOp::build ensures that there is a block terminator which is great for
the default use case when there are no iter_args and loop.for returns no
results.  In non-zero results case we always need to call replaceOpWithNewOp
which is not the nicest thing in the world. We can stop inserting YieldOp when
iter_args is non-empty. IfOp::build already behaves similarly.
2020-04-05 11:48:22 +02:00
Kazuaki Ishizaki 5aacce3db2 [mlir] NFC: Fix trivial typo
Differential Revision: https://reviews.llvm.org/D77473
2020-04-05 11:30:30 +09:00
Alex Zinenko 340e1b2077 [mlir] LoopToStandard conversion: support "if/else" with results
Summary:
A recent extension allowed the `loop.if` operation to return results yielded by
its regions. However, such operations could not be lowered to a CFG of standard
operations because it would have required to modify the argument list of a
block, which is not allowed in a conversion pattern. Now that the conversion
infrastructure supports block creation, use it to create a block with an
argument list that dominates the operations following the `loop.if` and forward
the results as arguments of this block.

Depends On D77416

Differential Revision: https://reviews.llvm.org/D77418
2020-04-03 23:49:03 +02:00
Nicolas Vasilache e33a636e26 [mlir][Linalg] Employ finer-grained control of C interface emission
Summary:
Linalg makes it possible to interface codegen with externally precompiled HPC libraries. The mechanism to allow such interop uses a normalized ABI and the emission of C interface wrappers.

The mechanism controlling these C interface emission is too aggressive and makes it very easy to obtained undefined symbols for external function (e.g. the ones coming from libm).

This revision uses the newly introduced llvm.emit_c_interface function attribute which allows controlling this behavior at a function granularity. As a consequence LinalgToLLVM does not need to activate the C wrapper emission when adding the StdToLLVM patterns.

Differential Revision: https://reviews.llvm.org/D77364
2020-04-03 16:14:53 -04:00
Denis Khalikov 0718e3ae31 [mlir][vulkan-runner] Add support for 3D memrefs.
Summary:
Add support for 3D memrefs in mlir-vulkan-runner and simple test.

Differential Revision: https://reviews.llvm.org/D77157
2020-04-03 15:10:40 +03:00
Nicolas Vasilache add9f1a5dc [mlir][LLVM] Finer-grained control for C interface emission
C interface emission is controlled by a flag and has coarse granularity.
With this coarse control, interfaces are emitted for all external functions.
This makes is easy to get undefined symbols.

This revision adds support for controlling per-function emission with an "emit_c_interface" attribute.
2020-04-02 13:07:10 -04:00
Alex Zinenko 802bb8b5c2 [mlir] StandardToLLVM conversion: remove dead code
This code is unused since 04ed07bc17,
but it was not removed in that commit.
2020-04-02 18:52:40 +02:00
MaheshRavishankar 9b31e595d7 [mlir] Modify GPU to SPIR-V conversion to respect
spv.interface_var_abi attributes if it exists already.

Differential Revision: https://reviews.llvm.org/D77195
2020-04-01 09:34:49 -07:00
River Riddle 9a277af2d4 [mlir][Pass] Add support for generating pass utilities via tablegen
This revision adds support for generating utilities for passes such as options/statistics/etc. that can be inferred from the tablegen definition. This removes additional boilerplate from the pass, and also makes it easier to remove the reliance on the pass registry to provide certain things(e.g. the pass argument).

Differential Revision: https://reviews.llvm.org/D76659
2020-04-01 02:10:46 -07:00
River Riddle 3dddd8969f [mlir][Pass] Move the registration of conversion passes to tablegen
This removes the need to statically register conversion passes, and also puts all of the conversions within one centralized file.

Differential Revision: https://reviews.llvm.org/D76658
2020-04-01 02:10:46 -07:00
Hanhan Wang 69ddee1d2a [mlir][Linalg] Introduce linalg.pooling_min/max/sum op.
Summary:
Performs an N-D pooling operation similarly to the description in the TF
documentation:
https://www.tensorflow.org/api_docs/python/tf/nn/pool

Different from the description, this operation doesn't perform on batch and
channel. It only takes tensors of rank `N`.

```
  output[x[0], ..., x[N-1]] =
    REDUCE_{z[0], ..., z[N-1]}
      input[
            x[0] * strides[0] - pad_before[0] + dilation_rate[0]*z[0],
            ...
            x[N-1]*strides[N-1] - pad_before[N-1] + dilation_rate[N-1]*z[N-1]
            ],
```

The required optional arguments are:
  - strides: an i64 array specifying the stride (i.e. step) for window
    loops.
  - dilations: an i64 array specifying the filter upsampling/input
    downsampling rate
  - padding: an i64 array of pairs (low, high) specifying the number of
    elements to pad along a dimension.

If strides or dilations attributes are missing then the default value is
one for each of the input dimensions. Similarly, padding values are zero
for both low and high in each of the dimensions, if not specified.

Differential Revision: https://reviews.llvm.org/D76414
2020-03-31 21:21:54 -07:00
Uday Bondhugula 5f9bf3f656 [MLIR][NFC] Move test/Transforms/lower-affine.mlir -> test/Conversion
Move lower-affine.mlir from test/Transforms to
test/Conversion/AffineToStandard/. Other related NFC.

Signed-off-by: Uday Bondhugula <uday@polymagelabs.com>

Differential Revision: https://reviews.llvm.org/D77008
2020-03-31 23:34:50 +05:30
Aaron Smith 6dab806712 [mlir] Add exp2 conversion to llvm.intr.exp2 2020-03-29 01:23:08 -07:00
Kazuaki Ishizaki e5a8512655 [mlir] NFC: fix trivial typo in source files
Summary: fix trivial typos in the source files

Reviewers: mravishankar, antiagainst, nicolasvasilache, herhut, rriddle, aartbik

Reviewed By: antiagainst, rriddle

Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, csigg, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, Joonsoo, bader, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76876
2020-03-28 10:12:49 +09:00
Stephan Herhut ac9d742bbe [MLIR][LLVM] Make index type bitwidth configurable.
This change adds a new option to the StandardToLLVM lowering to configure
the bitwidth of the index type independently of the target architecture's
pointer size.

Differential revision: https://reviews.llvm.org/D76353
2020-03-27 12:42:54 +01:00
Denis Khalikov 8f4ab8c7d7 [mlir][vulkan-runner] Add support for 2D memref.
Summary:
This patch adds support for 2D memref in mlir-vulkan-runner.

Differential Revision: https://reviews.llvm.org/D76737
2020-03-27 13:59:17 +03:00
Alex Zinenko c16c07d4b9 [mlir] StandardToLLVM: use template aliases instead of dummy classes
Multiple operation conversions from the Standard dialect to the LLVM dialect
are trivial one-to-one conversions that use only the pattern defined in base
utility classes such as OneToOneConvertToLLVMPattern and
VectorConvertToLLVMPattern. Use template aliases ("using" declarations) instead
of creating derived classes without new functionality.
2020-03-27 11:14:44 +01:00
Alex Zinenko 04ed07bc17 [mlir] StandardToLLVM: clean up conversion patterns for vector operations
Summary:
Provide a public VectorConvertToLLVMPattern utility class to implement
conversions with automatic unrolling of operation on multidimensional vectors
to lists of operations on single-dimensional vectors when lowering to the LLVM
dialect. Drop the template-based check on the number of operands since the
actual implementation does not depend on the operand number anymore. This check
only creates spurious concepts (UnaryOpLowering, BinaryOpLowering, etc).

Differential Revision: https://reviews.llvm.org/D76865
2020-03-26 18:24:10 +01:00
Alex Zinenko 987fbae0ad [mlir] StandardToLLVM: make one-to-one convresion pattern publicly available
Summary:
The Standard-to-LLVM dialect convresion has a set of utility classes that
simplify conversions, including patterns that provide one-to-one conversion
operation conversion with optional result packing. Expose these classes in a
public header so that conversions other than Standard-to-LLVM (e.g. vectors, or
LLVM-based intrinsics) could also use them. Since the patterns are implemented
as class templates and in order to keep the code size limited, keep the
implementation private by resorting to op identifiers instead of template-based
builders.

Differential Revision: https://reviews.llvm.org/D76864
2020-03-26 18:24:07 +01:00
Marcel Koester 2b529a396d [mlir] Removed TanHOp lowering from ConvertStandardToLLVM since there is no reasonable TanH representation in LLVM.
Summary: The current ConvertStandardToLLVM phase lowers the standard TanHOp to function calls to external tanh symbols. However, this leads to misunderstandings since these external symbols are not defined anywhere. This commit removes the TanHOp lowering functionality from ConvertStandardToLLVM, adapts the LowerGpuOpsToNVVMOps and LowerGpuOpsToROCDLOps passes and adjusts the affected test cases.

Reviewers: mravishankar, herhut

Subscribers: jholewinski, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, csigg, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, Joonsoo, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D75509
2020-03-25 16:43:45 +01:00
MaheshRavishankar 46bb6613a3 [mlir][GPU] Use StructAttr to drive lowering from loop.parallel to
gpu.launch

Current implementation of lowering from loop.parallel to gpu.launch
uses a DictionaryAttr to specify the mapping. Moving this attribute to
be auto-generated from specification as a StructAttr. This simplifies
a lot the logic of looking up and creating this attribute.

Differential Revision: https://reviews.llvm.org/D76165
2020-03-24 16:16:55 -07:00
Hanhan Wang 58cdb8bff0 [mlir][StandardToSPIRV] Add support for lowering unary ops
Differential Revision: https://reviews.llvm.org/D76661
2020-03-24 09:16:10 -04:00
Benjamin Kramer b6732056a4 Make helpers static. NFC. 2020-03-24 13:43:00 +01:00
Uday Bondhugula 56e1c20bfd [MLIR][NFC] rename ConvertStandardToLLVM, ConvertLoopToStandard to drop Convert prefix
This is in line with the convention agreed on https://llvm.discourse.group/t/rfc-canonical-file-paths-to-dialects/621

Signed-off-by: Uday Bondhugula <uday@polymagelabs.com>

Differential Revision: https://reviews.llvm.org/D76583
2020-03-23 08:24:11 +05:30
River Riddle e8f5c072f6 [mlir] Move the testing pass for GpuKernelToCubin to the test/ directory
Summary:
This removes the static pass registration, and also cleans up some lingering technical debt.

Differential Revision: https://reviews.llvm.org/D76554
2020-03-22 03:38:09 -07:00
River Riddle e9482ed194 [mlir] Move several static cl::opts to be pass options instead.
This removes the reliance on global options, and also simplifies the pass registration.

Differential Revision: https://reviews.llvm.org/D76552
2020-03-22 03:16:21 -07:00
Rob Suderman e708471395 [mlir][NFC] Cleanup AffineOps directory structure
Summary:
Change AffineOps Dialect structure to better group both IR and Tranforms. This included extracting transforms directly related to AffineOps. Also move AffineOps to Affine.

Differential Revision: https://reviews.llvm.org/D76161
2020-03-20 14:23:43 -07:00
Nicolas Vasilache 462db62053 [mlir][AVX512] Start a primitive AVX512 dialect
The Vector Dialect [document](https://mlir.llvm.org/docs/Dialects/Vector/) discusses the vector abstractions that MLIR supports and the various tradeoffs involved.

One of the layer that is missing in OSS atm is the Hardware Vector Ops (HWV) level.

This revision proposes an AVX512-specific to add a new Dialect/Targets/AVX512 Dialect that would directly target AVX512-specific intrinsics.

Atm, we rely too much on LLVM’s peephole optimizer to do a good job from small insertelement/extractelement/shufflevector. In the future, when possible, generic abstractions such as VP intrinsics should be preferred.

The revision will allow trading off HW-specific vs generic abstractions in MLIR.

Differential Revision: https://reviews.llvm.org/D75987
2020-03-20 14:11:57 -04:00
River Riddle 2c1ba63ede [mlir] Change missed usage PatternMatchResult to LogicalResult 2020-03-18 21:22:24 -07:00
Rob Suderman cd1212deff [mlir] Introduced CallOp Dialect Conversion
Summary:
Utility to perform CallOp Dialect conversion, specifically handling cases where
an argument type has changed and the corresponding CallOp needs to be updated.

Differential Revision: https://reviews.llvm.org/D76326
2020-03-18 20:07:38 -07:00
Lei Zhang 73431a492b [mlir][spirv] Consolidate std.constant to spv.constant conversions
This commit merges the DRR pattern for std.constant to spv.constant
conversion into the C++ OpConversionPattern. This allows us to have
remove the DRR pattern file. Along the way, this commit enhanced
std.constant to spv.constant conversion to consider type conversions,
which means converting the underlying attributes if necessary.

Differential Revision: https://reviews.llvm.org/D76246
2020-03-18 20:11:05 -04:00
Lei Zhang ffd4583c6a [mlir][spirv] Change standard op patterns to consider type conversion
Previously we have a few patterns that were written with DRR. DRR
at the moment does not work nicely with dialect conversion framework.
It generates normal RewritePatterns, while the dialect conversion
framework requires ConversionPatterns to take into consideration
the type conversion. So this commit starts to change existing DRR
patterns for standard ops to OpConversionPattern to incorporate the
SPIR-V type conversion. All patterns are converted except the one
for constant ops, which will happen in a subsequent commit.

Differential Revision: https://reviews.llvm.org/D76245
2020-03-18 20:11:05 -04:00
Lei Zhang 58df5e6d9a [mlir][spirv] Plumbing target environment into type converter
This commit unifies target environment queries into a new wrapper
class spirv::TargetEnv and shares across various places needing
the functionality. We still create multiple instances of TargetEnv
though given the parent components (type converters, passes,
conversion targets) have different lifetimes.

In the meantime, LowerABIAttributesPass is updated to take into
consideration the target environment, which requires updates to
tests to provide that.

Differential Revision: https://reviews.llvm.org/D76242
2020-03-18 20:11:05 -04:00
Lei Zhang 3b35f9d8b5 [mlir][spirv] Use memref memory space for storage class
Previously in SPIRVTypeConverter, we always convert memref types
to StorageBuffer regardless of their memory spaces. This commit
fixes that to let the conversion to look into memory space
properly. For this purpose, a mapping between SPIR-V storage class
and memref memory space is introduced. The mapping is arbitary
decided at the moment and the hope is that we can leverage
string memory space later to be more clear.

Now spv.interface_var_abi cannot contain storage class unless it's
attached to a scalar value, where we need the storage class as side
channel information. Verifications and tests are properly adjusted.

Differential Revision: https://reviews.llvm.org/D76241
2020-03-18 20:11:04 -04:00
River Riddle 3145427dd7 [mlir][NFC] Replace all usages of PatternMatchResult with LogicalResult
This also replaces usages of matchSuccess/matchFailure with success/failure respectively.

Differential Revision: https://reviews.llvm.org/D76313
2020-03-17 20:21:32 -07:00
Nicolas Vasilache 2fae7878d5 [mlir][Vector] Mostly-NFC - Restructure options for lowering to LLVM Matrix Intrinsics
Summary:
This revision restructures the calling of vector transforms to make it more flexible to ask for lowering through LLVM matrix intrinsics.
This also makes sure we bail out in degenerate cases (i.e. 1) in which LLVM complains about not being able to scalarize.

Differential Revision: https://reviews.llvm.org/D76266
2020-03-17 22:58:02 -04:00
Rob Suderman 4d60f47b08 [mlir][NFC] Renamed VectorOps to Vector
Summary: Renamed VectorOps to Vector to avoid the redundant Ops suffix.

Differential Revision: https://reviews.llvm.org/D76317
2020-03-17 15:28:08 -07:00
Alex Zinenko e119980f3f [mlir] LLVM dialect: move ensureDistinctSuccessors out of std->LLVM conversion
MLIR supports terminators that have the same successor block with different
block operands, which cannot be expressed in the LLVM's phi-notation as the
block identifier is used to tell apart the predecessors. This limitation can be
worked around by branching to a new block instead, with this new block
unconditionally branching to the original successor and forwarding the
argument. Until now, this transformation was performed during the conversion
from the Standard to the LLVM dialect. This does not scale well to multiple
dialects targeting the LLVM dialect as all of them would have to be aware of
this limitation and perform the preparatory transformation. Instead, do it as a
separate pass and run it immediately before the translation.

Differential Revision: https://reviews.llvm.org/D75619
2020-03-17 15:22:14 +01:00
Denis Khalikov bfb2ce0256 [mlir][vulkan-runner] Use C-compatible wrapper emission.
A memref argument is converted into a pointer-to-struct argument
of type `{T*, T*, i64, i64[N], i64[N]}*` in the wrapper function,
where T is the converted element type and N is the memref rank.

Differential Revision: https://reviews.llvm.org/D76059
2020-03-17 07:54:41 -04:00
River Riddle bd5941b9ce [mlir] Remove the PatternState class and simplify PatternMatchResult.
Summary: PatternState was a mechanism to pass state between the match and rewrite calls of a RewritePattern. With the rise of matchAndRewrite, this class is unused and unnecessary. This revision removes PatternState and simplifies PatternMatchResult to just be a LogicalResult. A future revision will replace all usages of PatternMatchResult/matchSuccess/matchFailure with LogicalResult equivalents.

Differential Revision: https://reviews.llvm.org/D76202
2020-03-16 17:55:54 -07:00
Nicolas Vasilache bbf3ef8541 [mlir][Vector]Lower vector.contract to llvm.intr.matrix_multiply
Summary:
This revision adds lowering of vector.contract to llvm.intr.matrix_multiply.
Note that there is currently a mismatch between the MLIR vector dialect which
expects row-major layout and the LLVM matrix intrinsics which expect column
major layout.

As a consequence, we currently only match a vector.contract with indexing maps
that express column-major matrix multiplication.
Other cases would require additional transposes and it is better to wait for
LLVM intrinsics to provide a per-operation attribute that would specify which
layout is expected.

A separate integration test, not submitted to MLIR core, has independently
verified that correct execution occurs on a 2x2x2 matrix multiplication.

Differential Revision: https://reviews.llvm.org/D76014
2020-03-13 16:33:23 -04:00
aartbik a213ece30b [mlir] [VectorOps,LinAlg] Remove direct LLVM lowering for vector.broadcast
Summary:
The direct lowering of vector.broadcast into LLVM has been replaced by
progressive lowering into elementary vector ops. This also required a
small refactoring of a llvm.mlir test that used a direct vector.broadcast
operator (just to define a matmul).

Reviewers: nicolasvasilache, andydavis1, rriddle

Reviewed By: nicolasvasilache

Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76143
2020-03-13 11:42:51 -07:00
Lei Zhang 3148f10b17 [mlir][spirv] Use spv.vce in spv.module and wire up (de)serialization
This commits changes the definition of spv.module to use the #spv.vce
attribute for specifying (version, capabilities, extensions) triple
so that we can have better API and custom assembly form. Since now
we have proper modelling of the triple, (de)serialization is wired up
to use them.

With the new UpdateVCEPass, we don't need to manually specify the
required extensions and capabilities anymore when creating a spv.module.
One just need to call UpdateVCEPass before serialization to get the
needed version/extensions/capabilities.

Differential Revision: https://reviews.llvm.org/D75872
2020-03-12 19:37:45 -04:00
River Riddle 0ddba0bd59 [mlir][SideEffects] Replace HasNoSideEffect with the memory effect interfaces.
HasNoSideEffect can now be implemented using the MemoryEffectInterface, removing the need to check multiple things for the same information. This also removes an easy foot-gun for users as 'Operation::hasNoSideEffect' would ignore operations that dynamically, or recursively, have no side effects. This also leads to an immediate improvement in some of the existing users, such as DCE, now that they have access to more information.

Differential Revision: https://reviews.llvm.org/D76036
2020-03-12 14:26:15 -07:00
aartbik 078776a679 [mlir] [VectorOps] Progressively lower vector.outerproduct to LLVM
Summary:
This replaces the direct lowering of vector.outerproduct to LLVM with progressive lowering into elementary vectors ops to avoid having the similar lowering logic at several places.

NOTE1: with the new progressive rule, the lowered llvm is slightly more elaborate than with the direct lowering, but the generated assembly is just as optimized; still if we want to stay closer to the original, we should add a "broadcast on extract" to shuffle rewrite (rather than special cases all the lowering steps)

NOTE2: the original outerproduct lowering code should now be removed but some linalg test work directly on vector and contain some dead code, so this requires another CL

Reviewers: nicolasvasilache, andydavis1

Reviewed By: nicolasvasilache, andydavis1

Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D75956
2020-03-12 13:45:42 -07:00
Christian Sigg fc421d7ca3 [MLIR] Remove all-reduce lowering from GPU to NVVM. Use in-dialect lowering instead.
Reviewers: herhut, mravishankar

Reviewed By: herhut

Subscribers: merge_guards_bot, jholewinski, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, Joonsoo, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D73794
2020-03-11 15:17:54 +01:00
Valentin Clement c7380995f8 [MLIR] Add `and`, `or`, `xor`, `min`, `max` too gpu.all_reduce and the nvvm lowering
Summary:
This patch add some builtin operation for the gpu.all_reduce ops.
- for Integer only: `and`, `or`, `xor`
- for Float and Integer: `min`, `max`

This is useful for higher level dialect like OpenACC or OpenMP that can lower to the GPU dialect.

Differential Revision: https://reviews.llvm.org/D75766
2020-03-11 14:07:04 +01:00
Stephan Herhut f6790a1c63 Revert "[MLIR] Add `and`, `or`, `xor`, `min`, `max` too gpu.all_reduce and the nvvm lowering"
Attribution to original author got lost.
2020-03-11 14:07:04 +01:00
Lei Zhang 5b0c60c58e [mlir][vulkan-runner] Use std::make_tuple to create tuple 2020-03-10 16:21:35 -04:00
Stephan Herhut 2eff566b07 [MLIR] Add `and`, `or`, `xor`, `min`, `max` too gpu.all_reduce and the nvvm lowering
Summary:
This patch add some builtin operation for the gpu.all_reduce ops.
- for Integer only: `and`, `or`, `xor`
- for Float and Integer: `min`, `max`

This is useful for higher level dialect like OpenACC or OpenMP that can lower to the GPU dialect.

Differential Revision: https://reviews.llvm.org/D75766
2020-03-10 21:09:06 +01:00
Denis Khalikov 1090a83069 [mlir][vulkan-runner] Update mlir-vulkan-runner execution driver.
* Adds GpuLaunchFuncToVulkanLaunchFunc conversion pass.
* Moves a serialization of the `spirv::Module` from LaunchFuncToVulkanCalls pass to newly created pass.
* Updates LaunchFuncToVulkanCalls instrumentation pass, adds `initVulkan` and `deinitVulkan` runtime calls.
* Adds `bindResource` call to bind specifc resource by the given descriptor set and descriptor binding.
* Eliminates static construction and desctruction of `VulkanRuntimeManager`.

Differential Revision: https://reviews.llvm.org/D75192
2020-03-10 15:58:31 -04:00
Mehdi Amini f80c6d8dec Fix MLIR build when NVPTX backend is not configured in
The GPUToCUDA conversion needs to conditionally link it
in.
2020-03-10 04:11:49 +00:00
Nicolas Vasilache 63b683a816 [mlir][Vector] Add a vector.matrix_multiply op on 1-D vectors
Summary: This op mirrors the llvm.intr counterpart and allows lowering + type conversions in a progressive fashion.

Differential Revision: https://reviews.llvm.org/D75775
2020-03-09 13:34:03 -04:00
Stephen Neuendorffer 9f979d7ad5 [MLIR] Fixes for BUILD_SHARED_LIBS=on
Differential Revision: https://reviews.llvm.org/D75308
2020-03-06 13:25:18 -08:00
Stephen Neuendorffer 4594d0e943 [MLIR] Move from add_dependencies() to DEPENDS
add_llvm_library and add_llvm_executable may need to create new targets with
appropriate dependencies.  As a result, it is not sufficient in some
configurations (namely LLVM_BUILD_LLVM_DYLIB=on) to only call
add_dependencies().  Instead, the explicit TableGen dependencies must
be passed to add_llvm_library() or add_llvm_executable() using the DEPENDS
keyword.

Differential Revision: https://reviews.llvm.org/D74930
2020-03-06 13:25:17 -08:00
Stephen Neuendorffer 2488016bae [MLIR] Remove redundant library dependencies
In cmake, it is redundant to have a target list under target_link_libraries()
and add_dependency().  This patch removes the redundant dependency from
add_dependency().

Differential Revision: https://reviews.llvm.org/D74929
2020-03-06 10:12:31 -08:00
Stephen Neuendorffer 1c82dd39f9 [MLIR] Ensure that target_link_libraries() always has a keyword.
CMake allows calling target_link_libraries() without a keyword,
but this usage is not preferred when also called with a keyword,
and has surprising behavior.  This patch explicitly specifies a
keyword when using target_link_libraries().

Differential Revision: https://reviews.llvm.org/D75725
2020-03-06 09:14:01 -08:00
Adrian Kuegel 86306df7dd Extract common code to deal with multidimensional vectors.
Summary: Also replace dyn_cast_or_null with dyn_cast when possible.

Differential Revision: https://reviews.llvm.org/D75733
2020-03-06 13:54:54 +01:00
aartbik 0d924700a6 [mlir] [VectorOps] Merge VectorReduction/VectorReductionV2 into one Op
Summary:
Paying off some technical debt in VectorOps, where I introduced a special
op for a fused accumulator into reduction to avoid some issues around
printing and parsing an optional accumulator. This CL merges the two
into one op again and does things the right way (still would be nice
to have "assemblyFormat" for optional operands though....).

Reviewers: nicolasvasilache, andydavis1, ftynse, rriddle

Reviewed By: nicolasvasilache

Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D75699
2020-03-05 13:07:31 -08:00
River Riddle cb1777127c [mlir] Remove successor operands from the Operation class
Summary:
This revision removes all of the functionality related to successor operands on the core Operation class. This greatly simplifies a lot of handling of operands, as well as successors. For example, DialectConversion no longer needs a special "matchAndRewrite" for branching terminator operations.(Note, the existing method was also broken for operations with variadic successors!!)

This also enables terminator operations to define their own relationships with successor arguments, instead of the hardcoded "pass-through" behavior that exists today.

Differential Revision: https://reviews.llvm.org/D75318
2020-03-05 12:53:02 -08:00
River Riddle 988249a506 [mlir] Refactor a few users to no longer rely on the successor operand API of Operation.
The existing API for successor operands on operations is in the process of being removed. This revision simplifies a later one that completely removes the existing API.

Differential Revision: https://reviews.llvm.org/D75316
2020-03-05 12:51:59 -08:00
Frank Laub cdc5cba721 [MLIR][Affine][NFC] Expose expandAffineMap
Summary:
Expose expandAffineMap so that it can be used by lowerings defined outside of
MLIR core.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D75589
2020-03-04 14:17:17 -08:00
Alex Zinenko aff6bf4ff8 [mlir] support conversion of parallel reduction loops to std
Recently introduced support for converting sequential reduction loops to
CFG of basic blocks in the Standard dialect makes it possible to perform
a staged conversion of parallel reduction loops into a similar CFG by
using sequential loops as an intermediate step. This is already the case
for parallel loops without reduction, so extend the pattern to support
an additional use case.

Differential Revision: https://reviews.llvm.org/D75599
2020-03-04 16:37:17 +01:00
Adrian Kuegel 91acb5b3e1 Add rsqrt op to Standard dialect and lower it to LLVM dialect.
Summary:
This adds an rsqrt op to the standard dialect, and lowers
it as 1 / sqrt to the LLVM dialect.

Differential Revision: https://reviews.llvm.org/D75353
2020-03-04 13:13:31 +01:00
Alex Zinenko 8ba8ab8c95 [mlir] support reductions in loop to std conversion
Summary:
Introduce support for converting loop.for operations with loop-carried values
to a CFG in the standard dialect. This is achieved by passing loop-carried
values as block arguments to the loop condition block. This block dominates
both the loop body and the block immediately following the loop, so the
arguments of this block are remain visible there.

Differential Revision: https://reviews.llvm.org/D75513
2020-03-03 18:21:13 +01:00
Stephan Herhut 10ec1860a8 [MLIR][GPU] Add error checking to loop.parallel to gpu transform.
Summary:
Instead of crashing on malformed input, the pass now produces error
messages.

Differential Revision: https://reviews.llvm.org/D75468
2020-03-03 13:29:09 +01:00
Stephan Herhut d17428d951 [MLIR][GPU] fix loop trip count computation in LoopsToGPU
Summary: Added brackets to fix the loop trip count computation.
The brackets ensure the bounds are subtracted before we divide
the result by the step of the loop.

Differential Revision: https://reviews.llvm.org/D75449
2020-03-02 15:53:33 +01:00
Stephen Neuendorffer 798e661567 Revert "[MLIR] Move from using target_link_libraries to LINK_LIBS for llvm libraries."
This reverts commit 7a6c689771.
This breaks the build with cmake 3.13.4, but succeeds with cmake 3.15.3
2020-02-29 11:52:08 -08:00
Stephen Neuendorffer 0810acc7f6 Revert "[MLIR] Remove redundant library dependencies"
This reverts commit c4c8fbde64.
2020-02-29 11:52:08 -08:00
Stephen Neuendorffer d675df0379 Revert "[MLIR] Move from add_dependencies() to DEPENDS"
This reverts commit 31e07d716a.
2020-02-29 11:52:08 -08:00
Stephen Neuendorffer bc991500ac Revert "[MLIR] Fixes for BUILD_SHARED_LIBS=on"
This reverts commit 777e97cc1a.
2020-02-29 11:09:21 -08:00
Stephen Neuendorffer 777e97cc1a [MLIR] Fixes for BUILD_SHARED_LIBS=on
Differential Revision: https://reviews.llvm.org/D75308
2020-02-29 10:47:28 -08:00
Stephen Neuendorffer 31e07d716a [MLIR] Move from add_dependencies() to DEPENDS
add_llvm_library and add_llvm_executable may need to create new targets with
appropriate dependencies.  As a result, it is not sufficient in some
configurations (namely LLVM_BUILD_LLVM_DYLIB=on) to only call
add_dependencies().  Instead, the explicit TableGen dependencies must
be passed to add_llvm_library() or add_llvm_executable() using the DEPENDS
keyword.

Differential Revision: https://reviews.llvm.org/D74930
2020-02-29 10:47:27 -08:00
Stephen Neuendorffer c4c8fbde64 [MLIR] Remove redundant library dependencies
In cmake, it is redundant to have a target list under target_link_libraries()
and add_dependency().  This patch removes the redundant dependency from
add_dependency().

Differential Revision: https://reviews.llvm.org/D74929
2020-02-29 10:47:27 -08:00
Stephen Neuendorffer 7a6c689771 [MLIR] Move from using target_link_libraries to LINK_LIBS for llvm libraries.
When compiling libLLVM.so, add_llvm_library() manipulates the link libraries
being used.  This means that when using add_llvm_library(), we need to pass
the list of libraries to be linked (using the LINK_LIBS keyword) instead of
using the standard target_link_libraries call.  This is preparation for
properly dealing with creating libMLIR.so as well.

Differential Revision: https://reviews.llvm.org/D74864
2020-02-29 10:47:26 -08:00
Stephen Neuendorffer dc1056a3f1 Revert "[MLIR] Move from using target_link_libraries to LINK_LIBS for llvm libraries."
This reverts commit 2f265e3528.
2020-02-28 14:13:30 -08:00
Stephen Neuendorffer fed2acc7f5 Revert "[MLIR] Remove redundant library dependencies"
This reverts commit e1cb15c8f9.
2020-02-28 14:06:20 -08:00
Tim Shen 0d65000e11 [MLIR] Add llvm.mlir.cast op for semantic preserving cast between dialect types.
Summary: See discussion here: https://llvm.discourse.group/t/rfc-dialect-type-cast-op/538/11

Reviewers: ftynse

Subscribers: bixia, sanjoy.google, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, Joonsoo, llvm-commits

Differential Revision: https://reviews.llvm.org/D75141
2020-02-28 12:20:23 -08:00
Tim Shen 2a00ae3984 [MLIR] Add LLVMConversionTarget as a customization point. NFC.
This is in preparation for the next patch D75141. The purpose is to
provide a single place where LLVM dialect registers its ops as
legal/illegal.

Reviewers: ftynse, mravishankar, herhut

Subscribers: jholewinski, bixia, sanjoy.google, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, csigg, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, Joonsoo, llvm-commits

Differential Revision: https://reviews.llvm.org/D75140
2020-02-28 12:20:23 -08:00
Stephen Neuendorffer 67f2a43cf8 Revert "[MLIR] Move from add_dependencies() to DEPENDS"
This reverts commit 8a2b86b2c2.
2020-02-28 12:17:40 -08:00
Stephen Neuendorffer 29c6721be2 Revert "[MLIR] Fixes for BUILD_SHARED_LIBS=on"
This reverts commit c767dc9394.
2020-02-28 12:17:39 -08:00
Stephen Neuendorffer c767dc9394 [MLIR] Fixes for BUILD_SHARED_LIBS=on
Differential Revision: https://reviews.llvm.org/D75308
2020-02-28 11:35:19 -08:00
Stephen Neuendorffer 8a2b86b2c2 [MLIR] Move from add_dependencies() to DEPENDS
add_llvm_library and add_llvm_executable may need to create new targets with
appropriate dependencies.  As a result, it is not sufficient in some
configurations (namely LLVM_BUILD_LLVM_DYLIB=on) to only call
add_dependencies().  Instead, the explicit TableGen dependencies must
be passed to add_llvm_library() or add_llvm_executable() using the DEPENDS
keyword.

Differential Revision: https://reviews.llvm.org/D74930
2020-02-28 11:35:18 -08:00
Stephen Neuendorffer e1cb15c8f9 [MLIR] Remove redundant library dependencies
In cmake, it is redundant to have a target list under target_link_libraries()
and add_dependency().  This patch removes the redundant dependency from
add_dependency().

Differential Revision: https://reviews.llvm.org/D74929
2020-02-28 11:35:18 -08:00
Stephen Neuendorffer 2f265e3528 [MLIR] Move from using target_link_libraries to LINK_LIBS for llvm libraries.
When compiling libLLVM.so, add_llvm_library() manipulates the link libraries
being used.  This means that when using add_llvm_library(), we need to pass
the list of libraries to be linked (using the LINK_LIBS keyword) instead of
using the standard target_link_libraries call.  This is preparation for
properly dealing with creating libMLIR.so as well.

Differential Revision: https://reviews.llvm.org/D74864
2020-02-28 11:35:17 -08:00
Stephen Neuendorffer c07fb9e016 [MLIR] Refactor library handling for conversions.
Collect a list of conversion libraries in cmake, so we don't have to
list these explicitly in most binaries.

Differential Revision: https://reviews.llvm.org/D75222
2020-02-28 11:35:17 -08:00
Adrian Kuegel 39e1c1fa9e Add GPU lowerings for the different log ops.
Summary: This adds GPU lowerings for log, log10 and log2.

Reviewers: mravishankar, herhut

Subscribers: jholewinski, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, nicolasvasilache, csigg, arpith-jacob, mgester, lucyrfox, liufengdb, Joonsoo, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D75239
2020-02-27 15:25:02 +01:00