Commit Graph

12541 Commits

Author SHA1 Message Date
River Riddle ae97b5acf8 [mlir:Bytecode] Move variable to inside of the lambda to fix MSVC build
MSVC is not picking up a variable capture somehow, try moving it inside.
2022-08-23 17:43:53 -07:00
River Riddle df4e637ca7 [mlir:Bytecode] Use UNSUPPORTED instead of XFAIL for s390x
Some tests still pass even though we don't claim big-endian support. Using
UNSUPPORTED is a better indicator than XFAIL that we don't guarantee that
the tests work.
2022-08-23 16:56:04 -07:00
River Riddle 02c2ecb9c6 [mlir:Bytecode] Add initial support for dialect defined attribute/type encodings
Dialects can opt-in to providing custom encodings by implementing the
`BytecodeDialectInterface`. This interface provides hooks, namely
`readAttribute`/`readType` and `writeAttribute`/`writeType`, that will be used
by the bytecode reader and writer. These hooks are provided a reader and writer
implementation that can be used to encode various constructs in the underlying
bytecode format. A unique feature of this interface is that dialects may choose
to only encode a subset of their attributes and types in a custom bytecode
format, which can simplify adding new or experimental components that aren't
fully baked.

Differential Revision: https://reviews.llvm.org/D132498
2022-08-23 16:56:04 -07:00
River Riddle b3449392f5 [mlir:Bytecode][NFC] Cleanup Attribute/Type reading
This moves some parsing functionality from BytecodeReader to
AttrTypeReader, and removes some duplication between the attribute/type
code paths.

Differential Revision: https://reviews.llvm.org/D132497
2022-08-23 16:56:03 -07:00
River Riddle 83dc999948 [mlir:Bytecode][NFC] Refactor string section writing and reading
This extracts the string section writer and reader into dedicated
classes, which better separates the logic and will also simplify future
patches that want to interact with the string section.

Differential Revision: https://reviews.llvm.org/D132496
2022-08-23 16:56:03 -07:00
Mehdi Amini de54bcc54c Revert "[MLIR]Extend vector.gather to support n-D result"
This reverts commit 0cbfd6fd16.

A test is crashing with the shared_lib config.
2022-08-23 20:26:38 +00:00
Stanley Winata 35a56e5ddc [mlir][spirv] Map memory space to OpenCL/Kernel storage class
Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D132428
2022-08-23 16:01:54 -04:00
Jeff Niu c476be5ca4 [mlir][LLVMIR] Add `sym_name` attribute to LLVMFuncOp in ODS
This is a required attribute for FunctionOpInterface. Add an ODS
definition so that a more efficient accessor is generated.
2022-08-23 10:02:10 -07:00
Che-Yu Wu 0cbfd6fd16 [MLIR]Extend vector.gather to support n-D result
Currently vector.gather only supports reading memory into a 1-D result vector.
This patch extends it to support an n-D result vector with the indices, masks,
and passthroughs in n-D vectors.

As we are trying to vectorize tensor.extract with vector.gather
(https://github.com/iree-org/iree/issues/9198), it will need to gather the
elements into an n-D vector. Having vector.gather with n-D results allows us
to avoid flatten and reshape at the vectorization stage. The backends can then
decide the optimal ways to lower the vector.gather op.

Note that this is different from n-D gathering, which is about reading n-D
memory with the n-D indices. The indices here are still only 1-D offsets on
the base.

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D131905
2022-08-23 16:53:19 +00:00
Jakub Kuderski 6fa87ec10f [ADT] Deprecate is_splat and replace all uses with all_equal
See the discussion thread for more details:
https://discourse.llvm.org/t/adt-is-splat-and-empty-ranges/64692

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D132335
2022-08-23 11:36:27 -04:00
Christian Sigg 50c33a3a9c [MLIR] Harden gpu.func verification
GPUFuncOpLowering moves the body out of gpu.func op and erases it. An empty gpu.func may fail verification but should not crash it. Verification of an erased op is triggered e.g. with debug printing on.

Reviewed By: akuegel

Differential Revision: https://reviews.llvm.org/D132446
2022-08-23 14:58:46 +02:00
Johannes Reifferscheid 27a1a85662 buffer-deallocation: consider aliases introduced by arith.select.
Currently, buffer deallocation considers arith.select to be
non-aliasing, which results in deallocs being inserted incorrectly. Since
arith.select doesn't implement any useful interfaces, this change just handles
it explicitly. Eventually this should probably be fixed properly, if this pass
is going to be used long term.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D132460
2022-08-23 14:37:02 +02:00
Adrian Kuegel 4641c6386b [mlir][Bufferization] Use DestinationStyleOpInterface where possible.
Differential Revision: https://reviews.llvm.org/D132462
2022-08-23 14:21:47 +02:00
Adrian Kuegel d9cbefc4c7 [mlir] Extract DestinationStyleOpInterface from LinalgStructuredInterface.
There are several use cases where a destination style operation needs an interface
that contains a subset of the methods from LinalgStructuredInterface.
In this change, we move all such methods to a new interface, and add forwarding
methods to LinalgStructuredInterface to make the change the less invasive.
It may be possible to refactor the code later to get rid of (some or all) of the
forwarding methods.
This change also removes the cloneWithMapper interface methods, as it is not used anywhere.

RFC:
https://discourse.llvm.org/t/rfc-interface-for-destination-style-ops/64056

Differential Revision: https://reviews.llvm.org/D132125
2022-08-23 13:27:30 +02:00
Matthias Springer c37ed7762e [tensor][bufferize] Use affine.apply instead of arith.addi in PadOp lowering
Affine exprs compose better than arith ops.

Differential Revision: https://reviews.llvm.org/D132456
2022-08-23 11:46:11 +02:00
Guray Ozen db0c7dde6b [mlir][Linalg] Retire LinalgStrategyGeneralizePass
This revision deletes LinalgStrategyGeneralizePass.

Context: https://discourse.llvm.org/t/psa-retire-linalg-filter-based-patterns/63785

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D132363
2022-08-23 10:31:43 +02:00
Guray Ozen e625aea33a [mlir][Linalg] Retire Linalg generic interchange pattern and pass
This revision removes the Linalg generic interchange pattern and pass.

It also changes transform-patterns test to make use of transform dialect.

Context: https://discourse.llvm.org/t/psa-retire-linalg-filter-based-patterns/63785

Differential Revision: https://reviews.llvm.org/D132368
2022-08-23 09:28:16 +02:00
Adrian Kuegel 62cd26bf96 [mlir][Linalg] Remove unused method declaration.
The "real" method declaration is in LinalgInterfaces.h

Differential Revision: https://reviews.llvm.org/D132227
2022-08-23 08:38:44 +02:00
Stanley Winata f148f3b2e9 [mlir][spirv] Use spv.store instead of init in var
- Add spv.store instead of init for spv.variable to fix data issues in
  some GPU drivers

Reviewed By: antiagainst
Patch By: raikonenfnu

Differential Revision: https://reviews.llvm.org/D132427
2022-08-22 23:41:27 -04:00
Stanley Winata 23f0d78284 [mlir][spirv] Add vector.fma lowering to CL.fma
Reviewed By: antiagainst
Patch By: raikonenfnu

Differential Revision: https://reviews.llvm.org/D132424
2022-08-22 23:36:07 -04:00
jackalcooper ff9aab7572 [mlir][vulkan-runner] Use aligned pointer for host buffer
Fix incorrect pointer usage in Vulkan buffer and Memref descriptor binding:
- Vulkan runtime produced incorrect result if there is alignment.
- There was illegal memory access if binding a LLVM global materialized
  from a constant op (0xdeadbeef).

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D132291
2022-08-22 20:42:54 -04:00
Jacques Pienaar 7d273fde11 [mlir] Populate default attributes on op creation
Default attributes were only handled by ODS accessors generated with the
intention that these behave as if set attributes. This addresses the
long standing TODO to address this inconsistency. Moving the
initialization to construction vs every access. Removing need for
duplicated default attribute population in python bindings.

Switch some of the OpenMP ones to optional attribute with default as the
currently set default values are not legal. May need to dig more there.

Switched LinAlg generated ones to optional attribute with default as its
quite widely used and unclear where it falls on two different
interpretations.

Differential Revision: https://reviews.llvm.org/D130916
2022-08-22 16:49:46 -07:00
Jacques Pienaar b1f2e2664e [mlir][tosa] Switch TosaFoldConstantTranspose to use ElementsAttr.
Also avoid redoing index calculation.

Differential Revision: https://reviews.llvm.org/D132274
2022-08-22 15:45:23 -07:00
Jacques Pienaar af29db64b2 [mlir][shape] refine shape.func and shape.with_shape
- shape.with_shape supports ExtentTensorType
- add helper to create shape.func

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D131977
2022-08-22 14:52:18 -07:00
River Riddle 96fd3f2d5b [mlir:Bytecode] Fix asan failure
We were accessing the region state after it got popped from the stack.
2022-08-22 10:02:42 -07:00
Goran Flegar 59548fe873 [mlir] Fix compile errors with bytecode support 2022-08-22 17:59:51 +02:00
Matthias Springer 9ee12f4778 [mlir][tensor][bufferize] Bufferize tensor.pad
tensor.pad is lowered to tensor.generate + tensor.insert_slice during bufferization. For best performance with constant padding values, users should vectorize the IR before bufferizing it.

This change also relaxes tje restriction that no new ops that bufferize to a memory write should be added during bufferization. Since bufferization has been split into two steps a while ago (tensor copy insertion + bufferization), it is reasonable to allow this now.

Differential Revision: https://reviews.llvm.org/D132355
2022-08-22 17:00:33 +02:00
Matthias Springer 416ba2256d [mlir][linalg][transform] Support dynamic tile sizes in TileToForeachThreadOp
TileToForeachThreadOp now accepts mixed SSA value operands / index attributes for tile_sizes and num_threads. (Reusing OperandsOrIntegersSizesList.) In case of an operand, a PDL_Operation must be specified that is mapped to a payload op that returns the tile size or number of threads.

Differential Revision: https://reviews.llvm.org/D131949
2022-08-22 16:48:45 +02:00
Christian Sigg 459fd3fb34 [MLIR][GPU] Detect bounds with `arith.minsi ` in loops-to-gpu
Previously, `arith.constant`, `arith.muli` and `affine.min` were supported when deriving upper loop bounds when converting parallel loops to GPU.

Reviewed By: akuegel

Differential Revision: https://reviews.llvm.org/D132354
2022-08-22 11:14:04 +02:00
River Riddle 93cf0e8a28 [mlir] Fix bots after bytecode support was added in D131747
* Fix ambiguous Twine constructor call
* Ensure shift is 64-bit (for MSVC)
* Disable bytecode tests on s390x (we don't support big endian right now)
2022-08-22 01:31:39 -07:00
River Riddle f3acb54c1b [mlir] Add initial support for a binary serialization format
This commit adds a new bytecode serialization format for MLIR.
The actual serialization of MLIR to binary is relatively straightforward,
given the very very general structure of MLIR. The underlying basis for
this format is a variable-length encoding for integers, which gets heavily
used for nearly all aspects of the encoding (given that most of the encoding
is just indexing into lists).

The format currently does not provide support for custom attribute/type
serialization, and thus always uses an assembly format fallback. It also
doesn't provide support for resources. These will be added in followups,
the intention for this patch is to provide something that supports the
basic cases, and can be built on top of.

https://discourse.llvm.org/t/rfc-a-binary-serialization-format-for-mlir/63518

Differential Revision: https://reviews.llvm.org/D131747
2022-08-22 00:36:26 -07:00
Kai Sasaki 3bdc0f36a5 [NFC][mlir][affine] Typo in Affine Analysis test
Reviewed By: Groverkss

Differential Revision: https://reviews.llvm.org/D132201
2022-08-21 20:24:53 +05:30
Tung D. Le d4284baf1e [mlir][normalize-memrefs] NFC Follow-up D125854
NFC follow-up D125854 to reflect some remaining comments in D125854

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D132200
2022-08-21 16:04:41 +05:30
Kazu Hirata 8b1b0d1d81 Revert "Use std::is_same_v instead of std::is_same (NFC)"
This reverts commit c5da37e42d.

This patch seems to break builds with some versions of MSVC.
2022-08-20 23:00:39 -07:00
Kazu Hirata c5da37e42d Use std::is_same_v instead of std::is_same (NFC) 2022-08-20 22:36:26 -07:00
Kazu Hirata 258531b7ac Remove redundant initialization of Optional (NFC) 2022-08-20 21:18:28 -07:00
Kazu Hirata 06b551c944 Use llvm::is_contained (NFC) 2022-08-20 21:18:27 -07:00
Joe Loser 1b08b276ae
[mlir] Use std::is_invocable instead of llvm::is_invocable
Now that MLIR is built with C++17, use the standard library equivalent of
`llvm::is_invocable`: `std::is_invocable`.

Differential Revision: https://reviews.llvm.org/D132318
2022-08-20 17:33:51 -06:00
Kazu Hirata 9e296584ce Fix unused variable warnings
These warnings came up with gcc-11.3.0.
2022-08-20 00:12:35 -07:00
Fangrui Song 35e56771ab [mlir][docs] Migrate away from deprecated llvm::Optional methods 2022-08-19 23:29:31 -07:00
Rainer Orth 8847d9a242 Reland "[mlir][test] Fix IR/AttributeTest.cpp compilation on Solaris"
The `IR/AttributeTest.cpp` test fails to compile on Solaris:
```
/vol/llvm/src/llvm-project/local/mlir/unittests/IR/AttributeTest.cpp:223:36: error: no matching function for call to 'allocate'
      AttrT::get(type, "resource", UnmanagedAsmResourceBlob::allocate(data));
                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/vol/llvm/src/llvm-project/local/mlir/unittests/IR/AttributeTest.cpp:237:3: note: in instantiation of function template specialization 'checkNativeAccess<mlir::detail::DenseResourceElementsAttrBase<int8_t>, char>' requested here
  checkNativeAccess<AttrT, T>(builder.getContext(), llvm::makeArrayRef(data),
  ^
/vol/llvm/src/llvm-project/local/mlir/unittests/IR/AttributeTest.cpp:258:3: note: in instantiation of function template specialization 'checkNativeIntAccess<mlir::detail::DenseResourceElementsAttrBase<int8_t>, char>' requested here
  checkNativeIntAccess<DenseI8ResourceElementsAttr, int8_t>(builder, 8);
  ^
/vol/llvm/src/llvm-project/local/mlir/include/mlir/IR/AsmState.h:221:3: note: candidate template ignored: requirement '!std::is_same<char, char>::value' was not satisfied [with T = char]
  allocate(ArrayRef<T> data, bool dataIsMutable = false) {
  ^
/vol/llvm/src/llvm-project/local/mlir/include/mlir/IR/AsmState.h:214:26: note: candidate function not viable: requires at least 2 arguments, but 1 was provided
  static AsmResourceBlob allocate(ArrayRef<char> data, size_t align,
                         ^
```
Because `char` is `signed` by default on Solaris and `int8_t` is
`char`. `std::is_same<int8_t, char>` is `true` unlike elsewhere, rejecting
the one-arg `allocate` overload.

Fixed by renaming the two overloads to avoid the ambiguity.

Tested on `amd64-pc-solaris2.11` ,`sparcv9-sun-solaris2.11`, and
`x86_64-pc-linux-gnu`.

Differential Revision: https://reviews.llvm.org/D131148
2022-08-19 21:59:57 +02:00
Christopher Bate 20ae22ba33 [mlir][Affine] Add affine.delinearize_index operation
This change adds a new AffineDelinearizeIndexOp to the affine dialect.
 The operation accepts an index type as well as a basis (array of index
 values) representing how the index should be decomposed into a
 multi-index. The decomposition obeys a canonical semantic that treats
 the final basis element as "fastest varying" and the first basis element
 as "slowest varying". A naive lowering of the operation using a sequence
 of AffineApplyOps is given.

RFC was discussed on discourse here: https://discourse.llvm.org/t/rfc-tensor-extracting-slices-from-tensor-collapse-shape/64034

Reviewed By: bondhugula, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D131997
2022-08-19 09:24:14 -06:00
Dominik Adamski b736179e34 [NFC][OpenMP] Add test for simd directive with nested loop
This test is follow up of the review: https://reviews.llvm.org/D131402

Co-Author of the patch: Kiran Chandramohan

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D132215
2022-08-19 05:45:32 -05:00
Archibald Elliott 3a729069e4 [IR] Update llvm.prefetch to match docs
The current llvm.prefetch intrinsic docs state "The rw, locality and
cache type arguments must be constant integers."

This change:
- Makes arg 3 (cache type) an ImmArg
- Improves the verifier error messages to reference the incorrect
  argument.
- Fixes two tests which contradict the docs.

This is needed as the lowering to GlobalISel is different for ImmArgs
compared to other constants. The non-ImmArgs create a G_CONSTANT MIR
instruction, the for ImmArgs the constant is put directly on the
intrinsic's MIR instruction as an immediate.

Differential Revision: https://reviews.llvm.org/D132042
2022-08-19 09:11:17 +01:00
Tung D. Le 183c4a391e [MLIR][normalize-memrefs] Non-normalizable operations with identity map layouts do not block normalization of the entire function
The current approach is convervative in which whenever there is a
non-normalizable operations in a function will the function be labelled
as non-normalizable. It means it requires that all operations must have
MemRefsNormalizable trait.

This patch relaxes the requirement that if the memref map layouts of a
non-normalizable operation are identity, this operation does not block
the normalization of the other operations in the same function.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D125854
2022-08-19 08:27:45 +05:30
John Ericson e941b031d3 Revert "[cmake] Use `CMAKE_INSTALL_LIBDIR` too"
This reverts commit f7a33090a9.

Unfortunately this causes a number of failures that didn't show up in my
local build.
2022-08-18 22:46:32 -04:00
lewuathe cdc8d0fcd7 [mlir][affine] Option to unroll cleanup loop if smaller trip count.
Add an option (cleanUpUnroll) to unroll cleanup loop even if the trip count is smaller the unroll factor.

Differential Revision: https://reviews.llvm.org/D129171
2022-08-19 09:35:20 +09:00
Thomas Raoux 06c02d5dbb [mlir][linalg] Fix tiling interface implementation offset calculation
The tiling interface implementation was making assumption on the code
generated by makeTiledShape which were wrong. The ExtractSliceOp create
may be combined with other ExtractSliceOp. To solve that we compute
directly the offset using the new utilities.

Differential Revision: https://reviews.llvm.org/D132182
2022-08-19 00:16:33 +00:00
Jacques Pienaar 18e616c3f0 [mlir][shape] add value_of op
Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D131976
2022-08-18 13:27:04 -07:00
Prabhdeep Singh Soni bce94ea551 [OMPIRBuilder] Add support for safelen clause
This patch adds OMPIRBuilder support for the safelen clause for the
simd directive.

Reviewed By: shraiysh, Meinersbur

Differential Revision: https://reviews.llvm.org/D131526
2022-08-18 15:43:08 -04:00
John Ericson f7a33090a9 [cmake] Use `CMAKE_INSTALL_LIBDIR` too
We held off on this before as `LLVM_LIBDIR_SUFFIX` conflicted with it.
Now we return this.

`LLVM_LIBDIR_SUFFIX` is kept as a deprecated way to set
`CMAKE_INSTALL_LIBDIR`. The other `*_LIBDIR_SUFFIX` are just removed
entirely.

I imagine this is too potentially-breaking to make LLVM 15. That's fine.
I have a more minimal version of this in the disto (NixOS) patches for
LLVM 15 (like previous versions). This more expansive version I will
test harder after the release is cut.

Reviewed By: sebastian-ne, ldionne, #libc, #libc_abi

Differential Revision: https://reviews.llvm.org/D130586
2022-08-18 15:33:35 -04:00
Michele Scuttari 8a10ee7590
[MLIR] DynamicMemRefType: iteration and access by indices
The methods to perform such operations have been implemented for the DynamicMemRefType in a way that is similar to the implementation for StridedMemRefType. Up until here one could pass an unranked memref to the library, and thus obtain a “dynamic” memref descriptor, but then there would have been no possibility to operate on its content.
2022-08-18 21:30:20 +02:00
Jeff Niu a48ff68888 [mlir][LLVMIR] "Modernize" LLVM insert/extract element operations
This patch "modernizes" the implementation of these operations by
switching them to assembly formats and type inference traits.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D131971
2022-08-18 14:43:06 -04:00
Peiming Liu 788b8b6b4a [mlir][memref] Remove extra brackets in doc
There is an (probably) unintended bracket in the documentation for `memref.subview`. See https://mlir.llvm.org/docs/Dialects/MemRef/#memrefsubview-mlirmemrefsubviewop below Example 5.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D132150
2022-08-18 18:26:29 +00:00
Vitaly Buka 14b480b092 [test][mlir] Disable tests with HWASAN
Re-enable after fixing https://github.com/llvm/llvm-project/issues/57231
2022-08-18 10:19:18 -07:00
Jeff Niu 491dd5a3b2 [mlir][LLVMIR] Fix syntax in integration tests (NFC) 2022-08-18 13:09:34 -04:00
Jeff Niu b2ccfb4d95 [mlir][LLVMIR] Change ShuffleVectorOp to use assembly format
This patch moves `LLVM::ShuffleVectorOp` to assembly format and in the
process drops the extra type that can be inferred (both operand types
are required to be the same) and switches to a dense integer array.

The syntax change:

```
// Before
%0 = llvm.shufflevector %0, %1 [0 : i32, 0 : i32, 0 : i32, 0 : i32] : vector<4xf32>, vector<4xf32>
// After
%0 = llvm.shufflevector %0, %1 [0, 0, 0, 0] : vector<4xf32>
```

Reviewed By: dcaballe

Differential Revision: https://reviews.llvm.org/D132038
2022-08-18 12:46:04 -04:00
Matthias Springer adc93e0d38 [mlir][SCF] Loop lb/ub are symbols during Affine Min/Max canonicalization
This fixes a bug in SCF/AffineCanonicalizationUtils.cpp. Loop lb/ub were previously considered dimensions, which caused a crash when a (non-optimizable) affine.min / affine.max expression was processed (due to multiplication of two dims). Lb/ub are now considered symbols and symbols may be multiplied. (The scope of the analysis is "within the loop body", at which point lb/ub are constants.)

Differential Revision: https://reviews.llvm.org/D132021
2022-08-18 11:44:48 +02:00
Matthias Springer 3f914d84c3 [mlir][bufferize] Better error handling: Fail if ToMemrefOps are found
bufferization.to_memref ops are not supported in One-Shot Analysis. They often trigger a failed assertion that can be confusing. Instead, scan for to_memref ops before running the analysis and immediately abort with a proper error message.

Differential Revision: https://reviews.llvm.org/D132027
2022-08-18 11:37:57 +02:00
Mats Petersson 5334a6337d [MLIR][NFC] Sort table of operations alphabetically
Arrange the operations in alphabetical order, rather than what
appears to be the order they were added. This was suggested in
a review when adding new operations.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D132046
2022-08-18 10:31:27 +01:00
Rainer Orth ca98e0dd6c [mlir][test] Require JIT support in JIT tests
A number of mlir tests `FAIL` on Solaris/sparcv9 with `Target has no JIT
support`.  This patch fixes that by mimicing `clang/test/lit.cfg.py` which
implements a `host-supports-jit` keyword for this.  The gtest-based unit
tests don't support `REQUIRES:`, so lack of support needs to be hardcoded
there.

Tested on `amd64-pc-solaris2.11` (`check-mlir` results unchanged) and
`sparcv9-sun-solaris2.11` (only one unrelated failure left).

Differential Revision: https://reviews.llvm.org/D131151
2022-08-18 11:26:07 +02:00
Uday Bondhugula d7ac92f27b [MLIR] Remove hardcoded usage of alloc/dealloc; use memory effects
Remove hardcoded usage of memref.alloc/dealloc ops in affine utils; use memory
effects instead. This is NFC for existing test cases, but strictly more
general/powerful in terms of functionality: it allows other allocation and
freeing ops (like gpu.alloc/dealloc) to work in conjunction with affine ops and
utilities.

Differential Revision: https://reviews.llvm.org/D132011
2022-08-18 14:46:03 +05:30
Güray Özen 85882e7d64 [mlir][Vector] Support 0-D vectors in ReductionOp
This commit adds support for 0-D vectors in ReductionOp.

Reviewed By: nicolasvasilache, dcaballe

Differential Revision: https://reviews.llvm.org/D131896
2022-08-18 09:12:47 +00:00
Vitaly Buka 3f5f2905c4 [test] Propagate HWASAN_OPTIONS 2022-08-17 18:59:49 -07:00
Jakub Kuderski 95c4e51839 [mlir][spirv] Add arith.addi_carry to spv.IAddCarry conversion
Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D131908
2022-08-17 21:33:34 -04:00
Aart Bik e3d64ccf9f [mlir][sparse] more concise sparse tensor type printing
This change omits default values from the sparse tensor type,
saving considerable text real estate for the common cases.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D132083
2022-08-17 17:35:50 -07:00
Jacques Pienaar 8fe0691161 [mlir] Update when to check for non-null optional default attr
OptionalAttr does the wrapping with Optional explicitly in ODS while
default valued optional attribute doesn't (and follows DefaultValuedAttr
in this behavior/meant as drop-in for updating old behavior), update
when to emit check for non-null to account for this.  Also add variant
for optional default valued string attribute to have same convenience as
default valued string attribute.
2022-08-17 17:00:47 -07:00
Jacques Pienaar 052669e71a [mlir] add missing pessimistic setting
This is updated on https://reviews.llvm.org/D127139, to mark op with region as pessimistic.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D131480
2022-08-17 16:20:13 -07:00
root 894e8a5446 [MLIR] Add dealloc alias check to bufferization
Traverse the cloneOp for aliases to find the alloc op

Reviewed By: frgossen, bondhugula

Differential Revision: https://reviews.llvm.org/D131797
2022-08-17 19:11:59 -04:00
Jakub Kuderski 4309170c87 [mlir] Add arith.addi_carry op
The `arith.addi_carry` op implements integer addition with overflows. The carry is returned via the second result, as `i1`.

Reviewed By: antiagainst, bondhugula

Differential Revision: https://reviews.llvm.org/D131893
2022-08-17 17:01:20 -04:00
Markus Böck f289c47b5e [mlir][ODS] Add `constBuilderCall` to `Dense*ArrayAttr`s
These are useful in builders of Ops taking DenseArrayAttrs or for use in Rewriter, to create constant instances.

Differential Revision: https://reviews.llvm.org/D132067
2022-08-17 22:18:13 +02:00
Nirvedh Meshram 6b8952554c [mlir][spirv] Modify intel joint matrix ops
Adding more supported storage classes in pointer type
Removing support for arithmetic ops as this is not available on the tensor hardware

Reviwed By: antiagainst

Differential Revision: https://reviews.llvm.org/D132039
2022-08-17 18:40:59 +00:00
Markus Böck a5b11f40f8 [mlir] Fix stale comment about how to use `VariadicOfVariadic`
The kind of attribute required has changed in 58a47508f0 but the comment was not updated to reflect it.
2022-08-17 20:15:17 +02:00
Peiming Liu 8f8c8168f3 [mlir][sparse] fix compiler warning
Reviewed By: aartbik, bixia

Differential Revision: https://reviews.llvm.org/D132040
2022-08-17 17:46:28 +00:00
Jim Kitchen c8bb23547f [mlir][sparse] Custom reduce with identity
Implement the new sparse_tensor.reduce operation which
accepts a starting identity value and a code block
describing how to perform the reduction.

Reviewed by: aartbik

Differential Revision: https://reviews.llvm.org/D130573
2022-08-17 11:21:46 -05:00
Mats Petersson 1e12793491 [MLIR]Add support for Arith MAX & MIN operations
There are some of this supported in various places, but the
basic conversion of single operations to LLVM was not supported.

Adding this to allow Flang to use these.

Reviewed By: bondhugula

Differential Revision: https://reviews.llvm.org/D131912
2022-08-17 17:00:53 +01:00
Thomas Joerg 5ad7cc7f21 Fix unused variable (introduced in
c248219b09)
2022-08-17 15:18:37 +02:00
Dominik Adamski 19bd4789b9 [NFC][OpenMP] Update simd loop collapse support description
Simdloop collapse clause is supported in the same way
as colllapse clause for worksharing loops.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D131674

Signed-off-by: Dominik Adamski <dominik.adamski@amd.com>
2022-08-17 03:57:22 -05:00
Thomas Joerg 42b2facbe2 Revert "DynamicMemRefType: iteration and access by indices"
This reverts commit b8ecf32f81.

This commit introduces undefined behavior according to UBSan:
UndefinedBehaviorSanitizer: nullptr-with-nonzero-offset third_party/llvm/llvm-project/mlir/include/mlir/ExecutionEngine/CRunnerUtils.h:377:5
2022-08-17 10:35:00 +02:00
Peiming Liu ee986ab727 [mlir][sparse] Refactoring: remove Operation * from the argument list in utility functions
This patch remove the Operation *op from the argument list in utility functions, and directly pass the Location instead of calling op->getLoc().

This should make the code more clear, as the utility function (logically) does not relies on the operation that we are currently rewriting, and they behave the same regardless of the operation.

Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D131991
2022-08-16 21:26:43 +00:00
Peiming Liu c248219b09 [mlir][sparse] Implements concatenate operation for sparse tensor
This patch implements the conversion rule for operation introduced in https://reviews.llvm.org/D131200.
Also contains integration test for correctness

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D131200
2022-08-16 20:47:47 +00:00
Slava Zakharin f9d988f1ac [mlir][math] Added basic support for FPowI operation.
The operation computes pow(b, p), where 'b' is floating point
and 'p' is a signed integer. The result's type matches 'b' type.
The operands must have the same shape.

Differential Revision: https://reviews.llvm.org/D129811
2022-08-16 09:24:01 -07:00
Nikita Popov 8f555a52e0 [cmake] Fix tablegen exports
This fixes some fallout from D131282. Currently, add_tablegen() will add the tablegen target to LLVM_EXPORTS and associates the install with LLVMExports. For non-standalone builds, this means that you end up with mlir-tblgen and clang-tblgen in LLVMExports.

However, these projects should instead be using MLIR_EXPORTS/MLIRTargets and CLANG_EXPORTS/ClangTargets. To fix this, add an extra EXPORT option and make use of get_target_export_arg() to create the correct export argument.

Reviewed By: ashay-github

Differential Revision: https://reviews.llvm.org/D131565
2022-08-16 14:17:23 +02:00
Johannes Reifferscheid 375a5cb648 Don't lower log1p(x) to log(1 + x).
The latter has accuracy issues around 0. The lowering in MathToLLVM is kept for now.

Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/D131676
2022-08-16 06:58:00 +02:00
Nirvedh Meshram b8f62dc22a [MLIR][SPIRV] Add intel joint matrix ops
Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D131586
2022-08-15 23:49:45 +00:00
Thomas Raoux 444b4fda17 [mlir][tosa] Fix clamp float lowering
min and max were mixed up after switching to using float min/max

Differential Revision: https://reviews.llvm.org/D131923
2022-08-15 22:25:14 +00:00
Lei Zhang 6f4f9e316c [mlir][spirv] Add a test covering load/store with memory operands
Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D131904
2022-08-15 17:48:33 -04:00
Kai Sasaki 5605f17a4d [mlir][complex] Mark all supported operation illegal explicitly for complex to libm
We can mark all supported operations illegal explicitly to ensure no convertible ops remain.

Differential Revision: https://reviews.llvm.org/D131877
2022-08-15 17:02:16 -04:00
Matthias Springer 31fbdab376 [mlir][transforms] Add topological sort analysis
This change add a helper function for computing a topological sorting of a list of ops. E.g. this can be useful in transforms where a subset of ops should be cloned without dominance errors.

The analysis reuses the existing implementation in TopologicalSortUtils.cpp.

Differential Revision: https://reviews.llvm.org/D131669
2022-08-15 21:09:18 +02:00
Diego Caballero e86119b4ff [mlir] Minor fixes after removing types from attributes
D130092 removed types from attributes. This patch fixes a minor
issues what was exposed when integrating that change in IREE. An
explicit cast is needed so that the template type of `makeArrayRef`
is automatically deduced.

Co-authored-by: Lei Zhang <antiagainst@google.com>

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D131604
2022-08-15 19:07:25 +00:00
Stella Laurenzo f55a6fde1d [mlir] Make sure that aggregate shared libraries define all of their symbols.
We were hitting issues on Linux where this was only being caught at runtime, and different linkers (BFD vs LLD) are differently strict in such situations. Such libraries will also fail to build properly on Windows (but test coverage of that is limited, so it is better to enforce globally).

Differential Revision: https://reviews.llvm.org/D131911
2022-08-15 12:02:24 -07:00
Slava Zakharin 2dde4ba639 [mlir][math] Added algebraic simplification for IPowI operation.
Differential Revision: https://reviews.llvm.org/D130390
2022-08-15 11:55:05 -07:00
Jeff Niu 133624acf9 [mlir][python] Fix build on windows
Reviewed By: stella.stamenova, ashay-github

Differential Revision: https://reviews.llvm.org/D131906
2022-08-15 14:51:04 -04:00
Slava Zakharin cd3a234fa9 [mlir][math] Added constant folding for IPowI operation.
Differential Revision: https://reviews.llvm.org/D130389
2022-08-15 11:31:14 -07:00
Diego Caballero 43beaeed38 Revert "antiagainst's fix"
This reverts commit 42d48d0966.
2022-08-15 18:27:35 +00:00
Diego Caballero 42d48d0966 antiagainst's fix 2022-08-15 17:57:38 +00:00
Zhixun Tan e42bfec9b6 [mlir][dataflow] Remove the unused AnalysisState::defaultInitialize().
Depends On D131660

`defaultInitialize()` was introduced for the "nudging" behavior, which has been deleted.

Reviewed By: Mogball, rriddle

Differential Revision: https://reviews.llvm.org/D131746
2022-08-15 13:24:08 -04:00
Zhixun Tan 4835441d02 [mlir][dataflow] Remove Abstract{Sparse,Dense}Lattice::isAtFixpoint() and an ineffective optimization to simplify public API
Currently, in the MLIR `{Sparse,Dense}DataFlowAnalysis` API, there is a small optimization:

Before running a transfer function, if the "out state" is already at the pessimistic fixpoint (bottom lattice value), then we know that it cannot possibly be changed, therefore we can skip the transfer function.

I benchmarked and found that this optimization is ineffective, so we can remove it and simplify `{Sparse,Dense}DataFlowAnalysis`. In a subsequent patch, I plan to change/remove the concept of the pessimistic fixpoint so that the API is further simplified.

Benchmark: I ran the following tests 5 times (after 3 warmup runs), and timed the `initializeAndRun()` function.

| Test | Before (us) | After (us) |
| mlir-opt -test-dead-code-analysis mlir/test/Analysis/DataFlow/test-dead-code-analysis.mlir | 181.2536 | 187.7074 |
| mlir-opt -- -test-dead-code-analysis mlir/test/Analysis/DataFlow/test-last-modified-callgraph.mlir | 109.5504 | 105.0654 |
| mlir-opt -- -test-dead-code-analysis mlir/test/Analysis/DataFlow/test-last-modified.mlir | 333.3646 | 322.4224 |
| mlir-opt -- -allow-unregistered-dialect -sccp mlir/test/Analysis/DataFlow/test-combined-sccp.mlir | 1027.1492 | 1081.818 |

Note: `test-combined-sccp.mlir` is crafted by combining `mlir/test/Transforms/sccp.mlir`, `mlir/test/Transforms/sccp-structured.mlir` and `mlir/test/Transforms/sccp-callgraph.mlir`.

Reviewed By: aartbik, Mogball

Differential Revision: https://reviews.llvm.org/D131660
2022-08-15 13:21:05 -04:00
Jakub Kuderski 53d4ff4a47 [mlir] Simplify is_splat use in MatchAllPred. NFC.
This is a simple cleanup after https://reviews.llvm.org/D131289.

Reviewed By: antiagainst, Mogball

Differential Revision: https://reviews.llvm.org/D131895
2022-08-15 13:18:11 -04:00
Thomas Raoux 91594b5b98 [mlir][nvpu] Prevent F32ToTF32 pattern to generate illegal IR
We shouldn't apply this pattern to non F32->F32 mma.sync operations.

Differential Revision: https://reviews.llvm.org/D131902
2022-08-15 16:46:18 +00:00
Mahesh Ravishankar f365e85c83 [mlir] Revisit `LinalgLoopDistributionOptions`.
This patch cleans up the way `LinalgLoopDistributionOptions` are meant
to be used. The option just contains a call back that takes the list
of loop ranges that represent the loops that are to be distributed.
These loops are the outer parallel loops of the tiled operation which
have non-zero tile sizes specified. The call back returns for each of
the loops,
- The procId to use,
- The number of processors,
- The distribution method to use for that loop.

Reviewed By: antiagainst, hanchung

Differential Revision: https://reviews.llvm.org/D131232
2022-08-15 15:56:17 +00:00
Michele Scuttari b8ecf32f81
DynamicMemRefType: iteration and access by indices
The methods to perform such operations have been implemented for the DynamicMemRefType in a way that is similar to the implementation for StridedMemRefType. Up until here one could pass an unranked memref to the library, and thus obtain a “dynamic” memref descriptor, but then there would have been no possibility to operate on its content.

Differential Revision: https://reviews.llvm.org/D131359
2022-08-15 17:37:39 +02:00
Matthias Springer 4911100dc6 [mlir][linalg][python] Add named constructor for MatchOp
This constructor makes it easier to match for ops by their name.

Differential Revision: https://reviews.llvm.org/D131882
2022-08-15 14:41:28 +02:00
Matthias Springer a36348c586 [mlir][bufferize] Fix bug in AllocTensorElimination
AllocTensorElimination does currently not support chains where the type is
changing. AllocTensorElimination used to generate invalid IR for such
inputs. With this commit, AllocTensorElimination does no longer apply to
such inputs. (It can be extended to support such IR if needed.)

Differential Revision: https://reviews.llvm.org/D131880
2022-08-15 11:45:58 +02:00
Kazu Hirata 6d9cd9199a Use llvm::all_of (NFC) 2022-08-14 16:25:36 -07:00
Kazu Hirata 3a6da9ebcb [mlir] Remove redundant member initialization (NFC)
Identified with readability-redundant-member-init.
2022-08-14 12:51:59 -07:00
Jacques Pienaar b23e72a2e7 [mlir][shape] Rename dim.dim to dim.index (NFC)
dim member on dim was confusing, change to be consistent with
tensor::dim.
2022-08-13 09:34:37 -07:00
Jeff Niu bca889524a [mlir][python] add a todo to replace throw in dense array iterator 2022-08-12 23:35:38 -04:00
Jeff Niu 96da738dc5 [mlir] Remove colon from empty dense array syntax
E.g. `array<i32:>` -> `array<i32>`

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D131823
2022-08-12 22:36:54 -04:00
Jeff Niu e35ca70eb3 [mlir][ods] Rename Confined and AllAttrConstraintsOf
Confined -> ConfinedAttr
AllAttrConstraintsOf -> AllOfAttr

To be in line with ConfinedType and AllOfType.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D131822
2022-08-12 22:36:17 -04:00
Kazu Hirata 5c4674d67b [mlir] Deprecate OptionalParseResult::{hasValue,getValue}
This patch deprecates hasValue and getValue for consistency with
std::optional and llvm::Optional.  Note that I've migrated all known
uses of them to has_value and value, respectively.

Differential Revision: https://reviews.llvm.org/D131366
2022-08-12 19:19:24 -07:00
Kazu Hirata 7082e1506d [mlir] Use has_value instead of hasValue (NFC) 2022-08-12 19:19:23 -07:00
Jeff Niu 60d2769239 [mlir][ods] OpFormat: ensure that regions don't follow `attr-dict`
An optional attribute dictionary before a region in an assembly format
is a potential format ambiguity because they both start with `{`.

Fixes #53077

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D131636
2022-08-12 21:00:25 -04:00
Jeff Niu a2ad3ec7ac [mlir][ods] Support string literals in `custom` directives
This patch adds support for string literals as `custom` directive
arguments. This can be useful for re-using custom parsers and printers
when arguments have a known value. For example:

```
ParseResult parseTypedAttr(AsmParser &parser, Attribute &attr, Type type) {
  return parser.parseAttribute(attr, type);
}

void printTypedAttr(AsmPrinter &printer, Attribute attr, Type type) {
  return parser.printAttributeWithoutType(attr);
}
```

And in TableGen:

```
def FooOp : ... {
  let arguments = (ins AnyAttr:$a);
  let assemblyFormat = [{ custom<TypedAttr>($a, "$_builder.getI1Type()")
                          attr-dict }];
}

def BarOp : ... {
  let arguments = (ins AnyAttr:$a);
  let assemblyFormat = [{ custom<TypedAttr>($a, "$_builder.getIndexType()")
                          attr-dict }];
}
```

Instead of writing two separate sets of custom parsers and printers.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D131603
2022-08-12 20:55:11 -04:00
Jeff Niu 58a47508f0 (Reland) [mlir] Switch segment size attributes to DenseI32ArrayAttr
This reland includes changes to the Python bindings.

Switch variadic operand and result segment size attributes to use the
dense i32 array. Dense integer arrays were introduced primarily to
represent index lists. They are a better fit for segment sizes than
dense elements attrs.

Depends on D131801

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D131803
2022-08-12 19:44:52 -04:00
Jeff Niu 619fd8c2ab [mlir][python] Add python bindings for DenseArrayAttr
This patch adds python bindings for the dense array variants.

Fixes #56975

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131801
2022-08-12 19:44:49 -04:00
Eric Kunze 9dec80be72 [MLIR][TOSA] Remove ReluN operator from TOSA dialect
ReluN has been removed from the TOSA specification. It can be replaced
in all instances with Clamp(0,N)

Signed-off-by: Eric Kunze <eric.kunze@arm.com>

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D128683
2022-08-12 16:00:11 -07:00
Jeff Niu 8a3481b958 [mlir] Add AllOfType and ConfinedType constraints
`AllOfType` is a type constraint that satisfies all given type
constraints and `ConfinedType` is a type that satisfies additional
predicates. These shorthands simplify type constraint definition mostly
by removing the need to deal with `myType.predicate` manipulation.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D131788
2022-08-12 16:25:36 -04:00
Jacques Pienaar d6f2e32fc1 [mlir] Flip Tensor dialect and ViewLike (NFC)
Also requires updating view like interface to use prefixed form.

Differential Revision: https://reviews.llvm.org/D131361
2022-08-12 12:41:32 -07:00
Frederik Gossen 2c3ca3b684 [MLIR] Add utility function to create values for all dimensions of a tensor value
This is a variant of the already provided `createDynamicDimValues` helper.

Differential Revision: https://reviews.llvm.org/D131798
2022-08-12 14:42:27 -04:00
Aart Bik 8dd07e36ca [mlir][sparse] enable integral abs recognition
The end-to-end test for this new feature also exposed a bug
in LLVM IR lowering (since then, fixed), where we need to account
for the min-poison bit as extra argument.

    declare i32 @llvm.abs.i32(i32 <src>, i1 <is_int_min_poison>)

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D131712
2022-08-12 11:36:40 -07:00
Jacques Pienaar 2f025e0e78 [mlir][shape] Add dim op
Convenience op that allows for simple expression of common crossing of
value/shape divide.

Differential Revision: https://reviews.llvm.org/D131497
2022-08-12 11:02:08 -07:00
Jeff Niu bc8d966433 [mlir][math] Fix lowering of AbsIOp
The LLVM intrinsic has a bool flag `is_int_min_poison` that needs to be
set.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D131785
2022-08-12 12:10:15 -04:00
Alex Zinenko 4b45999bdd [mlir] fix crossed transform dialect patches
One patch added a use of the IR syntax that was changed by another.
2022-08-12 15:48:15 +00:00
Alex Zinenko a60ed95419 [mlir][transform] failure propagation mode in sequence
Introduce two different failure propagation mode in the Transform
dialect's Sequence operation. These modes specify whether silenceable
errors produced by nested ops are immediately propagated, thus stopping
the sequence, or suppressed. The latter is useful in end-to-end
transform application scenarios where the user cannot correct the
transformation, but it is robust enough to silenceable failures. It
can be combined with the "alternatives" operation. There is
intentionally no default value to avoid favoring one mode over the
other.

Downstreams can update their tests using:

  S='s/sequence \(%.*\) {/sequence \1 failures(propagate) {/'
  T='s/sequence {/sequence failures(propagate) {/'
  git grep -l transform.sequence | xargs sed -i -e "$S"
  git grep -l transform.sequence | xargs sed -i -e "$T"

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D131774
2022-08-12 15:31:22 +00:00
Matthias Springer 0581ab65ea [mlir][linalg][transform] Support matching of attributes (and their values)
Do not just check if an attribute exists on the payload op. Also check its value.

Differential Revision: https://reviews.llvm.org/D131760
2022-08-12 14:55:00 +02:00
Jinyun Joey Ye 921b13f263 Missing climits header file in MemRefUtils.h
MemRefUtils.h uses UINT_MAX, which is not included in current header files list. This patch include climits to avoid compilation error in this header file.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131529
2022-08-12 14:17:36 +02:00
Nicolas Vasilache a6bf6f25f0 [mlir][Linalg] Let FuseIntoContainingOp return success when nothing is fused.
This composes better when the op is applied in situations where it does not match.

Differential Revision: https://reviews.llvm.org/D131734
2022-08-12 02:18:31 -07:00
Matthias Springer bf1b9528ff [mlir][bufferize] Fix missing copy when bufferizing loops
Using a loop init_arg inside of the loop is not supported. This change adds a pre-processing pass that resolves such IR with copies.

Differential Revision: https://reviews.llvm.org/D131689
2022-08-12 10:44:55 +02:00
Marius Brehler f3547fd541 [mlir][emitc][nfc] Clean up tests
With https://reviews.llvm.org/D131666 the types were removed from the
EmitC opaque attributes. This cleans up the tests accordingly.
2022-08-12 08:25:31 +00:00
Alex Zinenko e8e718fa4b Revert "[mlir] Switch segment size attributes to DenseI32ArrayAttr"
This reverts commit 30171e76f0.

Breaks Python tests in MLIR, missing C API and Python changes.
2022-08-12 10:22:47 +02:00
Marius Brehler 90736babca [mlir][EmitC] Remove the type from the OpaqueAttr
This removes the type from EmitC's opaque attribute. The value provided
as a StringRefParameter can always be emitted as is. In consquence the
constant and variable ops explicitly need to opaque attributes which are
no longer typed attributes.

Co-authored-by: Simon Camphausen <simon.camphausen@iml.fraunhofer.de>

Reviewed By: Mogball, jpienaar

Differential Revision: https://reviews.llvm.org/D131666
2022-08-12 07:12:24 +00:00
Jeff Niu 30171e76f0 [mlir] Switch segment size attributes to DenseI32ArrayAttr
Switch variadic operand and result segment size attributes to use the
dense i32 array. Dense integer arrays were introduced primarily to
represent index lists. They are a better fit for segment sizes than
dense elements attrs.

Depends on D131738

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D131702
2022-08-11 20:56:45 -04:00
Jeff Niu 2092d1438c [mlir] Change the syntax of dense arrays
Follow-up to D123774, where the syntax of dense arrays was discussed. It
was included that the syntax should be changed to `array<i32: 1, 2>`.
This patch changes the syntax but importantly preserves the `[1, 2]`
syntax when embedding these attributes in assembly formats through ODS.

Reviewed By: mehdi_amini, jpienaar

Differential Revision: https://reviews.llvm.org/D131738
2022-08-11 20:56:42 -04:00
not-jenni 7ff0ca1a4d [mlir][tosa] Updates tosa.logical_not to use the SameOperandsAndResultType trait
Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D131496
2022-08-11 15:42:30 -07:00
Uday Bondhugula 0e54d9dfdc [MLIR] Fix hasNoInterveningEffect in the presence of ops from different affine scopes
Fix hasNoInterveningEffect in the presence of ops from different affine
scopes. Also, correctly check for dependence failures as well instead of
just for the existence of a dependence.

Differential Revision: https://reviews.llvm.org/D131641
2022-08-11 21:07:24 +05:30
Benjamin Kramer 726719e970 [mlir][sparse] Refine f695554a2a wording a bit and fix the double conversion I broke with aggressive copy&paste 2022-08-11 11:58:49 +02:00
Marius Brehler 91b6f76a58 [mlir] Fix calling the native mlir-pdll-tblgen
This sets the `MLIR_PDLL_TABLEGEN_EXE` and `MLIR_PDLL_TABLEGEN_TARGET`
as cache variables which is necessary for cross-compiling projects that
rely on MLIR and the mlir-pdll-tblgen tool.

The patch is similar to https://reviews.llvm.org/D130350.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D131596
2022-08-11 09:34:13 +00:00
Benjamin Kramer f695554a2a [mlir][sparse] Use the correct ABI on x86 and re-enable tests
c7ec6e19d5 made LLVM adhere to the x86
psABI and pass bf16 in SSE registers instead of GPRs. This breaks the
custom versions of runtime functions we have for bf16 conversion. A
great fix for this would be to use __bf16 types instead which carry the
right ABI, but that type isn't widely available.

Instead just pretend it's a 32 bit float on the ABI boundary and
carefully cast it to the right type.

Fixes #57042
2022-08-11 10:41:53 +02:00
Jeff Niu 5e0c3b4309 [mlir][LLVMIR] Clean up the definitions of ReturnOp/CallOp 2022-08-11 00:35:02 -04:00
Uday Bondhugula 3d93885f7c [MLIR][NFC] Refactor affine analysis helpers
Refactor affine analysis helpers: the existing ones were using
unnecessary or improperly named arguments. NFC.

Differential Revision: https://reviews.llvm.org/D131557
2022-08-11 06:28:59 +05:30
Uday Bondhugula b6164ec9c2 [MLIR] Add missing check for unsupported affine dependence analysis
Add missing check in affine dependence analysis when dependence analysis
isn't possible due to the ops being in different affine scopes.  The
lack of such a check could lead to a crash or incorrect behavior in
several dependent utilities.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131556
2022-08-11 06:02:39 +05:30
Uday Bondhugula 82973067ac [MLIR] Fix build breakage due to 5c5af910fe
Fix build breakage due to 5c5af910fe.

```commit 5c5af910fe
Author: Jeff Niu <jeff@modular.com>
Date:   Tue Aug 9 22:07:35 2022 -0400

    [mlir][LLVMIR] "Modernize" Insert/ExtractValueOp ```

results in

```lib/Conversion/LLVMCommon/MemRefBuilder.cpp:170:70: error: call of overloaded ‘makeArrayRef<int64_t>(<brace-enclosed initializer list>)’ is ambiguous
       llvm::makeArrayRef<int64_t>({kStridePosInMemRefDescriptor,
       pos}));

llvm/include/llvm/ADT/ArrayRef.h:505:15: note: candidate: ‘llvm::ArrayRef<T> llvm::makeArrayRef(const std::vector<T>&) [with T = long int]’
   ArrayRef<T> makeArrayRef(const std::vector<T> &Vec) {
               ^~~~~~~~~~~~
llvm/include/llvm/ADT/ArrayRef.h:516:37: note: candidate: ‘llvm::ArrayRef<T> llvm::makeArrayRef(const llvm::ArrayRef<T>&) [with T = long int]’
   template <typename T> ArrayRef<T> makeArrayRef(const ArrayRef<T> &Vec) {
                                                                      ^

Differential Revision: https://reviews.llvm.org/D131637
2022-08-11 05:58:38 +05:30
Aart Bik 6b7459115f [mlir][sparse][bf16] disable two bf16 tests
Supposedly our ABI issues were fixed, per issue:
https://github.com/llvm/llvm-project/issues/55992

However, with a recent changes to bf16, these tests
fail again; not sure why yet:
https://reviews.llvm.org/D130832
https://lab.llvm.org/buildbot/#/builders/61/builds/30600

So we disable the tests for now. Issue is tracked in:
https://github.com/llvm/llvm-project/issues/57042

Differential Revision: https://reviews.llvm.org/D131621
2022-08-10 15:50:27 -07:00
Jeff Niu aa543b8094 [mlir][ods] (NFC) split out erroring format tests 2022-08-10 15:36:12 -04:00
jackalcooper f230d91592 [mlir][spirv] Turn various passes to plain OperationPass
Made passes converting ops from other dialects to spirv OperationPass,
so that downstream compiler could put them in a proper nested pass
manager to lower device code only.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D131591
2022-08-10 13:50:07 -04:00
Slava Zakharin 08b4cf3620 [mlir][math] Added basic support for IPowI operation.
The operation computes pow(b, p), where 'b' and 'p' are signed integers
of the same width. The result's type matches the operands' type.

Differential Revision: https://reviews.llvm.org/D129809
2022-08-10 10:11:53 -07:00
Jeff Niu 5c5af910fe [mlir][LLVMIR] "Modernize" Insert/ExtractValueOp
This patch "modernizes" the LLVM `insertvalue` and `extractvalue`
operations to use DenseI64ArrayAttr, since they only require an array of
indices and previously there was confusion about whether to use i32 or
i64 arrays, and to use assembly format.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131537
2022-08-10 12:51:11 -04:00
Alexander Belyaev 47cf004076 [mlir] Fix win build by using has_value() instead of hasValue() for Optional. 2022-08-10 17:48:35 +02:00
Alexander Belyaev 42f32f69ad [mlir] Extract offsets-sizes-strides computation from `makeTiledShape(s)`.
This change separates computation of the actual parameters of the subset and
the materialization of subview/extract_slice. That way the users can still use
Linalg tiling logic even if they use different operations to materialize the
subsets.

Differential Revision: https://reviews.llvm.org/D131053
2022-08-10 15:37:38 +02:00
Alex Zinenko 2e2ad53979 [mlir] Generate C++ doc comments for interfaces
When emitting the declarations for interface methods defined in ODS,
also emit their descriptions as C++ comments. This makes the
documentation accessible to C++ tooling such as IDEs that offers better
usability than reading it form the .td or the website.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D130478
2022-08-10 15:12:29 +02:00
Jacques Pienaar 1f02ad7131 [mlir][shape] Update meet to handle all size & shape types
Also tighten up return type inference & compatibility functions.

Differential Revision: https://reviews.llvm.org/D130866
2022-08-10 05:08:24 -07:00
Dominik Adamski 98ed6e1069 [Flang][OpenMP] Fix conversion of nested loops for SIMD directive
Flang was not able to convert simd directive which contains nested
Fortran loops. The nested Fortran loops inside SIMD directive
are modelled as FIR loops and they need to be translated into LLVM
MLIR dialect.

Differential Revision: https://reviews.llvm.org/D131402

Reviewed by: peixin

Signed-off-by: Dominik Adamski <dominik.adamski@amd.com>
2022-08-10 03:54:30 -05:00
John Demme d44f8a50d5 [MLIR] [Python] Fix the Windows build broken by d747a17
Windows builds require all control paths return. Since we don't have
`llvm_unreachable` in the Python bindings, just return `None`.
2022-08-09 20:07:33 -07:00
John Demme d747a170a4 [MLIR] [Python] Fix `Value.owner` to handle BlockArgs
Previously, calling `Value.owner()` would C++ assert in debug builds if
`Value` was a block argument. Additionally, the behavior was just wrong
in release builds. This patch adds support for BlockArg Values.
2022-08-09 19:37:04 -07:00
Aart Bik 5110eb8b67 [mlir][sparse] fix doc
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D131527
2022-08-09 17:19:39 -07:00
Jeff Niu 2f0e50c69b [mlir] Fix build of toyc-ch6 (NFC) 2022-08-09 15:58:49 -04:00
Jeff Niu f201476518 [mlir] Cleanup DenseArrayAttrBase definition and expose raw API
This patch cleans up the definition of `DenseArrayAttrBase` by relying
more on ODS-generated methods. It also exposes an API for using the raw
data of a dense array, similar to `DenseIntOrFPElementsAttr::getRaw`.

Reviewed By: lattner, mehdi_amini

Differential Revision: https://reviews.llvm.org/D131450
2022-08-09 15:43:45 -04:00
Jeff Niu 0af643f3ce [mlir][LLVMIR] (NFC) Add convenience builders for ConstantOp
And clean up some of the user code
2022-08-09 15:34:36 -04:00
Aart Bik 60076a9eaf [mlir][sparse] fix switch statement bug on two binary ops
They appeared at the wrong place in the switch, treating
them as unary op rather than binary op.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D131509
2022-08-09 11:58:05 -07:00
Aart Bik 0cc6aac9fa [mlir][sparse] update bibliography of sparse tensor dialect
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D131513
2022-08-09 11:57:01 -07:00
Lei Zhang 15135553c4 [mlir][spirv] Use functors for default memory space mappings
This makes it easier to use as a utility function to query the
mappings, including the reverse.

This commit also drops some storage classes that aren't needed
for now.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D131411
2022-08-09 14:38:27 -04:00
Lei Zhang 89b595e141 [mlir][spirv] Detach memory space mapping from type conversion
This commit moves MemRef memory space to SPIR-V storage class
conversion out of the main SPIR-V type converter. Now the mapping
should happen as a prelimiary step before performing the final
conversion to SPIR-V. Flows are expect to write their own memory
space mappings like the `MapMemRefStorageClassPass` to handle
memory space mappings according to their needs.

This is needed because SPIR-V is serving multiple client APIs,
including Vulkan and OpenCL. Different client APIs might want
to use different storage classes for buffers in a particular
memory space, e.g., `StorageBuffer` for Vulkan vs. `CrossWorkgroup`
for OpenCL when converting the default 0 memory space.  Hardcoding
a specific mapping makes that hard. While it's possible to embed
selection logic further inside the main type converter, it will
make the main type converter even complicated. So it's better to
separate the concerns, as mapping the memory space is really
concretizing the meaning of those numeric memory spaces in the
particular context of SPIR-V lowering.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D131410
2022-08-09 14:30:43 -04:00
Lei Zhang b83d0d46c0 [mlir][spirv] Make MemRef memory space mapping pass more flexible
* Avoid restricting the pass to to builtin module ops. The pass
  should be able to run on any region ops.
* Avoid hardcoding func FuncOp when handling functions. Instead,
  use the function op interface.
* Assigns the default mapping in the constructor. So for cases
  where we are using the pass in a pipeline, we still have a
  meaningful default.

Along the way, dropped uncessary unrealized conversion casts and
use full conversion. The pass should be able to convert all sorts
of ops; there is really no need to have such bridages.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D131409
2022-08-09 14:21:50 -04:00
Jerry Wu 66c2b76846 [MLIR] Extend vector.gather to accept tensor as base
In addition to memref, accept ranked tensor as the base operand of vector.gather, similar to vector.trasnfer_read.

This will allow us to vectorize noncontiguous tensor.extract into vector.gather. Full discussion can be found here: https://github.com/iree-org/iree/issues/9198

Reviewed By: hanchung, dcaballe

Differential Revision: https://reviews.llvm.org/D130097
2022-08-09 11:19:16 -07:00
Jerry Wu 3597727fa7 [MLIR] Support lowering n-D arith.index_cast to LLVM
Previously we can only lower arith.index_cast with 1-D vectors to LLVM. This change added the support for n-D vectors.

Reviewed By: ftynse, hanchung

Differential Revision: https://reviews.llvm.org/D129907
2022-08-09 11:17:52 -07:00
Lei Zhang a29fffc475 [mlir][spirv] Migrate to use specalized enum attributes
Previously we are using IntegerAttr to back all SPIR-V enum
attributes. Therefore we all such attributes are showed like
IntegerAttr in IRs, which is barely readable and breaks
roundtripability of the IR. This commit changes to use
`EnumAttr` as the base directly so that we can have separate
attribute definitions and better IR printing.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D131311
2022-08-09 14:14:54 -04:00
Aart Bik b09f6b471f [mlir][sparse] improve semi-ring doc
Spell out SparseVector instead of just using SparseVec

Reviewed By: jim22k, bixia

Differential Revision: https://reviews.llvm.org/D131511
2022-08-09 10:59:08 -07:00
Jakub Kuderski 08cc03befd [mlir][spirv] Clean up SPIRVOps.cpp. NFC.
Resolve almost all clang tidy warnings in this file:
1. Clean up string constants.
2. Use consistent argument names across function declarations and definitions. Rename `state` - > `result`, which is consistent with the other dialects.
3. Remove misleading function parameter name comments (`argTypes`). This did not match the actual function argument (`bool enableNameShadowing`).
4. Simplify calls to `is_splat`.

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D131297
2022-08-09 10:40:43 -04:00
Benjamin Kramer 9fa59e7643 [mlir] Use C++17 structured bindings instead of std::tie where applicable. NFCI 2022-08-09 13:34:17 +02:00
Shraiysh Vaishay dd14d471a5 [mlir][OpenMP] omp.parallel side effects
Add `NoSideEffects` trait to omp.parallel operation.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D130657
2022-08-09 08:40:47 +00:00
Fangrui Song 5146f84fd6 LLVM_NODISCARD => [[nodiscard]]. NFC 2022-08-09 07:16:34 +00:00
Fangrui Song fc63c0542c [mlir] LLVM_FALLTHROUGH => [[fallthrough]]. NFC 2022-08-08 20:56:05 -07:00
Thomas Raoux 2eb50cee11 [mlir][tosa] Use arith::maxf/arith::minf in lowering from tosa
now that `arith` dialect has maxf/minf use it instead of cmp/select.
Also refactor clamp helpers to make them simlper.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D131426
2022-08-09 01:10:32 +00:00
not-jenni b73e8325fb [mlir][tosa] Updates tosa.equal to use the InferTensorType interface
Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D130373
2022-08-08 16:11:30 -07:00
Thomas Raoux 8d7c1c55a4 [mlir][vector] Fix warp distribution test
The test was using a missing prefix. Add the prefix and fix the naming.

Found by @csigg

Reviewed By: csigg

Differential Revision: https://reviews.llvm.org/D131428
2022-08-08 20:57:44 +00:00
Aart Bik 6b3bc7cd3c [mlir][sparse] improve sparse attribute documentation
Moved some parts from comments (not user facing) to the actual description
(user facing). Rephrased a bit as well.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D131418
2022-08-08 13:21:24 -07:00
Lei Zhang d9728a9baa [mlir][spirv] Unify mixed scalar/vector primitive type resources
This further relaxes the requirement to allow aliased resources
to have different primitive types and some are scalars while the
other are vectors.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D131207
2022-08-08 14:30:14 -04:00
Peiming Liu de907138ec [mlir][sparse] Add new concatente operator to sparse tensor
See https://www.tensorflow.org/xla/operation_semantics#concatenate for the operator semantics

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D131111
2022-08-08 17:23:43 +00:00
Benjamin Kramer 2cbfa93f42 [mlir][math] Fix pythong bindings after 00f7096d31 2022-08-08 19:14:44 +02:00
Mehdi Amini 95b3ff08f5 [mlir][doc] Cross link the dependent dialect section to the tablegen field definition 2022-08-08 16:57:33 +00:00
Rajas Vanjape 333f98b4b6 [mlir][sparse][nfc] Use tensor.generate in sparse integration tests
Currently, dense tensors are initialized in Sparse Integration tests using
"buffer.tensor_alloc and scf.for" . This makes code harder to read and maintain.
This diff uses tensor.generate instead to initialize dense tensors.

Testing: Ran integration tests after building with -DLLVM_USE_SANITIZER=Address flag.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D131404
2022-08-08 16:44:45 +00:00
lorenzo chelini fb33b406df [MLIR][Linalg] Remove `TiledLoops` from tiling options
TiledLoopOp has been removed in: 1a829d2d06

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131383
2022-08-08 18:24:58 +02:00
Benjamin Kramer 2960299986 [ADT] Retire llvm::apply_tuple in favor of C++17 std::apply 2022-08-08 18:23:38 +02:00
Lei Zhang 4bd25d0b81 [mlir][spirv] Refresh base definitions to latest spec (v1.6)
This commit updates all SPIR-V enum definitions to match the latest
specification (v1.6 revision 2). Along the way, fixed some issues
in `gen_spirv_dialect.py` and added a new script for refreshing
all op definitions for such cases.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D131293
2022-08-08 12:22:32 -04:00
Jeff Niu 7d9fc95b85 [mlir][math] Add `math.absi` op
Adds an integer absolute value op to the math dialect.

When converting to LLVM, this op is lowered to the LLVM `abs` intrinsic.
When converting to SPIRV, this op is lowered to `spv.GL.SAbs`.

Depends on D131325

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131327
2022-08-08 11:05:01 -04:00
Jeff Niu 00f7096d31 [mlir][math] Rename math.abs -> math.absf
To make room for introducing `math.absi`.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131325
2022-08-08 11:04:58 -04:00
Arjun P dc07d2c91d [MLIR][Presburger] make sample test with integer lexmin use containsPointNoLocal
IntegerPolyhedron::findIntegerLexmin currently does not return values of
the local ids, so when a test for sampling includes a set with locals, the
result of findIntegerLexmin should be checked using containsPointNoLocal,
not containsPoint.
2022-08-08 14:46:02 +01:00
Markus Böck 8805cf2660 [mlir] Remove redundant `inline` from D131323 2022-08-08 13:18:09 +02:00
Ashay Rane d1bb3016dd
[mlir] fix `add_tablegen()` macro to allow installing mlir-pdll
Prior to this patch, the `add_tablegen()` macro in
llvm/cmake/modules/TableGen.cmake added the install rule only if
`project` matched `LLVM` or `MLIR`.  This patch adds an optional
`DESTINATION` argument, which, if non-empty, decides whether (and where)
to install the tablegen tool, thus eliminating the need for
project-specific overrides.  This patch also updates all other
invocations of the `add_tablegen()` macro.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D131282
2022-08-07 15:48:38 -07:00
Jacques Pienaar 10de551297 [mlir][python] Address deprecation warning for hasValue 2022-08-07 15:28:18 -07:00
Kazu Hirata f616a63db2 [mlir] Use value instead of getValue 2022-08-07 11:59:37 -07:00
Kazu Hirata 5b14c7aee8 [mlir] Fix a warning
This patch fixes:

  llvm-project/mlir/include/mlir/IR/OpDefinition.h:1544:19: error: use
  of bitwise '|' with boolean operands
  [-Werror,-Wbitwise-instead-of-logical]
2022-08-07 11:13:19 -07:00
Jacques Pienaar 7602e285f6 [mlir] Flip to prefixed accessors (NFC)
Missed these in td files.
2022-08-07 08:46:15 -07:00
Jacques Pienaar d3b3f7653d [mlir] Flip to prefixed accessors (NFC) 2022-08-07 04:55:58 -07:00
Markus Böck 26d811b3ec [mlir] Make use of C++17 language features
Now that C++17 is enabled in LLVM, a lot of the TODOs and patterns to emulate C++17 features can be eliminated.
The steps I have taken were essentially:
```
git grep C++17
git grep c++17
git grep "initializer_list<int>"
```
and address given comments and patterns.
Most of the changes boiled down to just using fold expressions rather than initializer_list.

While doing this I also discovered that Clang by default restricts the depth of fold expressions to 256 elements. I specifically hit this with `TestDialect` in `addOperations`. I opted to not replace it with fold expressions because of that but instead adding a comment documenting the issue.
If any other functions may be called with more than 256 elements in the future we might have to revert other parts as well.
I don't think this is a common occurence besides the `TestDialect` however. If need be, this could potentially be fixed via `mlir-tblgen` in the future.

Differential Revision: https://reviews.llvm.org/D131323
2022-08-07 11:16:49 +02:00
Kazu Hirata af2d2d7759 [mlir] Remove redundaunt return statements (NFC)
Identified with readability-redundant-control-flow.
2022-08-07 00:16:13 -07:00
John Demme 5e9b6a2243 [MLIR] Add MlirValue to PybindAdapters
Allows out-of-tree users to automatically cast to/from MlirValue.
2022-08-06 21:58:46 -07:00
Jacques Pienaar 4401bde317 [mlir][tosa] Flip to prefixed form.
Flipped to _Both 2 weeks ago, flipping to _Prefixed now.
2022-08-06 19:24:40 -07:00
Kazu Hirata c8e6ebd74e Use value instead of getValue (NFC) 2022-08-06 11:21:39 -07:00
Kazu Hirata 9750648cb4 [mlir, flang] Use has_value instead of hasValue (NFC) 2022-08-06 11:12:47 -07:00
Markus Böck 1c5a50e328 [mlir][tblgen] Refact mlir-tblgen main into its own library
This has previously been done for `mlir-opt` and `mlir-reduce` and roughly the same approach has been done here.

The use case for having a separate library is that it is easier for downstream to make custom TableGen backends/executable that work on top of the utilities that are defined in `mlir/TableGen`.
The customization point here is the same one as for any upstream TableGen backends: One can add a new generator by simply creating a global instance of `mlir::GenRegistration`.

Differential Revision: https://reviews.llvm.org/D131112
2022-08-06 14:07:37 +02:00
Eugene Zhulenev 5f1c7e2cc5 [mlir] Use SymbolTableCollection to lookup referenced symbol in AddressOfOp
Depends On D131285

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D131291
2022-08-05 14:05:03 -07:00
Eugene Zhulenev 51bc82d147 [mlir] Implement SymbolUserOpInterface in LLVM::CallOp
Avoid expensive calls to `SymbolTable::lookupNearestSymbolFrom` in verifier

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D131285
2022-08-05 13:50:31 -07:00
Jakub Kuderski 5c16eeb7ee [mlir][spirv] Define spv.IAddCarry
Based on `spv.ISubBorrow` from D127909.
Also resolved some clang-tidy warnings.

Reviewed By: antiagainst, ThomasRaoux

Differential Revision: https://reviews.llvm.org/D131281
2022-08-05 16:45:51 -04:00
Eugene Zhulenev 292e8ed49a [mlir] Use SymbolUserOpInterface in LLVM::AddressOfOp verifier
Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D131271
2022-08-05 10:51:30 -07:00
Lei Zhang 1f7544a679 [mlir][spirv] Add default Vulkan memory space to storage class mapping
Reviewed By: ThomasRaoux, kuhar

Differential Revision: https://reviews.llvm.org/D131128
2022-08-05 12:30:14 -04:00
Lei Zhang 713f85d595 [mlir][spirv] Add a pass to map memref memory space
MemRef types now can carry an attribute to represent the memory
space. Still, upper layers in the compilation stack mostly use
nuemric values. They don't mean much (other than differentiating
separate memory domains) in MLIR's multi-level settings. Those
numeric memory space inside MemRef types need to be translated
into concrete SPIR-V storage classes during lowering to pin down
to concrete memory types.

Thus far we have been hardcoding an arbitrary mapping from memory
space to storage class for converting MemRef types. This works fine
for only targeting Vulkan; it falls apart if we want to target other
SPIR-V consumers like OpenCL, as different consumers might want
different storage classes for the buffer/variable of the same
lifetime. For example, StorageClass in Vulkan vs. CrossWorkgroup
in OpenCL.

So putting up a new pass to let the user to control how to map
MemRef memory spaces into SPIR-V storage classes. This provides
more flexibility and can address the awkwardness in the current
SPIR-V type converter. This pass should be the prelimiary step
towards lowering MemRef related types/ops into SPIR-V.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D130317
2022-08-05 12:20:06 -04:00
Alexander Belyaev 6b03bae346 Revert "[mlir] Extract offsets-sizes-strides computation from `makeTiledShape(s)`."
This reverts commit 56d94b3b90.
2022-08-05 14:53:35 +02:00
jacquesguan 40d74fcb55 [mlir][Math] Add constant folder for Atan2Op.
This patch adds constant folder for Atan2Op which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D131050
2022-08-05 10:30:58 +08:00
Kevin Gleason 1bfc052705 [MLIR] Fix arith.cmpi assembly syntax in the doc to match the implementation (NFC) 2022-08-04 23:22:50 +00:00
Kevin Gleason e38b0fb008 [MLIR] Fix arith.cmpf assembly syntax in the doc to match the implementation (NFC) 2022-08-04 23:15:36 +00:00
Aart Bik 7f5b167336 [mlir][sparse] fix bug in complex zero detection
We were checking real-part twice, not real/imag-part.
The new test only passes after the bug fix.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D131190
2022-08-04 13:35:13 -07:00
Krzysztof Drewniak 6329562249 [mlir][AMDGPU] Explicitly truncate memory addresses in buffer ops
As a percaution, truncate memory addresses passed to kernels to 48 bits,
since bits 48-63 of the buffer descriptor are used for the stride field
and, on gfx10, to control swizzling.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D131016
2022-08-04 19:42:33 +00:00
Mehdi Amini 0e051c02d5 Revert "[mlir][test] Fix IR/AttributeTest.cpp compilation on Solaris"
This reverts commit 07aaa35f74.

This breaks the Windows bot, and while the fix addressed the
`char`/`int8_t` case, it does not make sense for other cases like
`float`.
2022-08-04 18:48:22 +00:00
Arjun P 1486a2eaf0 [MLIR][Presburger] SlowMPInt::gcd: fix crash when sizes differ
Reviewed By: Groverkss

Differential Revision: https://reviews.llvm.org/D131186
2022-08-04 19:15:50 +01:00
Arjun P bad95b72ee [MLIR][Presburger] fourier-motzkin: check if all LCMs are 1 using a bool instead of by multiplying them
This can easily overflow and it is possible for these unsigned overflows to result in incorrect results.
For example, the two LCMs could be 641 and 6700417, which multiply to 2^32 + 1, which overflows to 1.
Unsigned overflows already occur in the existing tests.

Also, when switching to arbitrary-precision arithmetic, this results in a many
large integer multiplications resulting in a significant slowdown.

Reviewed By: Groverkss

Differential Revision: https://reviews.llvm.org/D131184
2022-08-04 19:14:39 +01:00
lorenzo chelini 954de25a92 [MLIR] TilingInterface: Avoid map when tile divides iteration domain
Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131080
2022-08-04 19:43:59 +02:00
Rainer Orth 07aaa35f74 [mlir][test] Fix IR/AttributeTest.cpp compilation on Solaris
The `IR/AttributeTest.cpp` test fails to compile on Solaris:

  /vol/llvm/src/llvm-project/local/mlir/unittests/IR/AttributeTest.cpp:223:36: error: no matching function for call to 'allocate'
        AttrT::get(type, "resource", UnmanagedAsmResourceBlob::allocate(data));
                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /vol/llvm/src/llvm-project/local/mlir/unittests/IR/AttributeTest.cpp:237:3: note: in instantiation of function template specialization 'checkNativeAccess<mlir::detail::DenseResourceElementsAttrBase<int8_t>, char>' requested here
    checkNativeAccess<AttrT, T>(builder.getContext(), llvm::makeArrayRef(data),
    ^
  /vol/llvm/src/llvm-project/local/mlir/unittests/IR/AttributeTest.cpp:258:3: note: in instantiation of function template specialization 'checkNativeIntAccess<mlir::detail::DenseResourceElementsAttrBase<int8_t>, char>' requested here
    checkNativeIntAccess<DenseI8ResourceElementsAttr, int8_t>(builder, 8);
    ^
  /vol/llvm/src/llvm-project/local/mlir/include/mlir/IR/AsmState.h:221:3: note: candidate template ignored: requirement '!std::is_same<char, char>::value' was not satisfied [with T = char]
    allocate(ArrayRef<T> data, bool dataIsMutable = false) {
    ^
  /vol/llvm/src/llvm-project/local/mlir/include/mlir/IR/AsmState.h:214:26: note: candidate function not viable: requires at least 2 arguments, but 1 was provided
    static AsmResourceBlob allocate(ArrayRef<char> data, size_t align,
                           ^

I suspect this happens because `char` is `signed` by default on Solaris.

Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D131148
2022-08-04 18:56:05 +02:00
Aart Bik c7bb69bc75 [mlir][sparse] replace zero yield generic op with copy in allocation
This prepares patterns that sometimes are generated by the front-end
and would prohibit fusion of SDDMM flavored kernels.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D131126
2022-08-04 09:33:57 -07:00
Frederik Gossen f98d73f4e0 [MLIR] Make the implementations for getMixedOffsets/Sizes/Strides independent of OffsetSizeAndStrideOpInterface
The functions are effectively independent of the interface already, however, they take it as an argument for no reason.
The current state complicates reuse outside of MLIR.

Differential Revision: https://reviews.llvm.org/D131120
2022-08-04 11:58:15 -04:00
Eugene Zhulenev ec7f4a7c5d [mlir:LLVM] Do not lookup symbol twice in the addressof verifier
`SymbolTable::lookupSymbolIn` is an expensive operation and we do not want to do it twice

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D131145
2022-08-04 08:42:38 -07:00
Tres Popp 448adfee05 [mlir] Only conditionally lower CF branching ops to LLVM
Previously cf.br cf.cond_br and cf.switch always lowered to their LLVM
equivalents. These ops are all ops that take in some values of given
types and jump to other blocks with argument lists of the same types. If
the types are not the same, a verification failure will later occur. This led
to confusions, as everything works when func->llvm and cf->llvm lowering
both occur because func->llvm updates the blocks and argument lists
while cf->llvm updates the branching ops. Without func->llvm though,
there will potentially be a type mismatch.

This change now only lowers the CF ops if they will later pass
verification. This is possible because the parent op and its blocks will
be updated before the contained branching ops, so they can test their
new operand types against the types of the blocks they jump to.

Another plan was to have func->llvm only update the entry block
signature and to allow cf->llvm to update all other blocks, but this had
2 problems:
1. This would create a FuncOp lowering in cf->llvm lowering which is
   awkward
2. This new pattern would only be applied if the containing FuncOp is
   marked invalid. This is infeasible with the shared LLVM type
   conversion/target infrastructure.

See previous discussions at
https://discourse.llvm.org/t/lowering-cf-to-llvm/63863 and
https://github.com/llvm/llvm-project/issues/55301

Differential Revision: https://reviews.llvm.org/D130971
2022-08-04 16:36:27 +02:00
Jeff Niu d0541b4700 [mlir] Add I1 support to DenseArrayAttr
This patch adds a DenseI1ArrayAttr to support arrays of i1. Importantly,
the implementation is as a simple `ArrayRef<bool>` instead of using bit
compression, which was problematic in DenseElementsAttr.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D130957
2022-08-04 10:24:45 -04:00
Alexander Belyaev 56d94b3b90 [mlir] Extract offsets-sizes-strides computation from `makeTiledShape(s)`.
This change separates computation of the actual parameters of the subset and
the materialization of subview/extract_slice. That way the users can still use
Linalg tiling logic even if they use different operations to materialize the
subsets.

Differential Revision: https://reviews.llvm.org/D131053
2022-08-04 11:23:58 +02:00
Nikita Popov 57a9bccec7 [MLIR] Fix checks for native arch
Using if (TARGET ${LLVM_NATIVE_ARCH}) only works if MLIR is built
together with LLVM, but not for standalone builds of MLIR. The
correct way to check this is
if (${LLVM_NATIVE_ARCH} IN_LIST LLVM_TARGETS_TO_BUILD), as the
LLVM build system exports LLVM_TARGETS_TO_BUILD.

To avoid repeating the same check many times, add a
MLIR_ENABLE_EXECUTION_ENGINE variable.

Differential Revision: https://reviews.llvm.org/D131071
2022-08-04 11:10:08 +02:00
Adrian Kuegel 84d4bb78e0 [mlir][Linalg] Inline an interface method to its only user.
It seems only the default implementation is ever used, so it doesn't seem
necessary to include this method in the interface.

Differential Revision: https://reviews.llvm.org/D130986
2022-08-04 08:39:41 +02:00
Michele Scuttari e90deaf121 [MLIR] Reconciliation of chains of unrealized casts
The reconciliation pass has been improved to introduce the support for chains of casts, thus not limiting anymore the reconciliation to just consider pairs of unrealized casts.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D130711
2022-08-03 11:57:20 +00:00
Rainer Orth 75747e6e11 [mlir] Fix Analysis/Presburger/Utils.cpp compilation with GCC 11
As reported in Issue #56850, mlir/lib/Analysis/Presburger/Utils.cpp doesn't
compile on Solaris 11.4/SPARC with the bundled GCC 11, as seen when testing
LLVM 15.0.0 rc1:

  /var/llvm/reltest/llvm-15.0.0-rc1/rc1/llvm-project/mlir/include/mlir/Analysis/Presburger/MPInt.h:260:47:
error: inlining failed in call to ‘always_inline’ ‘int64_t
mlir::presburger::int64FromMPInt(const mlir::presburger::MPInt&)’: indirect
function call with a yet undetermined callee

This patch hacks around this and allowed the build to finish.

Tested on `sparcv9-sun-solaris2.11`.

Differential Revision: https://reviews.llvm.org/D131060
2022-08-03 11:48:44 +02:00
Alexander Belyaev 3285f94244 [mlir] Disable `misc-const-correctness` clang-tidy check.
https://clang.llvm.org/extra/clang-tidy/checks/misc/const-correctness.html
2022-08-03 10:18:43 +02:00
jacquesguan 752c9d0dab [mlir][Math] Add constant folder for AtanOp.
This patch adds constant folder for AtanOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130983
2022-08-03 14:10:02 +08:00
Kazu Hirata dc860d55eb [mlir] Add has_value and value to OptionalParseResult
llvm::Optional is in the process of switching to the
std::optional-like interface with has_value/value as opposed to
hasValue/getValue.

This patch adds has_value and value to enable the same transition.

Differential Revision: https://reviews.llvm.org/D130819
2022-08-02 22:16:55 -07:00
Mehdi Amini 4f0262c164 Fix use-after-free in SymbolTable::replaceAllSymbolUses
In some cases the recursion will grow the `visited` hash table and
invalidate the cached iterator.
(caught with ASAN)

Differential Revision: https://reviews.llvm.org/D131027
2022-08-02 22:30:17 +00:00
Krzysztof Drewniak c2fc8d9b95 [mlir][GPU] Allow bare pointer memrefs when calling GPU kernels
In the ROCm runtime (and probably CUDA as well), all kernel arguments
are aligned. Therefore, enable using bare pointers for memref
arguments to kernels when these memrefs have static shape and a
trivial layout.

This is a substantial optimization to launching kernels that use
memrefs with known, static sizes, since it causes the kernel launch
packet to no longer include information already known to the kernel,
which can enable packing the kernel launch arguments into launch
packets instead of having to allocate an entire separate structure to
hold unneeded memref information.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D130716
2022-08-02 20:58:34 +00:00
Aart Bik ce3d0e87ac [mlir][sparse] enable SDDMM-flavored fusion
This rewriting was no longer functional after recent migration to one shot
bufferization. However, this revision makes it work again, with a CHECK test
to ensure fusion happens. Note that functionality is tested by several
integration tests.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D130996
2022-08-02 12:40:04 -07:00
Aart Bik 9921ef73c8 [mlir][sparse] remove singleton dimension level type (for now)
Although we have plans to support this, and many other, dimension level type(s), currently the tag is not supported. It will be easy to add this back once support is added.

NOTE: based on discussion in https://discourse.llvm.org/t/overcoming-sparsification-limitation-on-level-types/62585

https://github.com/llvm/llvm-project/issues/51658

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D131002
2022-08-02 11:48:49 -07:00
Michele Scuttari 29fbe60009 [MLIR] Rename the generic LLVM allocation and deallocation functions
The generic allocation and deallocation instructions, which are optionally used during the MemRef -> LLVM conversion, should have a name that is specifically bound to their origin, that is the conversion pass itself.

Reviewed By: silvas

Differential Revision: https://reviews.llvm.org/D130588
2022-08-02 18:23:14 +00:00
jacquesguan 008ea1c201 [mlir][Math] Add constant folder for TanhOp.
This patch adds constant folder for TanhOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130960
2022-08-02 19:37:27 +08:00
Alex Zinenko 64bb0ae75f [mlir] add TOC to top-level documents
Multiple top-level MLIR documents did not have a table of contents tag,
making them harder to nagivate.
2022-08-02 13:22:40 +02:00
Adrian Kuegel b395c0f0cd [mlir] Update comment now that DenseArrayAttr has Tensor type. 2022-08-02 12:31:22 +02:00
Stephan Herhut 09ca1c0656 [mlir] Use EXPECT_DEBUG_DEATH in unit test
This allows the tests to also pass when compiled in opt mode.
2022-08-02 11:16:26 +02:00
Martin Storsjö 112499f35f [mlir] Fix calling the native mlir-tblgen tool when cross compiling flang
When the mlir-tblgen tool is set up, the `MLIR_TABLEGEN_EXE` variable
is set, which either points to the mlir-tblgen tool built in the current
cmake build, or points to one built in a nested cmake build (if cross
conpiling, or if building with e.g. `LLVM_OPTIMIZED_TABLEGEN`.

The `MLIR_TABLEGEN_EXE` variable is only set within the scope of the
mlir/CMakeLists.txt file, so it's unavailable in sibling level projects
such as flang.

Set the `MLIR_TABLEGEN_EXE` and the `MLIR_TABLEGEN_TARGET` variables
as global, so that flang can use them properly without guessing.

Differential Revision: https://reviews.llvm.org/D130350
2022-08-02 10:58:24 +03:00
jacquesguan f1033a3f47 [mlir][Math] Add constant folder for TanOp.
This patch adds constant folder for TanOp which only supports single and double precision floating-point.

Differential Revision: https://reviews.llvm.org/D130873
2022-08-02 11:20:53 +08:00
Jeff Niu ff52ad796c [mlir] Change DenseArrayAttr to TensorType
Previously, DenseArrayAttr used VectorType for its shaped type.
VectorType is problematic for arrays because it doesn't support zero
dimensions, meaning that an empty array would have `vector<i32>` as its
type. ElementsAttr would think that an empty dense array is size 1, not
0. This patch switches over to TensorType, which does support zero
dimensions.

Fixes #56860

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D130921
2022-08-01 22:17:28 -04:00
Manish Gupta 14d79afeae [mlir][NVGPU] nvgpu.mmasync on F32 through TF32
Adds optional attribute to support tensor cores on F32 datatype by lowering to `mma.sync` with TF32 operands. Since, TF32 is not a native datatype in LLVM we are adding `tf32Enabled` as an attribute to allow the IR to be aware of `MmaSyncOp` datatype. Additionally, this patch adds placeholders for nvgpu-to-nvgpu transformation targeting higher precision tf32x3.

For mma.sync on f32 input using tensor cores there are two possibilites:
(a) tf32   (1 `mma.sync` per warp-level matrix-multiply-accumulate)
(b) tf32x3 (3 `mma.sync` per warp-level matrix-multiply-accumulate)

Typically, tf32 tensor core acceleration comes at a cost of accuracy from missing precision bits. While f32 has 23 precision bits, tf32 has only 10 precision bits. tf32x3 aims to recover the precision bits by splitting each operand into two tf32 values and issue three `mma.sync` tensor core operations.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D130294
2022-08-01 23:23:27 +00:00
River Riddle 40abd7ea64 [mlir] Remove OpaqueElementsAttr
This attribute is technical debt from the early stages of MLIR, before
ElementsAttr was an interface and when it was more difficult for
dialects to define their own types of attributes. At present it isn't
used at all in tree (aside from being convenient for eliding other
ElementsAttr), and has had little to no evolution in the past three years.

Differential Revision: https://reviews.llvm.org/D129917
2022-08-01 15:00:54 -07:00
Mehdi Amini 0a17692148 [MLIR] Reduce precision check for expm1 folder: this is dependent on libm (NFC)
We noticed this failing depending on the platform, checking the last few
digit isn't necessary for this test anyway.
2022-08-01 21:26:51 +00:00
River Riddle 995ab92964 [mlir] Add a new builtin DenseResourceElementsAttr
This attributes is intended cover the current set of use cases that abuse
DenseElementsAttr, e.g. when the data is large. Using resources for large
data is one of the major reasons why they were added; e.g. they can be
deallocated mid-compilation, they support a wide variety of data origins
(e.g, heap allocated, mmap'd, etc.), they can support mutation, etc.

I considered at length not having a builtin variant of this, and instead
having multiple versions of this attribute for dialects that are interested,
but they all boiled down to the exact same attribute definition. Given the
generality of this attribute, it feels more aligned to keep it next to DenseArrayAttr
(given that DenseArrayAttr covers the "small" case, and DenseResourcesElementsAttr
covers the "large" case). The underlying infra used to build this attribute is
general, and having a builtin attribute doesn't preclude users from defining
their own when it makes sense (they can even share a blob manager with the
builtin dialect to avoid data duplication).

Differential Revision: https://reviews.llvm.org/D130022
2022-08-01 12:37:16 -07:00
River Riddle 5f58e14b36 [mlir] Add a generic DialectResourceBlobManager to simplify resource blob management
The DialectResourceBlobManager class provides functionality for managing resource blobs
in a generic, dialect-agnostic fashion. In addition to this class, a dialect interface and custom
resource handle are provided to simplify referencing and interacting with the manager. These
classes intend to simplify the work required for dialects that want to manage resource blobs
during compilation, such as for large elements attrs.  The old manager for the resource example
in the test dialect has been updated to use this, which provides and cleaner and more consistent API.

This commit also adds new HeapAsmResourceBlob and ImmortalAsmResourceBlob to simplify
creating resource blobs in common scenarios.

Differential Revision: https://reviews.llvm.org/D130021
2022-08-01 12:37:16 -07:00
Krzysztof Drewniak 938fe9f277 [mlir][Arith] Fix up integer range inference for truncation
Attempting to apply the range analysis to real code revealed that
trunci wasn't correctly handling the case where truncation would
create wider ranges - for example, if we truncate [255, 257] : i16 to
i8, the result can be 255, 0, or 1, which isn't a contiguous range of
values.

The previous implementation would naively map this to [255, 1], which
would cause issues with unsigned ranges and unification.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D130501
2022-08-01 19:29:53 +00:00