C API test uses FileCheck comments inside C code and needs to
temporarily switch off clang-format to prevent it from messing with
FileCheck directives. A recently landed commit forgot to turn it back on
after a block of FileCheck comments. Fix that.
ConvOp vectorization supports now only convolutions of static shapes with dimensions
of size either 3(vectorized) or 1(not) as underlying vectors have to be of static
shape as well. In this commit we add support for convolutions of any size as well as
dynamic shapes by leveraging existing matmul infrastructure for tiling of both input
and kernel to sizes accepted by the previous version of ConvOp vectorization.
In the future this pass can be extended to take "tiling mask" as a user input which
will enable vectorization of user specified dimensions.
Differential Revision: https://reviews.llvm.org/D87676
This patch provides C API for MLIR affine map.
- Implement C API for AffineMap class.
- Add Utils.h to include/mlir/CAPI/, and move the definition of the CallbackOstream to Utils.h to make sure mlirAffineMapPrint work correct.
- Add TODO for exposing the C API related to AffineExpr and mutable affine map.
Differential Revision: https://reviews.llvm.org/D87617
Add missing operands to represent copin with readonly modifier, copyout with zero
modifier, create with zero modifier and default clause.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D87733
Numerous MLIR functions return instances of `StringRef` to refer to a
non-owning fragment of a string (usually owned by the context). This is a
relatively simple class that is defined in LLVM. Provide a simple wrapper in
the MLIR C API that contains the pointer and length of the string fragment and
use it for Standard attribute functions that return StringRef instead of the
previous, callback-based mechanism.
Reviewed By: stellaraccident
Differential Revision: https://reviews.llvm.org/D87677
Add a verifier for the loop op in the OpenACC dialect. Check basic restriction
from 2.9 Loop construct from the OpenACC 3.0 specs.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D87546
This patch adds the missing print for the vector_length in the parallel operation.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D87630
This add canonicalizer for
- extracting an element from a dynamic_tensor_from_elements
- propagating constant operands to the type of dynamic_tensor_from_elements
Differential Revision: https://reviews.llvm.org/D87525
When packing function results into a structure during the standard-to-llvm
dialect conversion, do not assume the conversion was successful and propagate
nullptr as error state.
Fixes PR45184.
Reviewed By: nicolasvasilache
Differential Revision: https://reviews.llvm.org/D87605
Now backends spell out which namespace they want to be in, instead of relying on
clients #including them inside already-opened namespaces. This also means that
cppNamespaces should be fully qualified, and there's no implicit "::mlir::"
prepended to them anymore.
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D86811
Added support to the Std dialect cast operations to do casts in vector types when feasible.
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D87410
Type converter may fail and return nullptr on unconvertible types. The function
conversion did not include a check and was attempting to use a nullptr type to
construct an LLVM function, leading to a crash. Add a check and return early.
The rest of the call stack propagates errors properly.
Fixes PR47403.
Reviewed By: mehdi_amini
Differential Revision: https://reviews.llvm.org/D87075
This introduces a builder for the more general case that supports zero
elements (where the element type can't be inferred from the ValueRange,
since it might be empty).
Also, fix up some cases in ShapeToStandard lowering that hit this. It
happens very easily when dealing with shapes of 0-D tensors.
The SameOperandsAndResultElementType is redundant with the new
TypesMatchWith and prevented having zero elements.
Differential Revision: https://reviews.llvm.org/D87492
Addressed some CR issues pointed out in D87111. Formatting and other nits.
The original Diff D87111 - Add an option for unrolling loops up to a factor.
Reviewed By: bondhugula
Differential Revision: https://reviews.llvm.org/D87313
This revision refactors and cleans up a bunch of things to simplify StructuredOpInterface
before work can proceed on Linalg on tensors:
- break out pieces of the StructuredOps trait that are part of the StructuredOpInterface,
- drop referenceIterators and referenceIndexingMaps that end up being more confusing than useful,
- drop NamedStructuredOpTrait
Previously only the input type was printed, and the parser applied it to
both input and output, creating an invalid transpose. Print and parse
both types, and verify that they match.
Differential Revision: https://reviews.llvm.org/D87462
This patch adds a new named structured op to accompany linalg.matmul and
linalg.matvec. We needed it for our codegen, so I figured it would be useful
to add it to Linalg.
Reviewed By: nicolasvasilache, mravishankar
Differential Revision: https://reviews.llvm.org/D87292
Rationale:
After some discussion we decided that it is safe to assume 32-bit
indices for all subscripting in the vector dialect (it is unlikely
the dialect will be used; or even work; for such long vectors).
So rather than detecting specific situations that can exploit
32-bit indices with higher parallel SIMD, we just optimize it
by default, and let users that don't want it opt-out.
Reviewed By: nicolasvasilache, bkramer
Differential Revision: https://reviews.llvm.org/D87404
I was having a lot of trouble parsing the messages. In particular, the
messages like:
```
<stdin>:3:8: error: 'scf.if' op along control flow edge from Region #0 to scf.if source #1 type '!npcomprt.tensor' should match input #1 type 'tensor<?xindex>'
```
In particular, one thing that kept catching me was parsing the "to scf.if
source #1 type" as one thing, but really it is
"to parent results: source type #1".
Differential Revision: https://reviews.llvm.org/D87334
This commit specifies reduction dimensions for ConvOps. This prevents
running reduction loops in parallel and enables easier detection of kernel dimensions
which we will need later on.
Differential Revision: https://reviews.llvm.org/D87288
The current BufferPlacement transformation cannot handle loops properly. Buffers
passed via backedges will not be freed automatically introducing memory leaks.
This CL adds support for loops to overcome these limitations.
Differential Revision: https://reviews.llvm.org/D85513
Take advantage of the new `dynamic_tensor_from_elements` operation in `std`.
Instead of stack-allocated memory, we can now lower directly to a single `std`
operation.
Differential Revision: https://reviews.llvm.org/D86935
Currently, there is no option to allow for unrolling a loop up to a specific factor (specified by the user).
The code for doing that is there and there are benefits when unrolling is done to smaller loops (smaller than the factor specified).
Reviewed By: bondhugula
Differential Revision: https://reviews.llvm.org/D87111
This replaces the select chain for edge-padding with an scf.if that
performs the memory operation when the index is in bounds and uses the
pad value when it's not. For transfer_write the same mechanism is used,
skipping the store when the index is out of bounds.
The integration test has a bunch of cases of how I believe this should
work.
Differential Revision: https://reviews.llvm.org/D87241
In this commit a new way of convolution ops lowering is introduced.
The conv op vectorization pass lowers linalg convolution ops
into vector contractions. This lowering is possible when conv op
is first tiled by 1 along specific dimensions which transforms
it into dot product between input and kernel subview memory buffers.
This pass converts such conv op into vector contraction and does
all necessary vector transfers that make it work.
Differential Revision: https://reviews.llvm.org/D86619
With `dynamic_tensor_from_elements` tensor values of dynamic size can be
created. The body of the operation essentially maps the index space to tensor
elements.
Declare SCF operations in the `scf` namespace to avoid name clash with the new
`std.yield` operation. Resolve ambiguities between `linalg/shape/std/scf.yield`
operations.
Differential Revision: https://reviews.llvm.org/D86276
Vector to SCF conversion still had issues due to the interaction with the natural alignment derived by the LLVM data layout. One traditional workaround is to allocate aligned. However, this does not always work for vector sizes that are non-powers of 2.
This revision implements a more portable mechanism where the intermediate allocation is always a memref of elemental vector type. AllocOp is extended to use the natural LLVM DataLayout alignment for non-scalar types, when the alignment is not specified in the first place.
An integration test is added that exercises the transfer to scf.for + scalar lowering with a 5x5 transposition.
Differential Revision: https://reviews.llvm.org/D87150
* Resolves todos from D87091.
* Also modifies PyConcreteAttribute to follow suite (should be useful for ElementsAttr and friends).
* Adds a test to ensure that the ShapedType base class functions as expected.
Differential Revision: https://reviews.llvm.org/D87208
Based on the PyType and PyConcreteType classes, this patch implements the bindings of Shaped Type, Tensor Type and MemRef Type subclasses.
The Tensor Type and MemRef Type are bound as ranked and unranked separately.
This patch adds the ***GetChecked C API to make sure the python side can get a valid type or a nullptr.
Shaped type is not a kind of standard types, it is the base class for vectors, memrefs and tensors, this patch binds the PyShapedType class as the base class of Vector Type, Tensor Type and MemRef Type subclasses.
Reviewed By: stellaraccident
Differential Revision: https://reviews.llvm.org/D87091
Historically, the operations in the MLIR's LLVM dialect only checked that the
operand are of LLVM dialect type without more detailed constraints. This was
due to LLVM dialect types wrapping LLVM IR types and having clunky verification
methods. With the new first-class modeling, it is possible to define type
constraints similarly to other dialects and use them to enforce some
correctness rules in verifiers instead of having LLVM assert during translation
to LLVM IR. This hardening discovered several issues where MLIR was producing
LLVM dialect operations that cannot exist in LLVM IR.
Depends On D85900
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D85901
When allowed, use 32-bit indices rather than 64-bit indices in the
SIMD computation of masks. This runs up to 2x and 4x faster on
a number of AVX2 and AVX512 microbenchmarks.
Reviewed By: bkramer
Differential Revision: https://reviews.llvm.org/D87116
Sizes of tiles (subviews) are bigger by 1 than they should. Let's consider
1D convolution without batches or channels. Furthermore let m iterate over
the output and n over the kernel then input is accessed with m + n. In tiling
subview sizes for convolutions are computed by applying requested tile size
together with kernel size to the above mentioned expression thus let's say
for tile size of 2 the subview size is 2 + size(n), which is bigger by one
than it should since we move kernel only once. The problem behind it is that
range is not turned into closed interval before the composition. This commit
fixes the problem by turning ranges first into closed intervals by substracting
1 and after the composition back to half open by adding 1.
Differential Revision: https://reviews.llvm.org/D86638
Alex Zinenko