In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands).
Significant code changes occur here:
*) Vectorization: LoopAnalysis.cpp, Vectorize.cpp
*) Affine Transforms: Transforms/Utils/Utils.cpp
PiperOrigin-RevId: 256395088
This CL refactors tiling to enable tiling of views that are not just specified by a simple permutation. This allows the tiling of convolutions for which a new example is added.
PiperOrigin-RevId: 256346028
This CL is the first step of a refactoring unification of the control flow abstraction used in different dialects. The `std.for` loop accepts unrestricted indices to encode min, max and step and will be used as a common abstraction on the way to lower level dialects.
PiperOrigin-RevId: 256331795
This CL uses the generic CopyOp to promote a subview (constructed during tiling) into a new buffer + copy by:
1. Creating a new buffer for the subview.
2. Taking a view into the buffer and copying into it.
3. Adapting the linalg op to operating on the view from point 2.
Tiling is extended with a boolean flag to enable promoting views (all or nothing for now).
More specifically, the current implementation creates a buffer that is always of the full size of the ranges of the subview. This produces a buffer whose size may be bigger
than the actual size of the `subView` at the boundaries and is related to the full/partial tile problem.
In practice, we introduce a `buffer`, a `fullLocalView` and a `partialLocalView` such that:
* `buffer` is always the size of the subview in the full tile case.
* `fullLocalView` is a dense contiguous view into that buffer.
* `partialLocalView` is a dense non-contiguous slice of `fullLocalView`
that corresponds to the size of `subView` and accounting for boundary
effects.
The point of the full tile buffer is that constant static tile sizes are
folded and result in a buffer type with statically known size and alignment
properties.
Padding is introduced on the boundary tiles with a `fill` op followed by a partial `copy` op.
These behaviors will be refined later, on a per-need basis.
PiperOrigin-RevId: 256237319
As with Functions, Module will soon become an operation, which are value-typed. This eases the transition from Module to ModuleOp. A new class, OwningModuleRef is provided to allow for owning a reference to a Module, and will auto-delete the held module on destruction.
PiperOrigin-RevId: 256196193
about the buffer size. This is needed to resolve the operand
correctly. Add that information to view op
serialization/deserialization
Also modify the parsing of buffer type by splitting at 'x' to
side-step issues with StringRef number parsing.
PiperOrigin-RevId: 256188319
This saves us the excessive string conversions and comparisons in
verification and transformation and scopes them only to parsing
and printing, which are meant for I/O so string conversions should
be fine.
In order to do this, changed the custom assembly format of
spv.module regarding addressing model and memory model.
PiperOrigin-RevId: 256149856
The GPU Launch operation may take constants as arguments, in particular
affine-to-GPU mapping pass automatically forwards potentially constant lower
bounds of loops into the kernel. Define a canonicalization pattern for
LaunchOp that recreates the constants inside the kernel region instead of
accepting them as kernel arguments. This is currently restricted to standard
constants but may be extended to other constant operations.
Also fix an off-by-one indexing bug in OperandStorage::eraseOperand.
PiperOrigin-RevId: 256035437
Move the data members out of Function and into a new impl storage class 'FunctionStorage'. This allows for Function to become value typed, which will greatly simplify the transition of Function to FuncOp(given that FuncOp is also value typed).
PiperOrigin-RevId: 255983022
Type conversion does not necessarily affect all types, some of them may remain
untouched. The type conversion tool from the dialect conversion framework will
unconditionally insert a temporary cast operation from the type to itself
anyway, and will try to materialize it to a real conversion operation if there
are remaining uses. Simply use the original value instead.
PiperOrigin-RevId: 255975450
The RUN line was missing a call to FileCheck making the test always pass. Add
the call to FileCheck and temporarily disable one of the tests that does not
produce the expected result.
PiperOrigin-RevId: 255974805
In ODS, right now we use StringAttrs to emulate enum attributes. It is
suboptimal if the op actually can and wants to store the enum as a
single integer value; we are paying extra cost on storing and comparing
the attribute value.
This CL introduces a new enum attribute subclass that are backed by
IntegerAttr. The downside with IntegerAttr-backed enum attributes is
that the assembly form now uses integer values, which is less obvious
than the StringAttr-backed ones. However, that can be remedied by
defining custom assembly form with the help of the conversion utility
functions generated via EnumsGen.
Choices are given to the dialect writers to decide which one to use for
their enum attributes.
PiperOrigin-RevId: 255935542
This functionality is now moved to a new class, ModuleManager. This class allows for inserting functions into a module, and will auto-rename them on insert to ensure a unique name. This now means that users adding new functions to a module must ensure that the function name is unique, as the Module will no longer do it automatically. This also means that Module::getNamedFunction now operates in O(N) instead of the O(c) time it did before. This simplifies the move of Modules to Operations as the ModuleOp will not be able to have this functionality.
PiperOrigin-RevId: 255846088
This CL also fixes a parsing issue in the BufferType, adds LLVM lowering support for handling the static constant buffer size and a roundtrip test.
PiperOrigin-RevId: 255834356
These ops are analogues of the current standard ops dma_start/wait, with the exception that the memref operands are affine expressions of loop IVs and symbols (analogous to affine.load/store).
The addition of these operations will enable changes to affine transformation and analysis passes which operate on memory dereferencing operations.
PiperOrigin-RevId: 255658382
During conversion, if a type conversion has dangling uses a type conversion must persist after conversion has finished to maintain valid IR. In these cases, we now query the TypeConverter to materialize a conversion for us. This allows for the default case of a full conversion to continue working as expected, but also handle the degenerate cases more robustly.
PiperOrigin-RevId: 255637171
constant then it is represented as <size x elementType>. If the size
is not a compile time constant, then it is represented as
<? x elementType>.
PiperOrigin-RevId: 255619400
Remove the ability to print an attribute without a type, but allow for attributes to elide the type under certain circumstances. This fixes a bug where attributes within ArrayAttr, and other collection attributes, would never print the type.
PiperOrigin-RevId: 255306974
annotations.
Getters are required as there are currently no global constants in MLIR and this
is an easy way to unblock CUDA execution while waiting for those.
PiperOrigin-RevId: 255169002
The actual transformation from PTX source to a CUDA binary is now factored out,
enabling compiling and testing the transformations independently of a CUDA
runtime.
MLIR has still to be built with NVPTX target support for the conversions to be
built and tested.
PiperOrigin-RevId: 255167139
Now that Locations are attributes, they have direct access to the MLIR context. This allows for simplifying error emission by removing unnecessary context lookups.
PiperOrigin-RevId: 255112791
The current syntax separates the name and value with ':', but ':' is already overloaded by several other things(e.g. trailing types). This makes the syntax difficult to parse in some situtations:
Old:
"foo: 10 : i32"
New:
"foo = 10 : i32"
PiperOrigin-RevId: 255097928
This is the standard syntax for types on operations, and is also already used by IntegerAttr and FloatAttr.
Example:
dense<5> : tensor<i32>
dense<[3]> : tensor<1xi32>
PiperOrigin-RevId: 255069157
The OperationFolder currently just inserts into the entry block of a Function, but regions may be isolated above, i.e. explicit capture only, and blindly inserting constants may break the invariants of these regions.
PiperOrigin-RevId: 254987796
GPU dialect operations (launch and launch_func) use `index` type for thread and
block index values inside the kernel, for compatibility with affine loops.
NVVM dialect operations, following the NVVM intrinsics, use `!llvm.i32` type,
which does not necessarily have the same bit width as the lowered `index` type.
Optionally sign-extend (indices are signed) or truncate the result of the NVVM
dialect operation to the bit width of the lowered `index` type before passing
it to other operations. This behavior is consistent with `std.index_cast`. We
cannot use the latter since we are targeting LLVM dialect types directly,
rather than standard integer types.
PiperOrigin-RevId: 254980868
PTX backend in LLVM expects additional module-level metadata
`!nvvm.annotations` that lists functions that can be used as GPU kernels.
Generate this metadata based on the `gpu.kernel` attribute attached to
functions. This attribute is added automatically by the kernel outlining pass
in the GPU dialect lowering flow.
PiperOrigin-RevId: 254957345
The error would look like:
path/filename.mlir:32:23: error: use of value '%28' expects different type than prior uses: ''i32'' vs ''!_tf.control''
PiperOrigin-RevId: 254874859
This CL makes use of view_slice in tiling and fusion.
Using a higher level IR element greatly simplifies the IR produced during tiling and fusion.
Lowering to LLVM is updated to first translate view_slice into a sequence of dim, range and cmpi.
This level will also be useful when lowering to affine.
PiperOrigin-RevId: 254767814
Enable reusing the real mlir-opt main from unit tests and in case where
additional initialization needs to happen before main is invoked (e.g., when
using different command line flag libraries).
PiperOrigin-RevId: 254764575
The new operations affine.load and affine.store will take composed affine maps by construction.
These operations will eventually replace load and store operations currently used in affine regions and operated on by affine transformation and analysis passes.
PiperOrigin-RevId: 254754048
The ModuleOp contains a single region that must contain a single block. This block must be terminated by a new pseudo operation 'module_terminator'. The syntax for this operations is as follows:
`module` (`attributes` attr-dict)? region
Example:
module {
...
}
module attributes { ... } {
...
}
PiperOrigin-RevId: 254513752
This CL adds a conv op that corresponds to the TF description along with its lowering to loops (https://www.tensorflow.org/api_docs/python/tf/nn/convolution).
The dimension of the convolution is inferred from the rank of the views. The other logical
dimensions correspond to the TF description.
The computation of tiled views need to be updated to work for the input tensor. This is left for a future CL.
PiperOrigin-RevId: 254505644
This will allow for locations to be used in the same contexts as attributes. Given that attributes are nullable types, the 'Location' class now represents a non-nullable wrapper around a 'LocationAttr'. This preserves the desired semantics we have for non-optional locations.
PiperOrigin-RevId: 254505278
This CL adds the basic SPIR-V serializer and deserializer for converting
SPIR-V module into the binary format and back. Right now only an empty
module with addressing model and memory model is supported; (de)serialize
other components will be added gradually with subsequent CLs.
The purpose of this library is to enable importing SPIR-V binary modules
to run transformations on them and exporting SPIR-V modules to be consumed
by execution environments. The focus is transformations, which inevitably
means changes to the binary module; so it is not designed to be a general
tool for investigating the SPIR-V binary module and does not guarantee
roundtrip equivalence (at least for now).
PiperOrigin-RevId: 254473019
This CL adds support for O-D ops in Linalg ops by:
1. making the CopyOp maps optional instead of default valued
2. allowing certain map operations to accept and return empty maps
3. making linalg::LowerToLoops aware of these changes
4. providing a proper 0-D impl for CopyOp and FillOp
5. adding the relevant tests
PiperOrigin-RevId: 254381908
https://www.khronos.org/registry/spir-v/specs/1.0/SPIRV.html#OpTypeImage.
Add new enums to describe Image dimensionality, Image Depth, Arrayed
information, Sampling, Sampler User information, and Image format.
Doesn's support the Optional Access qualifier at this stage
Fix Enum generator for tblgen to add "_" at the beginning if the enum
starts with a number.
PiperOrigin-RevId: 254091423
* Support for 1->0 type mappings, i.e. when the argument is being removed.
* Reordering types when converting a type signature.
* Adding new inputs when converting a type signature.
This cl also lays down the initial foundation for supporting 1->N type mappings, but full support will come in a followup.
Moving forward, function signature changes will be driven by populating a SignatureConversion instance. This class contains all of the necessary information for adding/removing/remapping function signatures; e.g. addInputs, addResults, remapInputs, etc.
PiperOrigin-RevId: 254064665
These were likely added in error because of confusion about the flag when it was just called "-verify". The extra flag doesn't cause much harm, but it does make mlir-opt do more work and clutter the RUN line
PiperOrigin-RevId: 254037016
This name has caused some confusion because it suggests that it's running op verification (and that this verification isn't getting run by default).
PiperOrigin-RevId: 254035268
By default MSVC does not export any symbol and does not create a companion
.lib for a .dll. This will cause problems when trying to link against the
library.
PiperOrigin-RevId: 254033454
This is a direct modelling of SPIR-V's OpVariable. The custom assembly format
parsers/prints descriptor in a nicer way if presents. There are other common
decorations that can appear on variables like builtin, which can be supported
later.
This CL additionally deduplicates the parser/printer/verifier declaration
in op definitions by adding defaults to SPV_Op base.
by adding
PiperOrigin-RevId: 253828254
Index types integers of platform-specific bit width. They are used to index
memrefs and as loop induction variables, however they could not be obtained
from an integer until now, making it virtually impossible to express indirect
accesses (given that memrefs of indices are not allowed) or data-dependent
loops. Introduce `std.index_cast` to transform indices into integers and vice
versa. The semantics of this cast is to sign-extend when casting to a wider
integer, and to truncate when casting to a narrower integer. It belongs to
StandardOps because both types it operates on are standard types, and because
its results are likely to be used in std.load and std.store.
Introduce llvm.sext, llvm.zext and llvm.trunc operations to the LLVM dialect.
Provide the conversion of `std.index_cast` to llvm.sext or llvm.trunc,
depending on the actual bitwidth of `index` known during the conversion.
PiperOrigin-RevId: 253624100
This CL adds a generic CopyOp to Linalg and its lowering to loops.
The CopyOp supports input and output permutation maps.
When combined with tiling and allocating a new local buffer, this should provide basic support for implementing simple memory transfers with coalescing.
At the moment, lowering copies to a library call is not supported.
PiperOrigin-RevId: 253250497
This converts entire loops into threads/blocks. No check on the size of the
block or grid, or on the validity of parallelization is performed, it is under
the responsibility of the caller to strip-mine the loops and to perform the
dependence analysis before calling the conversion.
PiperOrigin-RevId: 253189268
`affine.apply` is supposed to operate on values of index types in context of
affine loops. It is possible to programmatically constuct an `affine.apply`
that takes values of other types as operands or returns them, but it would not
be parseable. Disallow such cases in the verifier.
PiperOrigin-RevId: 253021704
This terminator operation should appear at the end of the blocks in the body
region of `gpu.launch` when the control flow needs to be returned from the
kernel. Using `std.return` in this place is ambiguous: it may exit the body
region or the enclosing function. Furthermore, this allows the GPU dialect to
impose the absence of return values as required by the underlying kernel
execution models.
Update outlining transformation from `gpu.launch` to `gpu.launch_func` so that
it replaces `gpu.return` with `std.return`.
PiperOrigin-RevId: 252985992
Pointer types need to specify the storage class. We use the utility functions
generated from SPV_StorageClassAttr to parse and print the storage classes.
Also improved the case that no element type is provided for (runtime) array.
PiperOrigin-RevId: 252935599
This CL adds a generic FillOp to Linalg and its lowering to loops.
This is achieved by avoiding to specify the static NLoopTypes and ViewRanks type traits but instead defines the relevant methods as `extraClassDeclaration`.
The relevant AffineMap and scalar emission code are added, with relevant tests.
This gives us a first rank-agnostic Linalg op with its generic lowering to loops that should compose with view-based tiling and fusion.
PiperOrigin-RevId: 252869205
1) Lowest minimum pattern stack depth when legalizing.
- This leads the system to favor patterns that have lower legalization stacks, i.e. represent a more direct mapping to the target.
2) Pattern benefit.
- When considering multiple patterns with the same legalization depth, this favors patterns with a larger specified benefit.
PiperOrigin-RevId: 252713470
This CL adds a fusion pass for the Linalg dialect.
Fusion is backed by a simple analysis on SSA values and proceeds as follows:
1. A dependence and alias analyses are performed on views.
2. A Linalg op is tiled by a particular tile size. This creates a new Linalg op operating on tiled loops and tiled views.
3. The dependence analysis is used to obtain ops that produce views that are consumed by the original Linalg op.
4. Dependence analysis is used to determine whether op-level fusion would violate any dependence.
5. If fusion is safe, matching tiled views are sliced for the producing op.
6. A tiled clone of the producer op is written before the tiled consumer op.
If a producer is fused, its entire output view has been computed in tiled form.
The original producer op is then erased.
PiperOrigin-RevId: 252695194
This CL adds a lowering to LLVM for MamulOp and a corresponding integration test.
View descriptor manipulation is moved from MLIR's LLVM dialect to C++ code compiled on the side. To this end a separation is introduced between `cblas.cpp` and `cblas_interface.cpp`, the latter operating on view types whose ABI correspond to the LLVM signature generated by MLIR.
An intermediary step is introduced that allocates a new descriptor on the MLIR side for the purpose of passing it to LLVM. The reason for this extra step is that the ABI for by-value ViewType objects wants aligned descriptors, e.g.:
```
extern "C" void linalg_dot_impl(ViewType<float, 1> X, ViewType<float, 1> Y,
BaseViewType<float> Z) {
...
}
```
produces LLVM IR with the signature:
```
%struct.ViewType = type { %struct.BaseViewType, [1 x i64], [1 x i64] }
%struct.BaseViewType = type { float*, i64 }
define void @linalg_dot_impl(%struct.ViewType* byval align 8, %struct.ViewType* byval align 8, float*, i64) tensorflow/mlir#0 {
...
}
```
We don't seem to be able to make such aligned allocations in the MLIR -> LLVM converter atm.
Going through a level of indirection allows the test to pass.
The temporary tradeoff is that the MLIR shims have to be written by hand.
They will disappear in the future.
PiperOrigin-RevId: 252670672
This CL exposes a parseType method which allows standalone reuse of the MLIR type parsing mechanism. This is a free function for now because the underlying MLIR parser is not guaranteed to receive a StringRef which lives in the proper MemBuffer. This requires building a new MemBuffer/SourceMgr and modifying the Parser constructor to not require an mlir::Module.
The error diagnostic emitted by parseType has context limited to the local string.
For now the dialect has the additional option to emit its own extra error that has the FileLineColLoc context.
In the future, both error messages should be combined into a single error.
PiperOrigin-RevId: 252468911
This CL enables verification code generation for variadic operands and results.
In verify(), we use fallback getter methods to access all the dynamic values
belonging to one static variadic operand/result to reuse the value range
calculation there.
PiperOrigin-RevId: 252288219
This CL added getODSOperands() and getODSResults() as fallback getter methods for
getting all the dynamic values corresponding to a static operand/result (which
can be variadic). It should provide a uniform way of calculating the value ranges.
All named getter methods are layered on top of these methods now.
PiperOrigin-RevId: 252284270
This function returns the element type of the underlying type or the input type itself. This removes some of the casting and code from ODS and into C++ code.
I've not converted all the call sites (as this requires a new include and target) and wanted to run it past folks first.
PiperOrigin-RevId: 251978156
Considered adding more placeholders to designate types in the replacement pattern, but convinced for now sticking to simpler approach. This should at least enable specifying constraints across operands/results/attributes and we can start getting rid of the special cases.
PiperOrigin-RevId: 251564893
This CL adds SPV_ModuleEndOp for terminating the only block inside a
SPV_ModuleOp's only region. Verification now enforces a spv.module only
contains func or spv.* ops and no external or nested functions are
present. Because of the structural requirement of a block, spv.Return
is also added in this CL.
PiperOrigin-RevId: 251510706
The initial implementation of SDBM mistakenly swapped the order of variables in
the inequalities induced by a stripe equality: y = x # B actually implies
y - x <= 0 and x - y <= B - 1 rather than x - y <= 0 and y - x <= B - 1 as
implemented. Textual comments in the test files were correct but did not
correspond to the emitted IR. Round-tripping between SDBM and expression lists
was not affected because the wrong order was used in both directions of the
conversion. Use the correct order.
PiperOrigin-RevId: 251252980
When manipulating generic operations, such as in dialect conversion /
rewriting, it is often necessary to view a list of Values as operands to an
operation without creating the operation itself. The absence of such view
makes dialect conversion patterns, among others, to use magic numbers to obtain
specific operands from a list of rewritten values when converting an operation.
Introduce XOpOperandAdaptor classes that wrap an ArrayRef<Value *> and provide
accessor functions identical to those available in XOp. This makes it possible
for conversions to use these adaptors to address the operands with names rather
than rely on their position in the list. The adaptors are generated from ODS
together with the actual operation definitions.
This is another step towards making dialect conversion patterns specific for a
given operation.
Illustrate the approach on conversion patterns in the standard to LLVM dialect
conversion.
PiperOrigin-RevId: 251232899
Andy Davis