This syntax removes boilerplate and verbose list of region arguments in the
header of the entry block. It groups operands into segments related to GPU
blocks, GPU threads as well as the operands that are forwarded to the kernel.
The two former segments are also used to give names to the region arguments
that are used for GPU blocks and threads inside the kernel body region.
--
PiperOrigin-RevId: 246792329
The generic form of operations currently supports optional regions to be
located after the operation type. As we are going to add a type to each
region in a leading position in the region syntax, similarly to functions, it
becomes ambiguous to have regions immediately after the operation type. Put
regions between operands the optional list of successors in the generic
operation syntax and wrap them in parentheses. The effect on the exisitng IR
syntax is minimal since only three operations (`affine.for`, `affine.if` and
`gpu.kernel`) currently use regions.
--
PiperOrigin-RevId: 246787087
Added a definition for the newly added None type.
Fix TCopVTEtAreSameAt to be a Predicate rather than a PredOpTrait. This makes it more of a primitive and allows flexible composition.
--
PiperOrigin-RevId: 246631975
Trying to activate both LLVM and MLIR passes in mlir-cpu-runner showed name collisions when registering pass names.
One possible way of disambiguating that should also work across dialects is to prepend the dialect name to the passes that specifically operate on that dialect.
With this CL, mlir-cpu-runner tests still run when both LLVM and MLIR passes are registered
--
PiperOrigin-RevId: 246539917
This CL builds upon ftynse@'s Linalg dialect conversion (in examples/Linalg/Linalg1) and updates it to support buffers and the fully composed form of view and slice operations.
A new BufferSizeOp is introduced for the purpose of extracting the size information from a buffer.
This will be useful in a followup CL for an end-to-end LLVM execution path where mlir-cpu-runner will allocate a buffer.
--
PiperOrigin-RevId: 246358593
This CL adds a primitive tiling pass for Linalg.
The tiling pass uses the loopToOperandRangesMaps property which should be ideally Tablegen'd and in-class.
The tiling specification uses 0 as a convention to skip loops that should not be tiled.
Tiling proceeds in 3 steps, for each op:
1. Pad tile sizes with 0 to match the number of loops, this simplifies the implementation and avoids affine map manipulations to align dimensions.
2. Create loop ranges that represent the min/max/step by which to iterate. This should be later complemented by a range intersection to avoid the out-of-bounds case.
3. Map the loop ranges to view ranges in order to create subviews on which the op can be called.
Relevant utility and helper functions are added separately that support writing the transformation in a declarative fashion.
Simplifying assumptions are made for now on the views and the ranges that are constructed
in the function and are not passed as function arguments. This restriction will be lifted
in the future.
--
PiperOrigin-RevId: 246124419
Simple mechanism to allow specifying arbitrary function declarations. The modelling will not cover all cases so allow a means for users to declare a method function that they will define in their C++ files. The goal is to allow full C++ flexibility as the goal is to cover cases not modelled.
--
PiperOrigin-RevId: 245889819
This CL implements the previously unsupported parsing for Range, View and Slice operations.
A pass is introduced to lower to the LLVM.
Tests are moved out of C++ land and into mlir/test/Examples.
This allows better fitting within standard developer workflows.
--
PiperOrigin-RevId: 245796600
Enables specifying the documentation for dialect along with defining the ops of the dialect. The doc generator will be expanded in follow up to emit the documentation in the autogenerated files. This is precursor to allowing common base for all ops in a dialect.
All the dialect documentation is super sparse and just added as placeholder.
I was tempted (and started) to move ConstantOp to be generated too, but this will be easier post adding extra_methods, so deferring until then.
--
PiperOrigin-RevId: 245759984
Define a new dialect related to GPU kernels. Currently, it only contains a
single operation for launching a kernel on a three-dimensional grid of thread
blocks, following a model similar to that of CUDA. In particular, the body of
the kernel contains operations executed by each thread and uses region
arguments to accept thread and block identifiers (similar to how the loop body
region accepts the induction value).
--
PiperOrigin-RevId: 245713728
Instead, fold such operations. This way callers don't need to conditionally create cast operations depending on if a value already has the target type.
Also, introduce areCastCompatible to allow cast users to verify that the generated op will be valid before creating the operation.
TESTED with unit tests
--
PiperOrigin-RevId: 245606133
none-type ::= `none`
The `none` type is a unit type, i.e. a type with exactly one possible value, where its value does not have a defined dynamic representation.
--
PiperOrigin-RevId: 245599248
Currently, this is limited to operations that give access to the special registers of
NVIDIA gpus that represent block and thread indices.
--
PiperOrigin-RevId: 245378632
Certain ops can have multiple variadic operands/results, e.g., `tf.DynamicStitch`.
Even if an op has only one variadic operand/result, it is not necessarily the
very last one, e.g., `tf.RaggedGather`. This CL enhances TableGen subsystem to be
able to represent such cases.
In order to deduce the operand/result value range for each variadic operand,
currently we only support variadic operands/results all of the same size.
So two new traits, `SameVariadicOperandSize` and `SameVariadicResultSize` are
introduced.
--
PiperOrigin-RevId: 245310628
A unit attribute is an attribute that represents a value of `unit` type. The
`unit` type allows only one value forming a singleton set. This attribute value
is used to represent attributes that only have meaning from their existence.
One example of such an attribute could be the `swift.self` attribute. This attribute indicates that a function parameter is the self/context
parameter. It could be represented as a boolean attribute(true or false), but a
value of false doesn't really bring any value. The parameter either is the
self/context or it isn't.
```mlir {.mlir}
// A unit attribute defined with the `unit` value specifier.
func @verbose_form(i1 {unitAttr : unit})
// A unit attribute can also be defined without the `unit` value specifier.
func @simple_form(i1 {unitAttr})
```
--
PiperOrigin-RevId: 245254045
This CL adds linalg.dot, linalg.matvec and linalg.matmul ops with the proper roundtripping test. These are the first LinalgOp that operate on views and that will lower to library calls.
Linalg ops exhibit some common properties and behavior that are modeled with Traits.
A LinalgOp is defined as a generic Op that operates on input and output views (passed as operands) and has the following properties:
1. a number of input and outputs captured by the `NInputsAndOutputs` trait.
2. a list of ranks for each operand captured by the `ViewRanks` trait.
3. a set of parallel, reduction and windowing loops captured by `NLoopTypes` trait.
These represent are a first set of generic properties that will enable the definition of generic linear algebra operations and the properties necessary for upcoming transformations.
--
PiperOrigin-RevId: 244912754
Note that I broke this out as a separate pass because intermediate transformations often produce qcast/dcast ops that are integral to the transformation, and it is typical to want to lower any remaining, unmatched casts at the end of quantization. If this flexibility ends up not being needed, they can be collapsed into the same pass. This is included in the same cpp file because all of the math ops will need to defer to emitting quantize/dequantize logic for cases that they cannot be fully lowered to fixed-point math.
Also, the new convertistof op needs to be evaluated for inclusion in StandardOps.
--
PiperOrigin-RevId: 244768679
During the pattern rewrite, if the function is changed, i.e. ops created,
deleted or swapped, the pattern rewriter needs to re-scan the function entirely
and apply the patterns again, so the patterns whose root ops have been popped
out from the working list nor an immediate users of the changed ops can be
reconsidered.
A command line flag is added to set the max number of iterations rescanning the
function for pattern match. If the rewrite doesn' converge after this number,
this compiling will continue and the result can be sub-optimal.
One unit test is updated because this change fixed the missing optimization opportunities.
--
PiperOrigin-RevId: 244754190
Both cOp and tAttr were used to perform some native C++ code expression.
Unifying them simplifies the concepts and reduces cognitive burden.
--
PiperOrigin-RevId: 244731946
This allows accessing those bound source ops in result patterns, which can be
useful for invoking native C++ op creation.
We bind the op entirely here because ops can have multiple results. Design a
approach to bind to a specific result is not the concern of this commit.
--
PiperOrigin-RevId: 244724750
The per-layer format is now like:
!quant.uniform<i8<-8:7>:f32, 9.987200e-01:127>
and per-axis is:
!quant.uniform<i8:f32:1, {2.0e+2,0.99872:120}>
I used the following sed script to update the unit tests (invoked with commands like `sed -i -r -f fix_quant.sed $(find . -name '*.mlir')`).
---
# Per-layer
s|\!quant<"uniform\[([iu][0-9]+):([fb]+[0-9]+)\]\{([^\}]+)\}\s*">|!quant.uniform<\1:\2, \3>|g
s|\!quant<"uniform\[([iu][0-9]+)\(([^\)]+)\):([fb]+[0-9]+)\]\{([^\}]+)\}\s*">|!quant.uniform<\1<\2>:\3, \4>|g
# Per-axis
s|\!quant<"uniform\[([iu][0-9]+):([fb]+[0-9]+)(:[0-9]+)?\]\{([^\}]+)\}\s*">|!quant.uniform<\1:\2\3, {\4}>|g
s|\!quant<"uniform\[([iu][0-9]+)\(([^\)]+)\):([fb]+[0-9]+)(:[0-9]+)?\]\{([^\}]+)\}\s*">|!quant.uniform<\1<\2>:\3\4, {\5}>|g
---
I fixed up the one file of error cases manually.
Since this is a one time syntax fix, I am not persisting the script anywhere.
--
PiperOrigin-RevId: 244425331
This CL adds a linalg.slice op with the proper roundtripping test.
A slice op allows taking subviews that may be rank-reducing (if some indexing is of index type) or not (if all indexings are of linalg.range type).
A slice must be constructed directly from a base view (no chains of slices may exist in the IR). Helper functions that fold will be provided for construction if/when necessary.
This also renames base_view to view.
--
PiperOrigin-RevId: 244406827
This CL adds a linalg.view<?x?xf32> type and base_view op with the proper roundtripping test. The parser will be improved in a subsequent CL once portions of the mlir::Parser are exposed.
For now this only supports dynamic views, static views will be introduced at a later time when they are needed.
--
PiperOrigin-RevId: 244374180
This also does the following:
- Removes the poc POT add implementation in favor of a version that does not rescale.
- Adds a handful of FxpMathOps which are needed (these are for comment and we may want to move them to the StandardOps dialect).
- Adds a canonicalizer to the StorageCastOp, which removes some cruft once conversions have been done.
- Adds a couple of predicates to OpBase.
--
PiperOrigin-RevId: 244287706
This CL starts implementing a Linalg dialect with the objective of supporting
optimizing compilation of loops and library calls for a subset of common linear
algebra operations.
This CL starts by simply adding a linalg.range type and an operation with the
proper roundtripping test.
--
PiperOrigin-RevId: 244189468
For ops with the SameValueType trait, we generate a builder without requiring
result type; we get the result type from the operand. However, if the operand
is variadic, we need to index into the first value in the pack.
--
PiperOrigin-RevId: 243866647
other characters within the <>'s now that we can. This will allow quantized
types to use the pretty syntax (among others) after a few changes.
--
PiperOrigin-RevId: 243521268
There are no empty lines in output for three of these directives so removed
them and replaced the remaining one with 'CHECK-NOT:' as otherwise it is
failing with the following error.
error: found 'CHECK-EMPTY' without previous 'CHECK: line
TESTED = n/a
PiperOrigin-RevId: 243288605
This CL changes various predicates and rewrite rules to use $-placeholders and
`tgfmt` as the driver for substitution. This will make the predicates and rewrite
rules more consistent regarding their arguments and more readable.
--
PiperOrigin-RevId: 243250739
This adds parsing, printing and some folding/canonicalization.
Also extends rewriting of subi %0, %0 to handle vectors and tensors.
--
PiperOrigin-RevId: 242448164
This is only teaching the LLVM converter to propagate the attribute onto
the function type. MLIR will not recognize this arguments, so it would only
be useful when calling for example `printf` with the same arguments across
a module. Since varargs is part of the ABI lowering, this is not NFC.
--
PiperOrigin-RevId: 242382427
making the IR dumps much nicer.
This is part 2/3 of the path to making dialect types more nice. Part 3/3 will
slightly generalize the set of characters allowed in pretty types and make it
more principled.
--
PiperOrigin-RevId: 242249955
restricted grammar. This will make certain common types much easier to read.
This is part tensorflow/mlir#1 of 2, which allows us to accept the new syntax. Part 2 will
change the asmprinter to automatically use it when appropriate, which will
require updating a bunch of tests.
This is motivated by the EuroLLVM tutorial and cleaning up the LLVM dialect aesthetics a bit more.
--
PiperOrigin-RevId: 242234821
To support automatically constraint composition of ArrayAttr, a new
predicate combiner, Concat, is introduced. It prepends a prefix and
appends a postfix to a child predicate's final predicate string.
--
PiperOrigin-RevId: 242121186
Note: This now means that we cannot fold chains of operations, i.e. where constant foldable operations feed into each other. Given that this is a testing pass solely for constant folding, this isn't really something that we want anyways. Constant fold tests should be simple and direct, with more advanced folding/feeding being tested with the canonicalizer.
--
PiperOrigin-RevId: 242011744
There are two places containing constant folding logic right now: the ConstantFold
pass and the GreedyPatternRewriteDriver. The logic was not shared and started to
drift apart. We were testing constant folding logic using the ConstantFold pass,
but lagged behind the GreedyPatternRewriteDriver, where we really want the constant
folding to happen.
This CL pulled the logic into utility functions and classes for sharing between
these two places. A new ConstantFoldHelper class is created to help constant fold
and de-duplication.
Also, renamed the ConstantFold pass to TestConstantFold to make it clear that it is
intended for testing purpose.
--
PiperOrigin-RevId: 241971681
Previously, attribute constraints are basically unused: we set true for almost
anything. This CL refactors common attribute kinds and sets constraints on
them properly. And fixed verification failures found by this change.
A noticeable one is that certain TF ops' attributes are required to be 64-bit
integer, but the corresponding TFLite ops expect 32-bit integer attributes.
Added bitwidth converters to handle this difference.
--
PiperOrigin-RevId: 241944008
We can bind symbols to op arguments/results in source pattern and op results in
result pattern. Previously resolving these symbols is scattered across
RewriterGen.cpp. This CL aggregated them into a `PatternSymbolResolver` class.
While we are here, this CL also cleans up tests for patterns to make them more
focused. Specifically, one-op-one-result.td is superseded by pattern.td;
pattern-tAttr.td is simplified; pattern-bound-symbol.td is added for the change
in this CL.
--
PiperOrigin-RevId: 241913973
Previously we bundle the existence check and the MLIR attribute kind check
in one call. Further constraints (like element bitwidth) have to be split
into following checks. That is not a nice separation given that we have more
checks for constraints. Instead, this CL changes to generate a local variable
for every attribute, check its existence first, then check the constraints.
Creating a local variable for each attribute also avoids querying it multiple
times using the raw getAttr() API. This is a win for both performance the
readability of the generated code.
This CL also changed the error message to be more greppable by delimiting
the error message from constraints with boilerplate part with colon.
--
PiperOrigin-RevId: 241906132
Mainly a missing dependency caused the tests to pass if one already built
the repo, but not from a clean (or incremental) build.
--
PiperOrigin-RevId: 241852313
- Retains Quantization types and predicates.
- Retains utilities and example (testable) passes/ops.
- Retains unit tests for example passes/ops.
- Moves fixed point ops (and corresponding real ops) to FxpMathOps.
- Moves real -> fixed point pass to FxpMathOps.
- Sever the dependency on the TF dialect from Quantization. These dialects should now be open-sourcable.
--
PiperOrigin-RevId: 241825598
This CL looses the requirement that all result patterns in a rewrite rule must
replace a result of the root op in the source pattern. Now only the last N
result pattern-generated ops are used to replace a N-result source op.
This allows to generate additional ops to aid building up final ops used to
replace the source op.
--
PiperOrigin-RevId: 241783192
Includes a draft of documentation for the quantization setup.
Given how many comments such docs have garnered in the past, I've biased towards a lightly edited first-draft so that people can argue about terminology, approach and structure without having spent too much time on it.
Note that the sections under "Uniform quantization" were cribbed nearly verbatim from internal documentation that Daniel wrote.
PiperOrigin-RevId: 241768668
OptionalAttr is just wrapping around the actual attribute; so it should just use
the actual attribute's `convertFromStorage` to read the value and wrap it around
with `Optional<>` to return. Previously it was mandating how the actual attribute
reads the value with `{0}.getValue()`.
--
PiperOrigin-RevId: 241762355
Attributes can have default values or be optional. Checking the validity of
attributes in aggregate builder should consider that. And to be accurate,
we should check all required attributes are indeed provided in the list.
This is actually duplicating the work done by verifier. Checking the validity
of attributes should be the responsiblity of verifiers. This CL removes
the assertion for attributes in aggregate builders for the above reason.
(Assertions for operands/results are still kept since they are trivial.)
Also added more tests for aggregate builders.
--
PiperOrigin-RevId: 241746059
This version has been deprecated and can now be removed completely since the
last remaining user (Python bindings) migrated to declarative builders.
Several functions in lib/EDSC/Types.cpp construct core IR objects for the C
bindings. Move these functions into lib/EDSC/CoreAPIs.cpp until we decide
where they should live.
This completes the migration from the delayed-construction EDSC to Declarative
Builders.
--
PiperOrigin-RevId: 241716729
This CL fixes the non-determinism across compilers in an edsc::select expression used in LowerVectorTransfers. This is achieved by factoring the expression out of the function call to ensure a deterministic order of evaluation.
Since the expression is now factored out, fewer IR is generated and the test is updated accordingly.
--
PiperOrigin-RevId: 241679962
Historically, the LLVM IR dialect has been using the generic form of MLIR
operation syntax. It is verbose and often redundant. Introduce the custom
printing and parsing for all existing operations in the LLVM IR dialect.
Update the relevant documentation and tests.
--
PiperOrigin-RevId: 241617393
This CL introduces Confined as a general mechanism to compose complex attribute
constraints out of more primitive ones. It's particularly useful for automatically
generating op definitions from some external source, where we can have random
combinations of primitive constraints and it would be impractical to define a case
for each of such combination.
Two primitive attribute constraints, IntMinValue and ArrayMinCount, are added to be
used together with Confined.
--
PiperOrigin-RevId: 241435955
This CL adds EnumAttr as a general mechanism for modelling enum attributes. Right now
it is using StringAttr under the hood since MLIR does not have native support for enum
attributes.
--
PiperOrigin-RevId: 241334043
Example:
func @unknown_std_op() {
%0 = "std.foo_bar_op"() : () -> index
return
}
Will result in:
error: unregistered operation 'std.foo_bar_op' found in dialect ('std') that does not allow unknown operations
--
PiperOrigin-RevId: 241266009
have no standard ops for working with these yet, this is simply enough to
represent and round trip them in the printer and parser.
--
PiperOrigin-RevId: 241102728
A integer number can be specified in the pattern definition and used as the
adjustment to the default benefit score in the generated rewrite pattern C++
definition.
PiperOrigin-RevId: 240994192
This CL removes the reliance of the vectorize pass on the specification of a `fastestVaryingDim` parameter. This parameter is a restriction meant to more easily target a particular loop/memref combination for vectorization and is mainly used for testing.
This also had the side-effect of restricting vectorization patterns to only the ones in which all memrefs were contiguous along the same loop dimension. This simple restriction prevented matmul to vectorize in 2-D.
this CL removes the restriction and adds the matmul test which vectorizes in 2-D along the parallel loops. Support for reduction loops is left for future work.
PiperOrigin-RevId: 240993827
Most of the tests have been ported to be unit-tests and this pass is problematic in the way it depends on TableGen-generated files. This pass is also non-deterministic during multi-threading and a blocker to turning it on by default.
PiperOrigin-RevId: 240889154
Example:
%call:2 = call @multi_return() : () -> (f32, i32)
use(%calltensorflow/mlir#0, %calltensorflow/mlir#1)
This cl also adds parser support for uniquely named result values. This means that a test writer can now write something like:
%foo, %bar = call @multi_return() : () -> (f32, i32)
use(%foo, %bar)
Note: The printer will still print the collapsed form.
PiperOrigin-RevId: 240860058
This CL allows vectorization to be called and configured in other ways than just via command line arguments.
This allows triggering vectorization programmatically.
PiperOrigin-RevId: 240638208
The `Builder*` parameter is unused in both generated build() methods so that we can
leave it unnamed. Changed stand-alone parameter build() to take `_tblgen_state` instead
of `result` to allow `result` to avoid having name collisions with op operand,
attribute, or result.
PiperOrigin-RevId: 240637700
When multi-threading is enabled in the pass manager the meaning of the display
slightly changes. First, a new timing column is added, `User Time`, that
displays the total time spent across all threads. Secondly, the `Wall Time`
column displays the longest individual time spent amongst all of the threads.
This means that the `Wall Time` column will continue to give an indicator on the
perceived time, or clock time, whereas the `User Time` will display the total
cpu time.
Example:
$ mlir-opt foo.mlir -experimental-mt-pm -cse -canonicalize -convert-to-llvmir -pass-timing
===-------------------------------------------------------------------------===
... Pass execution timing report ...
===-------------------------------------------------------------------------===
Total Execution Time: 0.0078 seconds
---User Time--- ---Wall Time--- --- Name ---
0.0175 ( 88.3%) 0.0055 ( 70.4%) Function Pipeline
0.0018 ( 9.3%) 0.0006 ( 8.1%) CSE
0.0013 ( 6.3%) 0.0004 ( 5.8%) (A) DominanceInfo
0.0017 ( 8.7%) 0.0006 ( 7.1%) FunctionVerifier
0.0128 ( 64.6%) 0.0039 ( 50.5%) Canonicalizer
0.0011 ( 5.7%) 0.0004 ( 4.7%) FunctionVerifier
0.0004 ( 2.1%) 0.0004 ( 5.2%) ModuleVerifier
0.0010 ( 5.3%) 0.0010 ( 13.4%) LLVMLowering
0.0009 ( 4.3%) 0.0009 ( 11.0%) ModuleVerifier
0.0198 (100.0%) 0.0078 (100.0%) Total
PiperOrigin-RevId: 240636269
The spec allows zero-dimensional memrefs to exist and treats them essentially
as single-element buffers. Unlike single-dimensional memrefs of static shape
<1xTy>, zero-dimensional memrefs do not require indices to access the only
element they store. Add support of zero-dimensional memrefs to the LLVM IR
conversion. In particular, such memrefs are converted into bare pointers, and
accesses to them are converted to bare loads and stores, without the overhead
of `getelementptr %buffer, 0`.
PiperOrigin-RevId: 240579456
Due to legacy reasons (ML/CFG function separation), regions in affine control
flow operations require contained blocks not to have terminators. This is
inconsistent with the notion of the block and may complicate code motion
between regions of affine control operations and other regions.
Introduce `affine.terminator`, a special terminator operation that must be used
to terminate blocks inside affine operations and transfers the control back to
he region enclosing the affine operation. For brevity and readability reasons,
allow `affine.for` and `affine.if` to omit the `affine.terminator` in their
regions when using custom printing and parsing format. The custom parser
injects the `affine.terminator` if it is missing so as to always have it
present in constructed operations.
Update transformations to account for the presence of terminator. In
particular, most code motion transformation between loops should leave the
terminator in place, and code motion between loops and non-affine blocks should
drop the terminator.
PiperOrigin-RevId: 240536998
Before this CL, the result type of the pattern match results need to be as same
as the first operand type, operand broadcast type or a generic tensor type.
This CL adds a new trait to set the result type by attribute. For example, the
TFL_ConstOp can use this to set the output type to its value attribute.
PiperOrigin-RevId: 240441249
Currently, regions can only be constructed by passing in a `Function` or an
`Instruction` pointer referencing the parent object, unlike `Function`s or
`Instruction`s themselves that can be created without a parent. It leads to a
rather complex flow in operation construction where one has to create the
operation first before being able to work with its regions. It may be
necessary to work with the regions before the operation is created. In
particular, in `build` and `parse` functions that are executed _before_ the
operation is created in cases where boilerplate region manipulation is required
(for example, inserting the hypothetical default terminator in affine regions).
Allow creating standalone regions. Such regions are meant to own a list of
blocks and transfer them to other regions on demand.
Each instruction stores a fixed number of regions as trailing objects and has
ownership of them. This decreases the size of the Instruction object for the
common case of instructions without regions. Keep this behavior intact. To
allow some flexibility in construction, make OperationState store an owning
vector of regions. When the Builder creates an Instruction from
OperationState, the bodies of the regions are transferred into the
instruction-owned regions to minimize copying. Thus, it becomes possible to
fill standalone regions with blocks and move them to an operation when it is
constructed, or move blocks from a region to an operation region, e.g., for
inlining.
PiperOrigin-RevId: 240368183
inherited constructors, which is cleaner and means you can now use DimOp()
to get a null op, instead of having to use Instruction::getNull<DimOp>().
This removes another 200 lines of code.
PiperOrigin-RevId: 240068113
This should probably be changed to instead use the negated form (e.g., get predicate + negate it + get resulting template), but this fixes it locally.
PiperOrigin-RevId: 240067116
We just need a way to unpack ArrayRef<ValueHandle> to ArrayRef<Value*>.
No need to expose this to the user.
This reduces the cognitive overhead for the tutorial.
PiperOrigin-RevId: 240037425
tblgen be non-const. This requires introducing some const_cast's at the
moment, but those (and lots more stuff) will disappear in subsequent patches.
This significantly simplifies those patches because the various tblgen op emitters
get adjusted.
PiperOrigin-RevId: 239954566
Enable users specifying operand type constraint combinations (e.g., considering multiple operands). Some of these will be refactored (particularly the OpBase change and that should also not be needed to be done by most users), but the focus is more on user side (shown in test). The generated code for this does not take any known facts into account or perform any simplification.
Start with 2 primities to specify 1) whether an operand has a specific element type, and 2) whether an operand's element type matches another operands element type.
PiperOrigin-RevId: 239875712
This CL revisits the composition of AffineApplyOp for the special case where a symbol
itself comes from an AffineApplyOp.
This is achieved by rewriting such symbols into dims to allow composition to occur mathematically.
The implementation is also refactored to improve readability.
Rationale for locally rewriting symbols as dims:
================================================
The mathematical composition of AffineMap must always concatenate symbols
because it does not have enough information to do otherwise. For example,
composing `(d0)[s0] -> (d0 + s0)` with itself must produce
`(d0)[s0, s1] -> (d0 + s0 + s1)`.
The result is only equivalent to `(d0)[s0] -> (d0 + 2 * s0)` when
applied to the same mlir::Value* for both s0 and s1.
As a consequence mathematical composition of AffineMap always concatenates
symbols.
When AffineMaps are used in AffineApplyOp however, they may specify
composition via symbols, which is ambiguous mathematically. This corner case
is handled by locally rewriting such symbols that come from AffineApplyOp
into dims and composing through dims.
PiperOrigin-RevId: 239791597
Previously we emit both op declaration and definition into one file and include it
in *Ops.h. That pulls in lots of implementation details in the header file and we
cannot hide symbols local to implementation. This CL splits them to provide a cleaner
interface.
The way how we define custom builders in TableGen is changed accordingly because now
we need to distinguish signatures and implementation logic. Some custom builders with
complicated logic now can be moved to be implemented in .cpp entirely.
PiperOrigin-RevId: 239509594
This CL fixes an issue where cloned loop induction variables were not properly
propagated and beefs up the corresponding test.
PiperOrigin-RevId: 239422961
This CL introduces a ValueArrayHandle helper to manage the implicit conversion
of ArrayRef<ValueHandle> -> ArrayRef<Value*> by converting first to ValueArrayHandle.
Without this, boilerplate operations that take ArrayRef<Value*> cannot be removed easily.
This all seems to boil down to decoupling Value from Type.
Alternative solutions exist (e.g. MLIR using Value by value everywhere) but they would be very intrusive. This seems to be the lowest impedance change.
Intrinsics are also lowercased by popular demand.
PiperOrigin-RevId: 238974125
This CL removes the dependency of LowerVectorTransfers on the AST version of EDSCs which will be retired.
This exhibited a pretty fundamental staging difference in AST-based vs declarative based emission.
Since the delayed creation with an AST was staged, the loop order came into existence after the clipping expressions were computed.
This now changes as the loops first need to be created declaratively in fixed order and then the clipping expressions are created.
Also, due to lack of staging, coalescing cannot be done on the fly anymore and
needs to be done either as a pre-pass (current implementation) or as a local transformation on the generated IR (future work).
Tests are updated accordingly.
PiperOrigin-RevId: 238971631
* print-ir-before=(comma-separated-pass-list)
- Print the IR before each of the passes provided within the pass list.
* print-ir-before-all
- Print the IR before every pass in the pipeline.
* print-ir-after=(comma-separated-pass-list)
- Print the IR after each of the passes provided within the pass list.
* print-ir-after-all
- Print the IR after every pass in the pipeline.
* print-ir-module-scope
- Always print the Module IR, even for non module passes.
PiperOrigin-RevId: 238523649
- emit a note on the loop being parallel instead of setting a loop attribute
- rename the pass -test-detect-parallel (from -detect-parallel)
PiperOrigin-RevId: 238122847
Add support to create a new attribute from multiple attributes. It extended the
DagNode class to represent attribute creation dag. It also changed the
RewriterGen::emitOpCreate method to support this nested dag emit.
An unit test is added.
PiperOrigin-RevId: 238090229
- fix for getConstantBoundOnDimSize: floordiv -> ceildiv for extent
- make getConstantBoundOnDimSize also return the identifier upper bound
- fix unionBoundingBox to correctly use the divisor and upper bound identified by
getConstantBoundOnDimSize
- deal with loop step correctly in addAffineForOpDomain (covers most cases now)
- fully compose bound map / operands and simplify/canonicalize before adding
dim/symbol to FlatAffineConstraints; fixes false positives in -memref-bound-check; add
test case there
- expose mlir::isTopLevelSymbol from AffineOps
PiperOrigin-RevId: 238050395
multi-result upper bounds, complete TODOs, fix/improve test cases.
- complete TODOs for loop unroll/unroll-and-jam. Something as simple as
"for %i = 0 to %N" wasn't being unrolled earlier (unless it had been written
as "for %i = ()[s0] -> (0)()[%N] to %N"; addressed now.
- update/replace getTripCountExpr with buildTripCountMapAndOperands; makes it
more powerful as it composes inputs into it
- getCleanupLowerBound and getUnrolledLoopUpperBound actually needed the same
code; refactor and remove one.
- reorganize test cases, write previous ones better; most of these changes are
"label replacements".
- fix wrongly labeled test cases in unroll-jam.mlir
PiperOrigin-RevId: 238014653
This CL makes some minor changes to the declarative builder Helpers:
1. adds lb, ub, step methods to MemRefView to avoid always having to go through std::get + range;
2. drops MemRefView& from IndexedValue which was just creating ownership concerns. Instead, an IndexedValue only needs to keep track of the ValueHandle from which a MemRefView can be constructed on-demand if necessary.
PiperOrigin-RevId: 237861493
TensorFlow comparison ops like tf.Less supports broadcast behavior but the result
type have different element types as the input types. Extend broadcastable trait
to allow such cases. Added tf.Less to demonstrate it.
PiperOrigin-RevId: 237846127
So that we can use this function to deduce broadcasted shapes elsewhere.
Also added support for unknown dimensions, by following TensorFlow behavior.
PiperOrigin-RevId: 237846065
Below shows the output for an example mlir-opt command line.
mlir-opt foo.mlir -verify-each=false -cse -canonicalize -cse -cse -pass-timing
list view (-pass-timing-display=list):
* In this mode the results are displayed in a list sorted by total time; with each pass/analysis instance aggregated into one unique result. This mode is similar to the output of 'time-passes' in llvm-opt.
===-------------------------------------------------------------------------===
... Pass execution timing report ...
===-------------------------------------------------------------------------===
Total Execution Time: 0.0097 seconds (0.0096 wall clock)
---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
0.0051 ( 58.3%) 0.0001 ( 12.2%) 0.0052 ( 53.8%) 0.0052 ( 53.8%) Canonicalizer
0.0025 ( 29.1%) 0.0005 ( 58.2%) 0.0031 ( 31.9%) 0.0031 ( 32.0%) CSE
0.0011 ( 12.6%) 0.0003 ( 29.7%) 0.0014 ( 14.3%) 0.0014 ( 14.2%) DominanceInfo
0.0087 (100.0%) 0.0009 (100.0%) 0.0097 (100.0%) 0.0096 (100.0%) Total
pipeline view (-pass-timing-display=pipeline):
* In this mode the results are displayed in a nested pipeline view that mirrors the internal pass pipeline that is being executed in the pass manager. This view is useful for understanding specifically which parts of the pipeline are taking the most time, and can also be used to identify when analyses are being invalidated and recomputed.
===-------------------------------------------------------------------------===
... Pass execution timing report ...
===-------------------------------------------------------------------------===
Total Execution Time: 0.0082 seconds (0.0081 wall clock)
---User Time--- --System Time-- --User+System-- ---Wall Time--- --- Name ---
0.0042 (100.0%) 0.0039 (100.0%) 0.0082 (100.0%) 0.0081 (100.0%) Function Pipeline
0.0005 ( 11.6%) 0.0008 ( 21.1%) 0.0013 ( 16.1%) 0.0013 ( 16.2%) CSE
0.0002 ( 5.0%) 0.0004 ( 9.3%) 0.0006 ( 7.0%) 0.0006 ( 7.0%) (A) DominanceInfo
0.0026 ( 61.8%) 0.0018 ( 45.6%) 0.0044 ( 54.0%) 0.0044 ( 54.1%) Canonicalizer
0.0005 ( 11.7%) 0.0005 ( 13.0%) 0.0010 ( 12.3%) 0.0010 ( 12.4%) CSE
0.0003 ( 6.1%) 0.0003 ( 8.3%) 0.0006 ( 7.2%) 0.0006 ( 7.1%) (A) DominanceInfo
0.0002 ( 3.8%) 0.0001 ( 2.8%) 0.0003 ( 3.3%) 0.0003 ( 3.3%) CSE
0.0042 (100.0%) 0.0039 (100.0%) 0.0082 (100.0%) 0.0081 (100.0%) Total
PiperOrigin-RevId: 237825367
Declarative builders want to provide the same nesting interface for blocks and loops. MLIR on the other hand has different behaviors:
1. when an AffineForOp is created the insertion point does not enter the loop body;
2. when a Block is created, the insertion point does enter the block body.
Guard against the second behavior in EDSC to make the interface unsurprising.
This also surfaces two places in the eager branch API where I was guarding against this behavior indirectly by creating a new ScopedContext.
Instead, uniformize everything to properly reset the insertion point in the unique place that builds the mlir::Block*.
PiperOrigin-RevId: 237619513
Alex Zinenko