This CL also updates to use containing region as a fallback way to find
context since functions will eventually become ops with regions.
PiperOrigin-RevId: 253627322
Arguably, this function is only useful for transformations and should not
pollute the main IR. Also make sure it accepts a the resulting container
by-reference instead of returning it.
PiperOrigin-RevId: 253622981
This CL adds a generic CopyOp to Linalg and its lowering to loops.
The CopyOp supports input and output permutation maps.
When combined with tiling and allocating a new local buffer, this should provide basic support for implementing simple memory transfers with coalescing.
At the moment, lowering copies to a library call is not supported.
PiperOrigin-RevId: 253250497
This converts entire loops into threads/blocks. No check on the size of the
block or grid, or on the validity of parallelization is performed, it is under
the responsibility of the caller to strip-mine the loops and to perform the
dependence analysis before calling the conversion.
PiperOrigin-RevId: 253189268
llvm::maskTrailingOnes<char> runs into a static assertion on the type not being
unsigned. Use `unsigned char` instead of `char`.
PiperOrigin-RevId: 252827214
This introduces the support for region-containing operations to the dialect
conversion framework in order to support the conversion of affine control-flow
operations into the standard control flow with branches. Regions that belong
to an operation are converted before the operation itself. The
DialectConversionPattern can therefore access the converted regions of the
original operation and process them further if necessary. In particular, the
conversion is allowed to move the blocks from the original region to other
regions and to split blocks into multiple blocks. All block manipulations must
be performed through the PatternRewriter to ensure they will be undone if the
conversion fails.
Port the pass converting from the affine dialect (loops and ifs with bodies as
regions) to the standard dialect (branch-based cfg) to use DialectConversion in
order to exercise this new functionality. The modification to the lowering
functions are minor and are focused on using the PatterRewriter instead of
directly modifying the IR.
PiperOrigin-RevId: 252625169
* 'get' methods that allow constructing from an ArrayRef of integer or floating point values.
* A 'reshape' method to allow for changing the shape without changing the underlying data.
PiperOrigin-RevId: 252067898
* Add a getCurrentLocation that returns the location directly.
* Add parseOperandList/parseTrailingOperandList overloads without the required operand count.
PiperOrigin-RevId: 251585488
We want to support 64-bit shapes (even when the compiler is on a 32-bit architecture). Using int64_t consistently allows us to sidestep the bugginess of unsigned arithmetic.
Still unsigned: kind, memory space, and bit width. The first two are basically enums. We could have a discussion about the last one, but it's basically just a very large enum as well and we're not doing any math on it, I think.
--
PiperOrigin-RevId: 250985791
Extract common methods into ShapedType.
Simplify methods.
Remove some extraneous asserts.
Replace sentinel value with a helper method to check the same.
--
PiperOrigin-RevId: 250945261
MemRefs have the same notion of shape, rank, and fixed element type. This allows us to reuse utilities based on shape for memref.
All dyn_cast and isa calls for ShapedType have been checked and either modified to explicitly check for vector or tensor, or confirmed to not depend on the result being a vector or tensor.
Discussion in https://groups.google.com/a/tensorflow.org/forum/#!topic/mlir/cHLoyfGu8y8
--
PiperOrigin-RevId: 250945184
This better matches the other methods in ShapedType which only make sense for ranked types. There's now an explicit hasRank for checking the rank. Actual call sites rarely used the "-1" sentinel to combine checking for rankedness and checking that rank is a certain value. And in most cases they should actually be checking for rankedness at a higher level using type predicates. Using an explicit method is clearer than a sentinel anyway.
--
PiperOrigin-RevId: 250720853
The current logic assumes that ShapedType indicates a vector or tensor, which will not be true soon when MemRef subclasses ShapedType
--
PiperOrigin-RevId: 250586364
Fix Block::splitBlock and Block::eraseFromFunction that erronously assume
blocks belong to functions. They now belong to regions. When splitting, new
blocks should be created in the same region as the existing block. When
erasing a block, it should be removed from the region rather than from the
function body that transitively contains the region.
Also rename Block::eraseFromFunction to Block::erase for consistency with other
IR containers.
--
PiperOrigin-RevId: 250278272
Introduces a hasRank() method to make checking for rank a bit easier.
This is partially to make it easier to make MemRef subclass ShapedType
--
PiperOrigin-RevId: 249927442
* There is no longer a need to explicitly remap function attrs.
- This removes a potentially expensive call from the destructor of Function.
- This will enable some interprocedural transformations to now run intraprocedurally.
- This wasn't scalable and forces dialect defined attributes to override
a virtual function.
* Replacing a function is now a trivial operation.
* This is a necessary first step to representing functions as operations.
--
PiperOrigin-RevId: 249510802
This avoids crashing when trying to dump an operation nested in a region that isn't yet attached to an operation, which is quite useful when debugging.
This alone won't be enough to print an unlink Operation, it'll display `<<UNLINKED INSTRUCTION>>`.
--
PiperOrigin-RevId: 249496388
MLIRContext does not have to be aware of the SDBM unique data structures
directly. Move the SDBM storage uniquer from MLIRContext to the SDBM dialect
instance. Expressions that previously required a context to be constructed now
require an instance of the dialect in order to access the uniquer. While they
could look up the dialect in the context, it would have introduced a rather
expensive lookup into each construction. Instead, the caller is expected to
obtain the dialect instance and cache it.
--
PiperOrigin-RevId: 249245199
We now have sufficient extensibility in dialects to move attribute components
such as SDBM out of the core IR into a dedicated dialect and make them
optional. Introduce an SDBM dialect and move the code. This is a mostly
non-functional change.
--
PiperOrigin-RevId: 249244802
SDBM expressions are designed as components of an attribute, similarly to
affine expressions. As such, they need to be unique'd in the MLIRContext.
When SDBM expressions were implemented, uniqu'ing objects in a context required
to modify MLIRContext implementation. This is no longer the case as generic
StorageUniquer has been introduced. Port the SDBMExpr uniqu'ing to use a newly
introduced uniquer and remove SDBM construction from MLIRContext.cpp.
--
PiperOrigin-RevId: 249244739
Affine expressions are designed as components of an attribute and are unique'd
in the MLIRContext. When affine expressions were implemented, uniqu'ing
objects in a context required to modify MLIRContext implementation. This is no
longer the case as generic StorageUniquer has been introduced. Port the
AffineExpr construction to use the new infrastructure by introducing an
affineUniquer into the MLIRContext.
--
PiperOrigin-RevId: 249207539
The current implementation makes some assumptions about what can be a shaped type, which aren't really necessary. It also has strange behavior for types that aren't in the limited set it handles (e.g. dialect-defined types)
Updated the comment to match the implementation.
This is partially motivated by the desire to make MemRef a subclass of ShapedType
--
PiperOrigin-RevId: 248859674
Provide an "unsafe" version of the overloaded arithmetic operators for SDBM
expressions. These operators expect the operands to be of the right SDBM
expression subtype and assert if they are not. They also perform simple
folding operations as well as some semantically correct operations that
construct an SDBM expression of a different subtype, e.g., a difference
expression if the RHS of an operator+ is a negated variable. These operators
are scoped in a namespace to allow for a future "safe" version of the operators
that propagates null expressions to denote the error state when expressions
have wrong subtypes.
--
PiperOrigin-RevId: 248704153
Implement the storage class for striped difference-bound matrices (SDBM) as a
container with a difference bounds matrix and a list of stripe expressions. An
SDBM defines an integer set. Provide conversion mechanisms between lists of
SDBM expressions treated as equalities with zero or less-than-or-equal
inequalities with zero.
--
PiperOrigin-RevId: 248702871
tensor<*xf32> could be a tensor<1xf32> at runtime but this verifyShapeMatch would return failure and say function is invalid.
--
PiperOrigin-RevId: 248583038
This is in preparation for making it also support/be a parent class of MemRefType. MemRefs have similar shape/rank/element semantics and it would be useful to be able to use these same utilities for them.
This CL should not change any semantics and only change variables, types, string literals, and comments. In follow-up CLs I will prepare all callers to handle MemRef types or remove their dependence on ShapedType.
Discussion/Rationale in https://groups.google.com/a/tensorflow.org/forum/#!topic/mlir/cHLoyfGu8y8
--
PiperOrigin-RevId: 248476449
This CL gives a pattern-matching-y look and feel to AffineExpr.
For now this uses a shared_ptr instead of unique'ing into a bumpPtrAllocator.
SDBM gives a simple use case with more idiomatic syntax for matchers.
--
PiperOrigin-RevId: 248188075
Example:
/tmp/file_C.py:21:5: error: 'foo.bar' op attribute 'something'
raise app.UsageError('Too many command-line arguments.')
^
/tmp/file_D.py:20:3: note: called from
if len(argv) > 1:
^
/tmp/file_E.py:19:1: note: called from
def main(argv):
^
/tmp/file_F.py:24:3: note: called from
app.run(main)
^
--
PiperOrigin-RevId: 248151212
Example:
/tmp/file_C.py:17:1: error: 'foo.bar' op attribute 'something' ...
app.run(main)
^
/tmp/file_D.py:14:1: note: called from
raise app.UsageError('Too many command-line arguments.')
^
/tmp/file_E.py:12:1: note: called from
def main(argv):
^
/tmp/file_F.py:13:1: note: called from
if len(argv) > 1:
^
--
PiperOrigin-RevId: 248074804
This closely mirrors the llvm fcmp instruction, defining 16 different predicates
Constant folding is unsupported for NaN and Inf because there's no way to represent those as constants at the moment
--
PiperOrigin-RevId: 246932358
`#` alias `=` attribute-value
This also allows for dialects to define aliases for attributes in the AsmPrinter. The printer supports two types of attribute aliases, 'direct' and 'kind'.
* Direct aliases are synonymous with the current support for type aliases, i.e. this maps an alias to a specific instance of an attribute.
// A direct alias ("foo_str") for the string attribute "foo".
#foo_str = "foo"
* Kind aliases generates unique names for all instances of a given attribute kind. The generated aliases are of the form: `alias[0-9]+`.
// A kind alias ("strattr") for all string attributes could generate.
#strattr0 = "foo"
#strattr1 = "bar"
...
#strattrN = "baz"
--
PiperOrigin-RevId: 246851916
The generic form of operations currently supports optional regions to be
located after the operation type. As we are going to add a type to each
region in a leading position in the region syntax, similarly to functions, it
becomes ambiguous to have regions immediately after the operation type. Put
regions between operands the optional list of successors in the generic
operation syntax and wrap them in parentheses. The effect on the exisitng IR
syntax is minimal since only three operations (`affine.for`, `affine.if` and
`gpu.kernel`) currently use regions.
--
PiperOrigin-RevId: 246787087
This trait only works for tensor and vector types at the moment, verifying that the element type of an op with only tensor and vector types match. Added a unit test for it as there is no op currently in core that uses this trait.
--
PiperOrigin-RevId: 246661697
Notes are a way to add additional context to a diagnostic, but don't really make sense as standalone diagnostics. Moving forward, notes will no longer be able to be constructed directly and must be attached to a parent Diagnostic.
Notes can be attached via `attachNote`:
auto diag = ...;
diag.attachNote() << "This is a note";
--
PiperOrigin-RevId: 246545971
The Diagnostic class contains all of the information necessary to report a diagnostic to the DiagnosticEngine. It should generally not be constructed directly, and instead used transitively via InFlightDiagnostic. A diagnostic is currently comprised of several different elements:
* A severity level.
* A source Location.
* A list of DiagnosticArguments that help compose and comprise the output message.
* A DiagnosticArgument represents any value that may be part of the diagnostic, e.g. string, integer, Type, Attribute, etc.
* Arguments can be added to the diagnostic via the stream(<<) operator.
* (In a future cl) A list of attached notes.
* These are in the form of other diagnostics that provide supplemental information to the main diagnostic, but do not have context on their own.
The InFlightDiagnostic class represents an RAII wrapper around a Diagnostic that is set to be reported with the diagnostic engine. This allows for the user to modify a diagnostic that is inflight. The internally wrapped diagnostic can be reported directly or automatically upon destruction.
These classes allow for more natural composition of diagnostics by removing the restriction that the message of a diagnostic is comprised of a single Twine. They should also allow for nice incremental improvements to the diagnostics experience in the future, e.g. formatv style diagnostics.
Simple Example:
emitError(loc, "integer bitwidth is limited to " + Twine(IntegerType::kMaxWidth) + " bits");
emitError(loc) << "integer bitwidth is limited to " << IntegerType::kMaxWidth << " bits";
--
PiperOrigin-RevId: 246526439
This CL adds a primitive tiling pass for Linalg.
The tiling pass uses the loopToOperandRangesMaps property which should be ideally Tablegen'd and in-class.
The tiling specification uses 0 as a convention to skip loops that should not be tiled.
Tiling proceeds in 3 steps, for each op:
1. Pad tile sizes with 0 to match the number of loops, this simplifies the implementation and avoids affine map manipulations to align dimensions.
2. Create loop ranges that represent the min/max/step by which to iterate. This should be later complemented by a range intersection to avoid the out-of-bounds case.
3. Map the loop ranges to view ranges in order to create subviews on which the op can be called.
Relevant utility and helper functions are added separately that support writing the transformation in a declarative fashion.
Simplifying assumptions are made for now on the views and the ranges that are constructed
in the function and are not passed as function arguments. This restriction will be lifted
in the future.
--
PiperOrigin-RevId: 246124419
Define a new dialect related to GPU kernels. Currently, it only contains a
single operation for launching a kernel on a three-dimensional grid of thread
blocks, following a model similar to that of CUDA. In particular, the body of
the kernel contains operations executed by each thread and uses region
arguments to accept thread and block identifiers (similar to how the loop body
region accepts the induction value).
--
PiperOrigin-RevId: 245713728
Instead, fold such operations. This way callers don't need to conditionally create cast operations depending on if a value already has the target type.
Also, introduce areCastCompatible to allow cast users to verify that the generated op will be valid before creating the operation.
TESTED with unit tests
--
PiperOrigin-RevId: 245606133
none-type ::= `none`
The `none` type is a unit type, i.e. a type with exactly one possible value, where its value does not have a defined dynamic representation.
--
PiperOrigin-RevId: 245599248
Since SDBM expressions are a subset of affine expressions, they can be
converted to affine expressions in a straightforward way. The inverse
conversion may fail when the affine expression is not an SDBM. Implement the
inverse convresion assuming affine expressions are simplified and
canonicalizied, detect subtractive and multiplicative forms of the stripe
operation.
--
PiperOrigin-RevId: 245494735
Striped difference-bound matrix expressions are a subset of affine expressions
supporting low-complexity algorithms that can be useful for loop
transformations. This introduces the basic data data structures for building
such expressions and unique'ing them in a MLIRContext.
--
PiperOrigin-RevId: 245380206
A unit attribute is an attribute that represents a value of `unit` type. The
`unit` type allows only one value forming a singleton set. This attribute value
is used to represent attributes that only have meaning from their existence.
One example of such an attribute could be the `swift.self` attribute. This attribute indicates that a function parameter is the self/context
parameter. It could be represented as a boolean attribute(true or false), but a
value of false doesn't really bring any value. The parameter either is the
self/context or it isn't.
```mlir {.mlir}
// A unit attribute defined with the `unit` value specifier.
func @verbose_form(i1 {unitAttr : unit})
// A unit attribute can also be defined without the `unit` value specifier.
func @simple_form(i1 {unitAttr})
```
--
PiperOrigin-RevId: 245254045
Add member functions for Regions to query if another Region is a ancestor. The
implementation is naive and traverses all parent regions performing one-to-one
comparisons. As a side effect, this introduces `getContainingRegion` function
for Operations and Values to return the Region in which they are defined, and
for Regions to return the "parent" region if any.
--
PiperOrigin-RevId: 245057980
Extract common code from getAffineSymbolExpr and getAffineConstantExpr into a
utility function safeGetOrCreate, similarly to the existing overloads for sets
and maps. The position in the vector is used as indexing key. NFC.
--
PiperOrigin-RevId: 244820859
During the pattern rewrite, if the function is changed, i.e. ops created,
deleted or swapped, the pattern rewriter needs to re-scan the function entirely
and apply the patterns again, so the patterns whose root ops have been popped
out from the working list nor an immediate users of the changed ops can be
reconsidered.
A command line flag is added to set the max number of iterations rescanning the
function for pattern match. If the rewrite doesn' converge after this number,
this compiling will continue and the result can be sub-optimal.
One unit test is updated because this change fixed the missing optimization opportunities.
--
PiperOrigin-RevId: 244754190
The per-layer format is now like:
!quant.uniform<i8<-8:7>:f32, 9.987200e-01:127>
and per-axis is:
!quant.uniform<i8:f32:1, {2.0e+2,0.99872:120}>
I used the following sed script to update the unit tests (invoked with commands like `sed -i -r -f fix_quant.sed $(find . -name '*.mlir')`).
---
# Per-layer
s|\!quant<"uniform\[([iu][0-9]+):([fb]+[0-9]+)\]\{([^\}]+)\}\s*">|!quant.uniform<\1:\2, \3>|g
s|\!quant<"uniform\[([iu][0-9]+)\(([^\)]+)\):([fb]+[0-9]+)\]\{([^\}]+)\}\s*">|!quant.uniform<\1<\2>:\3, \4>|g
# Per-axis
s|\!quant<"uniform\[([iu][0-9]+):([fb]+[0-9]+)(:[0-9]+)?\]\{([^\}]+)\}\s*">|!quant.uniform<\1:\2\3, {\4}>|g
s|\!quant<"uniform\[([iu][0-9]+)\(([^\)]+)\):([fb]+[0-9]+)(:[0-9]+)?\]\{([^\}]+)\}\s*">|!quant.uniform<\1<\2>:\3\4, {\5}>|g
---
I fixed up the one file of error cases manually.
Since this is a one time syntax fix, I am not persisting the script anywhere.
--
PiperOrigin-RevId: 244425331
other characters within the <>'s now that we can. This will allow quantized
types to use the pretty syntax (among others) after a few changes.
--
PiperOrigin-RevId: 243521268
making the IR dumps much nicer.
This is part 2/3 of the path to making dialect types more nice. Part 3/3 will
slightly generalize the set of characters allowed in pretty types and make it
more principled.
--
PiperOrigin-RevId: 242249955
Currently, we only make the initial address aligned with 64-bit address but
allocate the buffer with the real size. This can cause issue when we extract
the value by the `readBits` method, which needs to read the memory in the
granularity of APINT_WORD_SIZE. In this CL, we rounded the allocation size to
the multiplies of APINT_WORD_SIZE to fix the issue.
--
PiperOrigin-RevId: 241816656
Example:
func @unknown_std_op() {
%0 = "std.foo_bar_op"() : () -> index
return
}
Will result in:
error: unregistered operation 'std.foo_bar_op' found in dialect ('std') that does not allow unknown operations
--
PiperOrigin-RevId: 241266009
have no standard ops for working with these yet, this is simply enough to
represent and round trip them in the printer and parser.
--
PiperOrigin-RevId: 241102728
Dialect conversion currently clones the operations that did not match any
pattern. This includes cloning any regions that belong to these operations.
Instead, apply conversion recursively to the nested regions.
Note that if an operation matched one of the conversion patterns, it is up to
the pattern rewriter to fill in the regions of the converted operation. This
may require calling back to the converter and is left for future work.
PiperOrigin-RevId: 240872410
Example:
%call:2 = call @multi_return() : () -> (f32, i32)
use(%calltensorflow/mlir#0, %calltensorflow/mlir#1)
This cl also adds parser support for uniquely named result values. This means that a test writer can now write something like:
%foo, %bar = call @multi_return() : () -> (f32, i32)
use(%foo, %bar)
Note: The printer will still print the collapsed form.
PiperOrigin-RevId: 240860058
Due to legacy reasons (ML/CFG function separation), regions in affine control
flow operations require contained blocks not to have terminators. This is
inconsistent with the notion of the block and may complicate code motion
between regions of affine control operations and other regions.
Introduce `affine.terminator`, a special terminator operation that must be used
to terminate blocks inside affine operations and transfers the control back to
he region enclosing the affine operation. For brevity and readability reasons,
allow `affine.for` and `affine.if` to omit the `affine.terminator` in their
regions when using custom printing and parsing format. The custom parser
injects the `affine.terminator` if it is missing so as to always have it
present in constructed operations.
Update transformations to account for the presence of terminator. In
particular, most code motion transformation between loops should leave the
terminator in place, and code motion between loops and non-affine blocks should
drop the terminator.
PiperOrigin-RevId: 240536998
Currently, regions can only be constructed by passing in a `Function` or an
`Instruction` pointer referencing the parent object, unlike `Function`s or
`Instruction`s themselves that can be created without a parent. It leads to a
rather complex flow in operation construction where one has to create the
operation first before being able to work with its regions. It may be
necessary to work with the regions before the operation is created. In
particular, in `build` and `parse` functions that are executed _before_ the
operation is created in cases where boilerplate region manipulation is required
(for example, inserting the hypothetical default terminator in affine regions).
Allow creating standalone regions. Such regions are meant to own a list of
blocks and transfer them to other regions on demand.
Each instruction stores a fixed number of regions as trailing objects and has
ownership of them. This decreases the size of the Instruction object for the
common case of instructions without regions. Keep this behavior intact. To
allow some flexibility in construction, make OperationState store an owning
vector of regions. When the Builder creates an Instruction from
OperationState, the bodies of the regions are transferred into the
instruction-owned regions to minimize copying. Thus, it becomes possible to
fill standalone regions with blocks and move them to an operation when it is
constructed, or move blocks from a region to an operation region, e.g., for
inlining.
PiperOrigin-RevId: 240368183
This combined match/rewrite functionality allows simplifying the majority of existing RewritePatterns, as they do not benefit from separate match and rewrite functions.
Some of the existing canonicalization patterns in StandardOps have been modified to take advantage of this functionality.
PiperOrigin-RevId: 240187856
tblgen be non-const. This requires introducing some const_cast's at the
moment, but those (and lots more stuff) will disappear in subsequent patches.
This significantly simplifies those patches because the various tblgen op emitters
get adjusted.
PiperOrigin-RevId: 239954566
This also eliminates some incorrect reinterpret_cast logic working around it, and numerous const-incorrect issues (like block argument iteration).
PiperOrigin-RevId: 239712029
River Riddle