This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
Concatting lists in TableGen is easy, creating unique lists less so. There is no reason for duplicated op traits so we could throw an error instead but duplicates could occur due to concatting different list of traits in ODS (e.g., for convenience reasons), so just dedup them during Operator trait construction instead.
PiperOrigin-RevId: 286488423
Certain operations can have multiple variadic operands and their size
relationship is not always known statically. For such cases, we need
a per-op-instance specification to divide the operands into logical
groups or segments. This can be modeled by attributes.
This CL introduces C++ trait AttrSizedOperandSegments for operands and
AttrSizedResultSegments for results. The C++ trait just guarantees
such size attribute has the correct type (1D vector) and values
(non-negative), etc. It serves as the basis for ODS sugaring that
with ODS argument declarations we can further verify the number of
elements match the number of ODS-declared operands and we can generate
handy getter methods.
PiperOrigin-RevId: 282467075
This changes changes the OpDefinitionsGen to automatically add the OpAsmOpInterface for operations with multiple result groups using the provided ODS names. We currently just limit the generation to multi-result ops as most single result operations don't have an interesting name(result/output/etc.). An example is shown below:
// The following operation:
def MyOp : ... {
let results = (outs AnyType:$first, Variadic<AnyType>:$middle, AnyType);
}
// May now be printed as:
%first, %middle:2, %0 = "my.op" ...
PiperOrigin-RevId: 281834156
The `Operator` class keeps an `arguments` field, which contains pointers
to `operands` and `attributes` elements. Thus it must be populated after
`operands` and `attributes` are finalized so to have stable pointers.
SmallVector may re-allocate when still having new elements added, which
will invalidate pointers.
PiperOrigin-RevId: 280466896
Previously DRR assumes attributes to appear after operands. This was the
previous requirements on ODS, but that has changed some time ago. Fix
DRR to also support interleaved operands and attributes.
PiperOrigin-RevId: 275983485
For ops in SPIR-V dialect that are a direct mirror of SPIR-V
operations, the serialization/deserialization methods can be
automatically generated from the Op specification. To enable this an
'autogenSerialization' field is added to SPV_Ops. When set to
non-zero, this will enable the automatic (de)serialization function
generation
Also adding tests that verify the spv.Load, spv.Store and spv.Variable
ops are serialized and deserialized correctly. To fully support these
tests also add serialization and deserialization of float types and
spv.ptr types
PiperOrigin-RevId: 258684764
Some operations need to override the default behavior of builders, in
particular region-holding operations such as affine.for or tf.graph want to
inject default terminators into the region upon construction, which default
builders won't do. Provide a flag that disables the generation of default
builders so that the custom builders could use the same function signatures.
This is an intentionally low-level and heavy-weight feature that requires the
entire builder to be implemented, and it should be used sparingly. Injecting
code into the end of a default builder would depend on the naming scheme of the
default builder arguments that is not visible in the ODS. Checking that the
signature of a custom builder conflicts with that of a default builder to
prevent emission would require teaching ODG to differentiate between types and
(optional) argument names in the generated C++ code. If this flag ends up
being used a lot, we should consider adding traits that inject specific code
into the default builder.
PiperOrigin-RevId: 256640069
Support for ops with variadic operands/results will come later; but right now
a proper message helps to avoid deciphering confusing error messages later in
the compilation stage.
PiperOrigin-RevId: 254071820
This CL enables verification code generation for variadic operands and results.
In verify(), we use fallback getter methods to access all the dynamic values
belonging to one static variadic operand/result to reuse the value range
calculation there.
PiperOrigin-RevId: 252288219
When manipulating generic operations, such as in dialect conversion /
rewriting, it is often necessary to view a list of Values as operands to an
operation without creating the operation itself. The absence of such view
makes dialect conversion patterns, among others, to use magic numbers to obtain
specific operands from a list of rewritten values when converting an operation.
Introduce XOpOperandAdaptor classes that wrap an ArrayRef<Value *> and provide
accessor functions identical to those available in XOp. This makes it possible
for conversions to use these adaptors to address the operands with names rather
than rely on their position in the list. The adaptors are generated from ODS
together with the actual operation definitions.
This is another step towards making dialect conversion patterns specific for a
given operation.
Illustrate the approach on conversion patterns in the standard to LLVM dialect
conversion.
PiperOrigin-RevId: 251232899
Similar to arguments and results, now we require region definition in ops to
be specified as a DAG expression with the 'region' operator. This way we can
specify the constraints for each region and optionally give the region a name.
Two kinds of region constraints are added, one allowing any region, and the
other requires a certain number of blocks.
--
PiperOrigin-RevId: 250790211
Previously we force the C++ namespaces to be `NS` if `SomeOp` is defined as
`NS_SomeOp`. This is too rigid as it does not support nested namespaces
well. This CL adds a "namespace" field into the Dialect class to allow
flexible namespaces.
--
PiperOrigin-RevId: 249064981
Simple mechanism to allow specifying arbitrary function declarations. The modelling will not cover all cases so allow a means for users to declare a method function that they will define in their C++ files. The goal is to allow full C++ flexibility as the goal is to cover cases not modelled.
--
PiperOrigin-RevId: 245889819
Enables specifying the documentation for dialect along with defining the ops of the dialect. The doc generator will be expanded in follow up to emit the documentation in the autogenerated files. This is precursor to allowing common base for all ops in a dialect.
All the dialect documentation is super sparse and just added as placeholder.
I was tempted (and started) to move ConstantOp to be generated too, but this will be easier post adding extra_methods, so deferring until then.
--
PiperOrigin-RevId: 245759984
Certain ops can have multiple variadic operands/results, e.g., `tf.DynamicStitch`.
Even if an op has only one variadic operand/result, it is not necessarily the
very last one, e.g., `tf.RaggedGather`. This CL enhances TableGen subsystem to be
able to represent such cases.
In order to deduce the operand/result value range for each variadic operand,
currently we only support variadic operands/results all of the same size.
So two new traits, `SameVariadicOperandSize` and `SameVariadicResultSize` are
introduced.
--
PiperOrigin-RevId: 245310628
Previously we bundle the existence check and the MLIR attribute kind check
in one call. Further constraints (like element bitwidth) have to be split
into following checks. That is not a nice separation given that we have more
checks for constraints. Instead, this CL changes to generate a local variable
for every attribute, check its existence first, then check the constraints.
Creating a local variable for each attribute also avoids querying it multiple
times using the raw getAttr() API. This is a win for both performance the
readability of the generated code.
This CL also changed the error message to be more greppable by delimiting
the error message from constraints with boilerplate part with colon.
--
PiperOrigin-RevId: 241906132
Before this CL, the result type of the pattern match results need to be as same
as the first operand type, operand broadcast type or a generic tensor type.
This CL adds a new trait to set the result type by attribute. For example, the
TFL_ConstOp can use this to set the output type to its value attribute.
PiperOrigin-RevId: 240441249
Previously we have multiple mechanisms to specify op definition and match constraints:
TypeConstraint, AttributeConstraint, Type, Attr, mAttr, mAttrAnyOf, mPat. These variants
are not added because there are so many distinct cases we need to model; essentially,
they are all carrying a predicate. It's just an artifact of implementation.
It's quite confusing for users to grasp these variants and choose among them. Instead,
as the OpBase TableGen file, we need to strike to provide an unified mechanism. Each
dialect has the flexibility to define its own aliases if wanted.
This CL removes mAttr, mAttrAnyOf, mPat. A new base class, Constraint, is added. Now
TypeConstraint and AttrConstraint derive from Constraint. Type and Attr further derive
from TypeConstraint and AttrConstraint, respectively.
Comments are revised and examples are added to make it clear how to use constraints.
PiperOrigin-RevId: 240125076
Previously Value was a pair of name & Type, but for operands/result a TypeConstraint rather then a Type is specified. Update C++ side to match declarative side.
PiperOrigin-RevId: 238984799
The existing implementation of the Op definition generator assumes and relies
on the fact that native Op Attributes appear after its value-based operands in
the Arguments list. Furthermore, the same order is used in the generated
`build` function for the operation. This is not desirable for some operations
with mandatory attributes that would want the attribute to appear upfront for
better consistency with their textual representation, for example `cmpi` would
prefer the `predicate` attribute to be foremost in the argument list.
Introduce support for using attributes and operands in the Arguments DAG in no
particular order. This is achieved by maintaining a list of Arguments that
point to either the value or the attribute and are used to generate the `build`
method.
PiperOrigin-RevId: 237002921
* Introduce a OpTrait class in C++ to wrap the TableGen definition;
* Introduce PredOpTrait and rename previous usage of OpTrait to NativeOpTrait;
* PredOpTrait allows specifying a trait of the operation by way of predicate on the operation. This will be used in future to create reusable set of trait building blocks in the definition of operations. E.g., indicating whether to operands have the same type and allowing locally documenting op requirements by trait composition.
- Some of these building blocks could later evolve into known fixed set as LLVMs backends do, but that can be considered with more data.
* Use the modelling to address one verify TODO in a very local manner.
This subsumes the current custom verify specification which will be removed in a separate mechanical CL.
PiperOrigin-RevId: 234827169
This CL extended TableGen Operator class to provide accessors for information on op
results.
In OpDefinitionGen, added checks to make sure only the last result can be variadic,
and adjusted traits and builders generation to consider variadic results.
PiperOrigin-RevId: 234596124
We specify op operands and results in TableGen op definition using the same syntax.
They should be modelled similarly in TableGen driver wrapper classes.
PiperOrigin-RevId: 234153332
For ops with the SameOperandsAndResultType trait, we know that all result types
should be the same as the first operand's type. So we can generate a build()
method without requiring result types as parameters and also invoke this method
when constructing such ops during expanding rewrite patterns.
Similarly for ops have broadcast behavior, we can define build() method to use
the deduced type as the result type. So we can also calling into this build()
method when constructing ops in RewriterGen.
PiperOrigin-RevId: 233988307
Previously, we were using the trait mechanism to specify that an op has variadic operands.
That led a discrepancy between how we handle ops with deterministic number of operands.
Besides, we have no way to specify the constraints and match against the variadic operands.
This CL introduced Variadic<Type> as a way to solve the above issues.
PiperOrigin-RevId: 232656104
Motivation for this change is to remove redundant TF type attributes for
TensorFlow ops. For example, tf$T: "tfdtype$DT_FLOAT". Type attributes can be derived using the MLIR operand or result MLIR types, attribute names and their mapping. This will also allow constant folding of instructions generated within MLIR (and not imported from TensorFlow) without adding type attributes for the instruction.
Derived attributes are populated while exporting MLIR to TF GraphDef using
auto-generated populators. Populators are only available for the ops that are generated by the TableGen.
Also, fixed Operator::getNumArgs method to exclude derived attributes as they are not
part of the arguments.
TESTED with unit test
PiperOrigin-RevId: 232531561
Similar to op operands and attributes, use DAG to specify operation's results.
This will allow us to provide names and matchers for outputs.
Also Defined `outs` as a marker to indicate the start of op result list.
PiperOrigin-RevId: 231422455
Similar to other tblgen:: abstractions, tblgen::Pattern hides the native TableGen
API and provides a nicer API that is more coherent with the TableGen definitions.
PiperOrigin-RevId: 231285143
Start doc generation pass that generates simple markdown output. The output is formatted simply[1] in markdown, but this allows seeing what info we have, where we can refine the op description (e.g., the inputs is probably redundant), what info is missing (e.g., the attributes could probably have a description).
The formatting of the description is still left up to whatever was in the op definition (which luckily, due to the uniformity in the .td file, turned out well but relying on the indentation there is fragile). The mechanism to autogenerate these post changes has not been added yet either. The output file could be run through a markdown formatter too to remove extra spaces.
[1]. This is not proposal for final style :) There could also be a discussion around single doc vs multiple (per dialect, per op), whether we want a TOC, whether operands/attributes should be headings or just formatted differently ...
PiperOrigin-RevId: 230354538
MLIR has support for type-polymorphic instructions, i.e. instructions that may
take arguments of different types. For example, standard arithmetic operands
take scalars, vectors or tensors. In order to express such instructions in
TableGen, we need to be able to verify that a type object satisfies certain
constraints, but we don't need to construct an instance of this type. The
existing TableGen definition of Type requires both. Extract out a
TypeConstraint TableGen class to define restrictions on types. Define the Type
TableGen class as a subclass of TypeConstraint for consistency. Accept records
of the TypeConstraint class instead of the Type class as values in the
Arguments class when defining operators.
Replace the predicate logic TableGen class based on conjunctive normal form
with the predicate logic classes allowing for abitrary combinations of
predicates using Boolean operators (AND/OR/NOT). The combination is
implemented using simple string rewriting of C++ expressions and, therefore,
respects the short-circuit evaluation order. No logic simplification is
performed at the TableGen level so all expressions must be valid C++.
Maintaining CNF using TableGen only would have been complicated when one needed
to introduce top-level disjunction. It is also unclear if it could lead to a
significantly simpler emitted C++ code. In the future, we may replace inplace
predicate string combination with a tree structure that can be simplified in
TableGen's C++ driver.
Combined, these changes allow one to express traits like ArgumentsAreFloatLike
directly in TableGen instead of relying on C++ trait classes.
PiperOrigin-RevId: 229398247
Expand type matcher template generator to consider a set of predicates that are known to
hold. This avoids inserting redundant checking for trivially true predicates
(for example predicate that hold according to the op definition). This only targets predicates that trivially holds and does not attempt any logic equivalence proof.
PiperOrigin-RevId: 228880468
This CL added a tblgen::Attribute class to wrap around raw TableGen
Record getValue*() calls on Attr defs, which will provide a nicer
API for handling TableGen Record.
PiperOrigin-RevId: 228581107
This CL added a tblgen::Type class to wrap around raw TableGen
Record getValue*() calls on Type defs, which will provide a
nicer API for handling TableGen Record.
The PredCNF class is also updated to work together with
tblgen::Type.
PiperOrigin-RevId: 228429090
Use "native" vs "derived" to differentiate attributes on ops: native ones
are specified when creating the op as a part of defining the op, while
derived ones are computed from properties of the op.
PiperOrigin-RevId: 228186962
Bind attributes similar to operands. Use to rewrite leakyreulo and const rewrite pattern. The attribute type/attributes are not currently checked so should only be used where the attributes match due to the construction of the op.
To support current attribute namespacing, convert __ in attribute name to "$" for matching purposes ('$' is not valid character in variable in TableGen).
Some simplification to make it simpler to specify indented ostream and avoid so many spaces. The goal is not to have perfectly formatted code generated but good enough so that its still easy to read for a user.
PiperOrigin-RevId: 228183639
Expand type to include matcher predicates. Use CNF form to allow specifying combinations of constraints for type. The matching call for the type is used to verify the construction of the operation as well as in rewrite pattern generation.
The matching initially includes redundant checks (e.g., even if the operand of the op is guaranteed to satisfy some requirement, it is still checked during matcher generation for now). As well as some of the traits specified now check what the generated code already checks. Some of the traits can be removed in future as the verify method will include the relevant checks based on the op definition already.
More work is needed for variadic operands.
CNF form is used so that in the follow up redundant checks in the rewrite patterns could be omitted (e.g., when matching a F32Tensor, one does not need to verify that op X's operand 0 is a Tensor if that is guaranteed by op X's definition). The alternative was to have single matcher function specified, but this would not allow for reasoning about what attributes already hold (at the level of PredAtoms).
Use this new operand type restrictions to rewrite BiasAdd with floating point operands as declarative pattern.
PiperOrigin-RevId: 227991412
* Match using isa
- This limits the rewrite pattern to ops defined in op registry but that is probably better end state (esp. for additional verification).
PiperOrigin-RevId: 227598946
Benjamin Kramer