Commit Graph

39 Commits

Author SHA1 Message Date
Mehdi Amini 02b6fb218e Fix clang-tidy issues in mlir/ (NFC)
Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115956
2021-12-20 20:25:01 +00:00
Jacques Pienaar 1ab3efac41 [mlir][python] Add fused location 2021-12-11 10:16:13 -08:00
Mehdi Amini be0a7e9f27 Adjust "end namespace" comment in MLIR to match new agree'd coding style
See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differential Revision: https://reviews.llvm.org/D115309
2021-12-08 06:05:26 +00:00
Stella Laurenzo bdc3183742 [mlir][python] Implement more SymbolTable methods.
* set_symbol_name, get_symbol_name, set_visibility, get_visibility, replace_all_symbol_uses, walk_symbol_tables
* In integrations I've been doing, I've been reaching for all of these to do both general IR manipulation and module merging.
* I don't love the replace_all_symbol_uses underlying APIs since they necessitate SYMBOL_COUNT walks and have various sharp edges. I'm hoping that whatever emerges eventually for this can still retain this simple API as a one-shot.

Differential Revision: https://reviews.llvm.org/D114687
2021-11-29 20:31:13 -08:00
Stella Laurenzo a6e7d024a9 [mlir][python] Add pyi stub files to enable auto completion.
There is no completely automated facility for generating stubs that are both accurate and comprehensive for native modules. After some experimentation, I found that MyPy's stubgen does the best at generating correct stubs with a few caveats that are relatively easy to fix:
  * Some types resolve to cross module symbols incorrectly.
  * staticmethod and classmethod signatures seem to always be completely generic and need to be manually provided.
  * It does not generate an __all__ which, from testing, causes namespace pollution to be visible to IDE code completion.

As a first step, I did the following:
  * Ran `stubgen` for `_mlir.ir`, `_mlir.passmanager`, and `_mlirExecutionEngine`.
  * Manually looked for all instances where unnamed arguments were being emitted (i.e. as 'arg0', etc) and updated the C++ side to include names (and re-ran stubgen to get a good initial state).
  * Made/noted a few structural changes to each `pyi` file to make it minimally functional.
  * Added the `pyi` files to the CMake rules so they are installed and visible.

To test, I added a `.env` file to the root of the project with `PYTHONPATH=...` set as per instructions. Then reload the developer window (in VsCode) and verify that completion works for various changes to test cases.

There are still a number of overly generic signatures, but I want to check in this low-touch baseline before iterating on more ambiguous changes. This is already a big improvement.

Differential Revision: https://reviews.llvm.org/D114679
2021-11-29 19:58:58 -08:00
Stella Laurenzo ace1d0ad3d [mlir][python] Normalize asm-printing IR behavior.
While working on an integration, I found a lot of inconsistencies on IR printing and verification. It turns out that we were:
  * Only doing "soft fail" verification on IR printing of Operation, not of a Module.
  * Failed verification was interacting badly with binary=True IR printing (causing a TypeError trying to pass an `str` to a `bytes` based handle).
  * For systematic integrations, it is often desirable to control verification yourself so that you can explicitly handle errors.

This patch:
  * Trues up the "soft fail" semantics by having `Module.__str__` delegate to `Operation.__str__` vs having a shortcut implementation.
  * Fixes soft fail in the presence of binary=True (and adds an additional happy path test case to make sure the binary functionality works).
  * Adds an `assume_verified` boolean flag to the `print`/`get_asm` methods which disables internal verification, presupposing that the caller has taken care of it.

It turns out that we had a number of tests which were generating illegal IR but it wasn't being caught because they were doing a print on the `Module` vs operation. All except two were trivially fixed:
  * linalg/ops.py : Had two tests for direct constructing a Matmul incorrectly. Fixing them made them just like the next two tests so just deleted (no need to test the verifier only at this level).
  * linalg/opdsl/emit_structured_generic.py : Hand coded conv and pooling tests appear to be using illegal shaped inputs/outputs, causing a verification failure. I just used the `assume_verified=` flag to restore the original behavior and left a TODO. Will get someone who owns that to fix it properly in a followup (would also be nice to break this file up into multiple test modules as it is hard to tell exactly what is failing).

Notes to downstreams:
  * If, like some of our tests, you get verification failures after this patch, it is likely that your IR was always invalid and you will need to fix the root cause. To temporarily revert to prior (broken) behavior, replace calls like `print(module)` with `print(module.operation.get_asm(assume_verified=True))`.

Differential Revision: https://reviews.llvm.org/D114680
2021-11-28 18:02:01 -08:00
River Riddle 120591e126 [mlir] Replace usages of Identifier with StringAttr
Identifier and StringAttr essentially serve the same purpose, i.e. to hold a string value. Keeping these seemingly identical pieces of functionality separate has caused problems in certain situations:

* Identifier has nice accessors that StringAttr doesn't
* Identifier can't be used as an Attribute, meaning strings are often duplicated between Identifier/StringAttr (e.g. in PDL)

The only thing that Identifier has that StringAttr doesn't is support for caching a dialect that is referenced by the string (e.g. dialect.foo). This functionality is added to StringAttr, as this is useful for StringAttr in generally the same ways it was useful for Identifier.

Differential Revision: https://reviews.llvm.org/D113536
2021-11-11 02:02:24 +00:00
Alex Zinenko 6981e5ec91 [mlir][python] fix constructor generation for optional operands in presence of segment attribute
The ODS-based Python op bindings generator has been generating incorrect
specification of the operand segment in presence if both optional and variadic
operand groups: optional groups were treated as variadic whereas they require
separate treatement. Make sure it is the case. Also harden the tests around
generated op constructors as they could hitherto accept the code for both
optional and variadic arguments.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D113259
2021-11-05 12:40:27 +01:00
rkayaith f78fe0b7b8 [mlir][python] Make Operation and Value hashable
This allows operations and values to be used as dict keys

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D112669
2021-11-03 10:40:03 +01:00
Alex Zinenko 30d61893fb [mlir] provide C API and Python bindings for symbol tables
Symbol tables are a largely useful top-level IR construct, for example, they
make it easy to access functions in a module by name instead of traversing the
list of module's operations to find the corresponding function.

Depends On D112886

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D112821
2021-11-02 14:22:58 +01:00
Alex Zinenko 24685aaeb7 [mlir][python] allow for detaching operations from a block
Provide support for removing an operation from the block that contains it and
moving it back to detached state. This allows for the operation to be moved to
a different block, a common IR manipulation for, e.g., module merging.

Also fix a potential one-past-end iterator dereference in Operation::moveAfter
discovered in the process.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D112700
2021-10-31 09:42:15 +01:00
Mehdi Amini f431d3878a Make Python MLIR Operation not iterable
The current behavior is conveniently allowing to iterate on the regions of an operation
implicitly by exposing an operation as Iterable. However this is also error prone and
code that may intend to iterate on the results or the operands could end up "working"
apparently instead of throwing a runtime error.
The lack of static type checking in Python contributes to the ambiguity here, it seems
safer to not do this and require and explicit qualification to iterate (`op.results`, `op.regions`, ...).

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D111697
2021-10-26 07:21:09 +00:00
Mehdi Amini e2f16be599 Fix clang-tidy warnings in MLIR Python bindings (NFC) 2021-10-19 17:15:20 +00:00
rkayaith d5429a13da [mlir][python] Add 'loc' property to ops
Add a read-only `loc` property to Operation and OpView

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D111972
2021-10-18 16:01:12 +02:00
Alex Zinenko 78f2dae00d [mlir][python] Provide some methods and properties for API completeness
When writing the user-facing documentation, I noticed several inconsistencies
and asymmetries in the Python API we provide. Fix them by adding:

- the `owner` property to regions, similarly to blocks;
- the `isinstance` method to any class derived from `PyConcreteAttr`,
  `PyConcreteValue` and `PyConreteAffineExpr`, similar to `PyConcreteType` to
  enable `isa`-like calls without having to handle exceptions;
- a mechanism to create the first block in the region as we could only create
  blocks relative to other blocks, with is impossible in an empty region.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D111556
2021-10-13 14:30:55 +02:00
Jacques Pienaar e67cbbef03 [mlir][python] Expose CallSiteLoc Python side
This exposes creating a CallSiteLoc with a callee & list of frames for
callers. Follows the creation approach in C++ side where a list of
frames may be provided.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D111670
2021-10-13 10:25:40 +02:00
Jacques Pienaar 04d76d3694 [mlir][python] Add nameloc getter
Expose the nameloc getter to Python API.

Differential Revision: https://reviews.llvm.org/D111663
2021-10-12 12:45:57 -07:00
Alex Zinenko ed9e52f3af [mlir][python] Usability improvements for Python bindings
Provide a couple of quality-of-life usability improvements for Python bindings,
in particular:

  * give access to the list of types for the list of op results or block
    arguments, similarly to ValueRange->TypeRange,

  * allow for constructing empty dictionary arrays,

  * support construction of array attributes by concatenating an existing
    attribute with a Python list of attributes.

All these are required for the upcoming customization of builtin and standard
ops.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D110946
2021-10-04 11:45:25 +02:00
Alex Zinenko afeda4b9ed [mlir][python] provide access to function argument/result attributes
Without this change, these attributes can only be accessed through the generic
operation attribute dictionary provided the caller knows the special operation
attribute names used for this purpose. Add some Python wrapping to support this
use case.

Also provide access to function arguments usable inside the function along with
a couple of quality-of-life improvements in using block arguments (function
arguments being the arguments of its entry block).

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D110758
2021-09-30 09:38:13 +02:00
John Demme 47cc166bc0 [MLIR] [Python] Make Attribute and Type hashable
Enables putting types and attributes in sets and in dicts as keys.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D110301
2021-09-22 19:59:03 -07:00
Stella Laurenzo 1fb2e842a9 [mlir][python] Forward _OperationBase _CAPIPtr to the Operation.
* ODS generated operations extend _OperationBase and without this, cannot be marshalled to CAPI functions.
* No test case updates: this kind of interop is quite hard to verify with in-tree tests.

Differential Revision: https://reviews.llvm.org/D110030
2021-09-20 18:52:05 -07:00
Stella Laurenzo f05ff4f757 [mlir][python] Apply py::module_local() to all classes.
* This allows multiple MLIR-API embedding downstreams to co-exist in the same process.
* I believe this is the last thing needed to enable isolated embedding.

Differential Revision: https://reviews.llvm.org/D108605
2021-08-30 22:18:43 -07:00
Stella Laurenzo 8e6c55c92c [mlir][python] Extend C/Python API to be usable for CFG construction.
* It is pretty clear that no one has tried this yet since it was both incomplete and broken.
* Fixes a symbol hiding issues keeping even the generic builder from constructing an operation with successors.
* Adds ODS support for successors.
* Adds CAPI `mlirBlockGetParentRegion`, `mlirRegionEqual` + tests (and missing test for `mlirBlockGetParentOperation`).
* Adds Python property: `Block.region`.
* Adds Python methods: `Block.create_before` and `Block.create_after`.
* Adds Python property: `InsertionPoint.block`.
* Adds new blocks.py test to verify a plausible CFG construction case.

Differential Revision: https://reviews.llvm.org/D108898
2021-08-30 08:28:00 -07:00
John Demme 96fbd5cd5e [MLIR] [Python] Add `owner` to `mlir.ir.Block`
Provides a way for python users to access the owning Operation from a Block.
2021-08-19 00:02:09 -07:00
John Demme 1689dade42 [MLIR] [Python] Allow 'operation.parent' to return 'None'
This is more Pythonic and better matches the C++ and C APIs.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D108183
2021-08-16 22:38:07 -07:00
River Riddle f8479d9de5 [mlir] Set the namespace of the BuiltinDialect to 'builtin'
Historically the builtin dialect has had an empty namespace. This has unfortunately created a very awkward situation, where many utilities either have to special case the empty namespace, or just don't work at all right now. This revision adds a namespace to the builtin dialect, and starts to cleanup some of the utilities to no longer handle empty namespaces. For now, the assembly form of builtin operations does not require the `builtin.` prefix. (This should likely be re-evaluated though)

Differential Revision: https://reviews.llvm.org/D105149
2021-07-28 21:00:10 +00:00
John Demme 5664c5e24e [MLIR] [Python] Add `owner` to PyValue and fix its parent reference
Adds `owner` python call to `mlir.ir.Value`.

Assuming that `PyValue.parentOperation` is intended to be the value's owner, this fixes the construction of it from `PyOpOperandList`.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D103853
2021-07-14 20:32:43 -07:00
John Demme c65bb760df [MLIR] [Python] Add Operation.parent
Attribute to get the parent operation of an operation.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D102981
2021-05-23 20:52:43 -07:00
Stella Laurenzo b57d6fe42e [mlir][Python] Add casting constructor to Type and Attribute.
* This makes them consistent with custom types/attributes, whose constructors will do a type checked conversion. Of course, the base classes can represent everything so never error.
* More importantly, this makes it possible to subclass Type and Attribute out of tree in sensible ways.

Differential Revision: https://reviews.llvm.org/D101734
2021-05-03 10:12:03 -07:00
Mike Urbach 49745f87e6 [mlir][python] Add `destroy` method to PyOperation.
This adds a method to directly invoke `mlirOperationDestroy` on the
MlirOperation wrapped by a PyOperation.

Reviewed By: stellaraccident, mehdi_amini

Differential Revision: https://reviews.llvm.org/D101422
2021-04-28 19:30:05 -06:00
Mike Urbach 6ff74f96fd [mlir][python] Update `PyOpResult.owner` to get the parent object.
Previously, this API would return the PyObjectRef, rather than the
underlying PyOperation.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D101416
2021-04-28 14:39:59 -06:00
Mike Urbach 63d16d06f5 [mlir] Support setting operand values in C and Python APIs.
This adds `mlirOperationSetOperand` to the IR C API, similar to the
function to get an operand.

In the Python API, this adds `operands[index] = value` syntax, similar
to the syntax to get an operand with `operands[index]`.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D101398
2021-04-27 20:17:47 -06:00
Mike Urbach 3f3d1c901d [MLIR][Python] Add capsule methods for pybind11 to PyValue.
Add the `getCapsule()` and `createFromCapsule()` methods to the
PyValue class, as well as the necessary interoperability.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D101090
2021-04-27 20:14:16 -06:00
Alex Zinenko 4acd8457d8 [mlir] Improve debug flag management in Python bindings
Expose the debug flag as a readable and assignable property of a
dedicated class instead of a write-only function. Actually test the fact
of setting the flag. Move test to a dedicated file, it has zero relation
to context_managers.py where it was added.

Arguably, it should be promoted from mlir.ir to mlir module, but we are
not re-exporting the latter and this functionality is purposefully
hidden so can stay in IR for now. Drop unnecessary export code.

Refactor C API and put Debug into a separate library, fix it to actually
set the flag to the given value.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D100757
2021-04-19 14:45:43 +02:00
Nicolas Vasilache caa159f044 [mlir][python] Add simple debugging and printing helpers
Differential Revision: https://reviews.llvm.org/D100643
2021-04-16 13:47:46 +00:00
John Demme 0126e90648 [MLIR] [Python] Add capsule methods for pybind11 to PyOperation
Add the `getCapsule()` and `createFromCapsule()` methods to the PyOperation class.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D99927
2021-04-06 14:29:03 -07:00
Stella Laurenzo 9a9214fa25 [mlir] Add C and python API for is_registered_operation.
* Suggested to be broken out of D99578

Differential Revision: https://reviews.llvm.org/D99638
2021-03-30 22:56:02 -07:00
Stella Laurenzo 8d05a28887 [mlir][python] Adapt to `segment_sizes` attribute type change.
* Broken by https://reviews.llvm.org/rG1a75be0023cd80fd8560d689999a63d4368c90e6
2021-03-19 18:47:00 -07:00
Stella Laurenzo 436c6c9c20 NFC: Break up the mlir python bindings into individual sources.
* IRModules.cpp -> (IRCore.cpp, IRAffine.cpp, IRAttributes.cpp, IRTypes.cpp).
* The individual pieces now compile in the 5-15s range whereas IRModules.cpp was starting to approach a minute (didn't capture a before time).
* More fine grained splitting is possible, but this represents the most obvious.

Differential Revision: https://reviews.llvm.org/D98978
2021-03-19 13:33:51 -07:00