Commit Graph

79 Commits

Author SHA1 Message Date
Stella Laurenzo ec294eb87b [mlir][linalg] Add an InitTensorOp python builder.
* This has the API I want but I am not thrilled with the implementation. There are various things that could be improved both about the way that Python builders are mapped and the way the Linalg ops are factored to increase code sharing between C++/Python.
* Landing this as-is since it at least makes the InitTensorOp usable with the right API. Will refactor underneath in follow-ons.

Differential Revision: https://reviews.llvm.org/D99000
2021-03-25 15:17:48 -07:00
Mehdi Amini 973ddb7d6e Define a `NoTerminator` traits that allows operations with a single block region to not provide a terminator
In particular for Graph Regions, the terminator needs is just a
historical artifact of the generalization of MLIR from CFG region.
Operations like Module don't need a terminator, and before Module
migrated to be an operation with region there wasn't any needed.

To validate the feature, the ModuleOp is migrated to use this trait and
the ModuleTerminator operation is deleted.

This patch is likely to break clients, if you're in this case:

- you may iterate on a ModuleOp with `getBody()->without_terminator()`,
  the solution is simple: just remove the ->without_terminator!
- you created a builder with `Builder::atBlockTerminator(module_body)`,
  just use `Builder::atBlockEnd(module_body)` instead.
- you were handling ModuleTerminator: it isn't needed anymore.
- for generic code, a `Block::mayNotHaveTerminator()` may be used.

Differential Revision: https://reviews.llvm.org/D98468
2021-03-25 03:59:03 +00:00
Stella Laurenzo 8d05a28887 [mlir][python] Adapt to `segment_sizes` attribute type change.
* Broken by https://reviews.llvm.org/rG1a75be0023cd80fd8560d689999a63d4368c90e6
2021-03-19 18:47:00 -07:00
Stella Laurenzo d9343e6153 [mlir][python] Function decorator for capturing a FuncOp from a python function.
* Moves this out of a test case where it was being developed to good effect and generalizes it.
* Having tried a number of things like this, I think this balances concerns reasonably well.

Differential Revision: https://reviews.llvm.org/D98989
2021-03-19 18:27:21 -07:00
Stella Laurenzo d4cba4a188 [mlir][linalg] Add structured op builders from python opdsl.
* Makes the wrapped functions of the `@linalg_structured_op` decorator callable such that they emit IR imperatively when invoked.
* There are numerous TODOs that I will keep working through to achieve generality.
* Will true up exception handling tests as the feature progresses (for things that are actually errors once everything is implemented).
* Includes the addition of an `isinstance` method on concrete types in the Python API.

Differential Revision: https://reviews.llvm.org/D98754
2021-03-19 11:20:36 -07:00
River Riddle e60d57451e [mlir][Python] Fix test broken after D98474 2021-03-16 16:52:20 -07:00
Alex Zinenko e82a30bdce [mlir] enable Python bindings for the MemRef dialect
A previous commit moved multiple ops from Standard to MemRef dialect.
Some of these ops are exercised in Python bindings. Enable bindings for
the newly created MemRef dialect and update a test accordingly.
2021-03-15 14:07:51 +01:00
Alex Zinenko 0fb4a201c0 [mlir] fix shared-lib build fallout of e2310704d8
The patch in question broke the build with shared libraries due to
missing dependencies, one of which would have been circular between
MLIRStandard and MLIRMemRef if added. Fix this by moving more code
around and swapping the dependency direction. MLIRMemRef now depends on
MLIRStandard, but MLIRStandard does _not_ depend on MLIRMemRef.
Arguably, this is the right direction anyway since numerous libraries
depend on MLIRStandard and don't necessarily need to depend on
MLIRMemref.

Other otable changes include:
- some EDSC code is moved inline to MemRef/EDSC/Intrinsics.h because it
  creates MemRef dialect operations;
- a utility function related to shape moved to BuiltinTypes.h/cpp
  because it only realtes to shaped types and not any particular dialect
  (standard dialect is erroneously believed to contain MemRefType);
- a Python test for the standard dialect is disabled completely because
  the ops it tests moved to the new MemRef dialect, but it is not
  exposed to Python bindings, and the change for that is non-trivial.
2021-03-15 13:41:38 +01:00
Vladislav Vinogradov f3bf5c053b [mlir] Model MemRef memory space as Attribute
Based on the following discussion:
https://llvm.discourse.group/t/rfc-memref-memory-shape-as-attribute/2229

The goal of the change is to make memory space property to have more
expressive representation, rather then "magic" integer values.

It will allow to have more clean ASM form:

```
gpu.func @test(%arg0: memref<100xf32, "workgroup">)

// instead of

gpu.func @test(%arg0: memref<100xf32, 3>)
```

Explanation for `Attribute` choice instead of plain `string`:

* `Attribute` classes allow to use more type safe API based on RTTI.
* `Attribute` classes provides faster comparison operator based on
  pointer comparison in contrast to generic string comparison.
* `Attribute` allows to store more complex things, like structs or dictionaries.
  It will allows to have more complex memory space hierarchy.

This commit preserve old integer-based API and implements it on top
of the new one.

Depends on D97476

Reviewed By: rriddle, mehdi_amini

Differential Revision: https://reviews.llvm.org/D96145
2021-03-10 12:57:27 +03:00
Stella Laurenzo e31c77b182 [mlir][python] Reorganize MLIR python into namespace packages.
* Only leaf packages are non-namespace packages. This allows most of the top levels to be split into different directories or deployment packages. In the previous state, the presence of __init__.py files at each level meant that the entire tree could only ever exist in one physical directory on the path.
* This changes the API usage slightly: `import mlir` will no longer do a deep import of `mlir.ir`, etc. This may necessitate some client code changes.
* Dialect gen code was restructured so that the user is responsible for providing the `my_dialect.py` file, which then must import its peer `_my_dialect_ops_gen`. This gives complete control of the dialect namespace to the user instead of to tablegen code, allowing further dialect-specific python APIs.
* Correspondingly, the previous extension modules `_my_dialect.py` are now `_my_dialect_ops_ext.py`.
* Now that the `linalg` namespace is open, moved the `linalg_opdsl` tool into it.
* This may require some corresponding downstream adjustments to npcomp, circt, et al:
  * Probably some shallow imports need to be converted to deep imports (i.e. not `import mlir` brings in the world).
  * Each tablegen generated dialect now needs an explicit `foo.py` which does a `from ._foo_ops_gen import *`. This is similar to the way that generated code operates in the C++ world.
  * If providing dialect op extensions, those need to be moved from `_foo.py` -> `_foo_ops_ext.py`.

Differential Revision: https://reviews.llvm.org/D98096
2021-03-08 23:01:34 -08:00
Stella Laurenzo 0b5f1b859f [mlir][linalg] Add linalg_opdsl tool first draft.
* Mostly imported from experimental repo as-is with cosmetic changes.
* Temporarily left out emission code (for building ops at runtime) to keep review size down.
* Documentation and lit tests added fresh.
* Sample op library that represents current Linalg named ops included.

Differential Revision: https://reviews.llvm.org/D97995
2021-03-05 11:45:09 -08:00
Mehdi Amini 13cb431719 Add basic JIT Python Bindings
This offers the ability to create a JIT and invoke a function by passing
ctypes pointers to the argument and the result.

Differential Revision: https://reviews.llvm.org/D97523
2021-03-03 18:19:40 +00:00
Jacques Pienaar fa211f3ce9 Update test error string post pass registration change 2021-02-20 15:54:52 -08:00
Stella Laurenzo 4c3f1be84f [mlir][python] Add python binding for AffineMapAttribute.
Differential Revision: https://reviews.llvm.org/D96815
2021-02-16 15:43:30 -08:00
Alex Zinenko b208e5bcd0 [mlir] Add Python bindings for IntegerSet
This follows up on the introduction of C API for the same object and is similar
to AffineExpr and AffineMap.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D95437
2021-01-26 17:32:51 +01:00
Stella Laurenzo fd226c9b02 [mlir][Python] Roll up of python API fixes.
* As discussed, fixes the ordering or (operands, results) -> (results, operands) in various `create` like methods.
* Fixes a syntax error in an ODS accessor method.
* Removes the linalg example in favor of a test case that exercises the same.
* Fixes FuncOp visibility to properly use None instead of the empty string and defaults it to None.
* Implements what was documented for requiring that trailing __init__ args `loc` and `ip` are keyword only.
* Adds a check to `InsertionPoint.insert` so that if attempting to insert past the terminator, an exception is raised telling you what to do instead. Previously, this would crash downstream (i.e. when trying to print the resultant module).
* Renames `_ods_build_default` -> `build_generic` and documents it.
* Removes `result` from the list of prohibited words and for single-result ops, defaults to naming the result `result`, thereby matching expectations and what is already implemented on the base class.
* This was intended to be a relatively small set of changes to be inlined with the broader support for ODS generating the most specific builder, but it spidered out once actually testing various combinations, so rolling up separately.

Differential Revision: https://reviews.llvm.org/D95320
2021-01-24 19:02:59 -08:00
Mehdi Amini 922b26cde4 Add Python bindings for the builtin dialect
This includes some minor customization for FuncOp and ModuleOp.

Differential Revision: https://reviews.llvm.org/D95022
2021-01-21 22:44:44 +00:00
Stella Laurenzo b62c7e0474 [mlir][python] Swap shape and element_type order for MemRefType.
* Matches how all of the other shaped types are declared.
* No super principled reason fro this ordering beyond that it makes the one that was different be like the rest.
* Also matches ordering of things like ndarray, et al.

Reviewed By: ftynse, nicolasvasilache

Differential Revision: https://reviews.llvm.org/D94812
2021-01-19 16:03:19 -08:00
Stella Laurenzo 71b6b010e6 [mlir][python] Factor out standalone OpView._ods_build_default class method.
* This allows us to hoist trait level information for regions and sized-variadic to class level attributes (_ODS_REGIONS, _ODS_OPERAND_SEGMENTS, _ODS_RESULT_SEGMENTS).
* Eliminates some splicey python generated code in favor of a native helper for it.
* Makes it possible to implement custom, variadic and region based builders with one line of python, without needing to manually code access to the segment attributes.
* Needs follow-on work for region based callbacks and support for SingleBlockImplicitTerminator.
* A follow-up will actually add ODS support for generating custom Python builders that delegate to this new method.
* Also includes the start of an e2e sample for constructing linalg ops where this limitation was discovered (working progressively through this example and cleaning up as I go).

Differential Revision: https://reviews.llvm.org/D94738
2021-01-19 09:29:57 -08:00
zhanghb97 c0f3ea8a08 [mlir][Python] Add checking process before create an AffineMap from a permutation.
An invalid permutation will trigger a C++ assertion when attempting to create an AffineMap from the permutation.
This patch adds an `isPermutation` function to check the given permutation before creating the AffineMap.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D94492
2021-01-13 09:32:32 +08:00
Stella Laurenzo 53c866c286 Enable python bindings for tensor, shape and linalg dialects.
* We've got significant missing features in order to use most of these effectively (i.e. custom builders, region-based builders).
* We presently also lack a mechanism for actually registering these dialects but they can be use with contexts that allow unregistered dialects for further prototyping.

Differential Revision: https://reviews.llvm.org/D94368
2021-01-11 12:35:49 -08:00
Alex Zinenko 547e3eef14 [mlir] Expose MemRef layout in Python bindings
This wasn't possible before because there was no support for affine expressions
as maps. Now that this support is available, provide the mechanism for
constructing maps with a layout and inspecting it.

Rework the `get` method on MemRefType in Python to avoid needing an explicit
memory space or layout map. Remove the `get_num_maps`, it is too low-level,
using the length of the now-avaiable pseudo-list of layout maps is more
pythonic.

Depends On D94297

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D94302
2021-01-11 19:57:16 +01:00
Alex Zinenko e79bd0b4f2 [mlir] More Python bindings for AffineMap
Now that the bindings for AffineExpr have been added, add more bindings for
constructing and inspecting AffineMap that consists of AffineExprs.

Depends On D94225

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D94297
2021-01-11 19:57:15 +01:00
Alex Zinenko 74628c4305 [mlir] Add Python bindings for AffineExpr
This adds the Python bindings for AffineExpr and a couple of utility functions
to the C API. AffineExpr is a top-level context-owned object and is modeled
similarly to attributes and types. It is required, e.g., to build layout maps
of the built-in memref type.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D94225
2021-01-11 19:57:13 +01:00
Alex Zinenko 906efeec0a [mlir] don't match the text produced only in debug mode in Python tests
Some Python bindings tests were using FileCheck to match parts of the
error description produced only in the debug compilation mode. Remove
these parts (but keep the main message) to ensure tests also pass when
running them in the release compilation mode.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D94221
2021-01-07 19:57:28 +01:00
zhanghb97 abb4cd3e74 [mlir][Python] Initial Affine Map Python Bindings.
- Add `PyAffineMap` to wrap around `MlirAffineMap`.
- Add `mlirPythonAffineMapToCapsule` and `mlirPythonCapsuleToAffineMap` to interoperate with python capsule.
- Add and test some simple bindings of `PyAffineMap`.

Differential Revision: https://reviews.llvm.org/D93200
2020-12-30 17:36:39 +08:00
Siddharth Krishna 14056c88d6 [mlir][Python] Add an Operation.name property
Reviewed By: stellaraccident, mehdi_amini

Differential Revision: https://reviews.llvm.org/D93474
2020-12-29 14:14:32 -08:00
Stella Laurenzo f5665a2486 [mlir][python] Add Operation.verify().
Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D93913
2020-12-29 14:10:31 -08:00
Stella Laurenzo df7ddeea66 [mlir][python] Add FlatSymbolRef attribute.
Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D93909
2020-12-29 12:24:28 -08:00
kweisamx c84b53ca9b [mlir] Add Python binding for MLIR Dict Attribute
Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D93004
2020-12-13 04:30:35 +00:00
Mehdi Amini 285c0aa262 Add MLIR Python binding for Array Attribute
Differential Revision: https://reviews.llvm.org/D92948
2020-12-10 20:51:34 +00:00
Mehdi Amini e56f398dd3 Add Python binding for MLIR Type Attribute
Differential Revision: https://reviews.llvm.org/D92711
2020-12-07 23:06:58 +00:00
Mehdi Amini e15ae454b4 Customize exception thrown from mlir.Operation.create() python bindings
The default exception handling isn't very user friendly and does not
point accurately to the issue. Instead we can indicate which of the
operands isn't valid and provide contextual information in the error
message.

Differential Revision: https://reviews.llvm.org/D92710
2020-12-07 23:06:58 +00:00
River Riddle 09f7a55fad [mlir][Types][NFC] Move all of the builtin Type classes to BuiltinTypes.h
This is part of a larger refactoring the better congregates the builtin structures under the BuiltinDialect. This also removes the problematic "standard" naming that clashes with the "standard" dialect, which is not defined within IR/. A temporary forward is placed in StandardTypes.h to allow time for downstream users to replaced references.

Differential Revision: https://reviews.llvm.org/D92435
2020-12-03 18:02:10 -08:00
Mehdi Amini 1c2159494d Use the generic form when printing from the python bindings and the verifier fails
This reduces the chances of segfault. While it is a good practice to ensure
robust custom printers, it is unfortunately common to have them crash on
invalid input.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D92536
2020-12-03 18:45:00 +00:00
Stella Laurenzo 62195b7548 [mlir][CAPI] Convert the rest of the API int -> bool.
* Follows on https://reviews.llvm.org/D92193
* I had a mid-air collision with some additional occurrences and then noticed that there were a lot more. Think I got them all.

Differential Revision: https://reviews.llvm.org/D92292
2020-11-29 20:36:42 -08:00
Stella Laurenzo ba0fe76b7e [mlir][Python] Add an Operation.result property.
* If ODS redefines this, it is fine, but I have found this accessor to be universally useful in the old npcomp bindings and I'm closing gaps that will let me switch.

Differential Revision: https://reviews.llvm.org/D92287
2020-11-29 18:09:07 -08:00
Stella Laurenzo bd2083c2fa [mlir][Python] Python API cleanups and additions found during code audit.
* Add capsule get/create for Attribute and Type, which already had capsule interop defined.
* Add capsule interop and get/create for Location.
* Add Location __eq__.
* Use get() and implicit cast to go from PyAttribute, PyType, PyLocation to MlirAttribute, MlirType, MlirLocation (bundled with this change because I didn't want to continue the pattern one more time).

Differential Revision: https://reviews.llvm.org/D92283
2020-11-29 18:09:07 -08:00
Alex Zinenko ee6255d207 [mlir] move lib/Bindings/Python/Attributes.td to include/mlir/Bindings/Python
This file is intended to be included by other files, including
out-of-tree dialects, and makes more sense in `include` than in `lib`.

Depends On D91652

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D91961
2020-11-24 09:19:01 +01:00
Alex Zinenko 029e199dbf [mlir] Make attributes mutable in Python bindings
Attributes represent additional data about an operation and are intended to be
modifiable during the lifetime of the operation. In the dialect-specific Python
bindings, attributes are exposed as properties on the operation class. Allow
for assigning values to these properties. Also support creating new and
deleting existing attributes through the generic "attributes" property of an
operation. Any validity checking must be performed by the op verifier after the
mutation, similarly to C++. Operations are not invalidated in the process: no
dangling pointers can be created as all attributes are owned by the context and
will remain live even if they are not used in any operation.

Introduce a Python Test dialect by analogy with the Test dialect and to avoid
polluting the latter with Python-specific constructs. Use this dialect to
implement a test for the attribute access and mutation API.

Reviewed By: stellaraccident, mehdi_amini

Differential Revision: https://reviews.llvm.org/D91652
2020-11-24 09:16:25 +01:00
zhanghb97 77133b29b9 [mlir] Get array from the dense elements attribute with buffer protocol.
- Add `mlirElementsAttrGetType` C API.
- Add `def_buffer` binding to PyDenseElementsAttribute.
- Implement the protocol to access the buffer.

Differential Revision: https://reviews.llvm.org/D91021
2020-11-18 15:50:59 +08:00
Alex Zinenko ef8e859c0b [mlir] Fix Python tests after "module_terminator" migrated to ODS
The "module_terminator" op now has a custom syntax and therefore is
printed without quotes. Adapt Python tests to check for this syntax.
2020-11-17 14:16:31 +01:00
Mehdi Amini 74207e78cf Fix python bindings tests after change in visibility requirement for symbol declarations 2020-11-17 04:09:35 +00:00
Alex Zinenko f9265de8c6 [mlir] Generate Op builders for Python bindings
Add an ODS-backed generator of default builders. This currently does not
support operation with attribute arguments, for which the builder is
just ignored. Attribute support will be introduced separately for
builders and accessors.

Default builders are always generated with the same number of result and
operand groups as the ODS specification, i.e. one group per each operand
or result. Optional elements accept None but cannot be omitted. Variadic
groups accept iterable objects and cannot be replaced with a single
object.

For some operations, it is possible to infer the result type given the
traits, but most traits rely on inline pieces of C++ that we cannot
(yet) forward to Python bindings. Since the Ops where the inference is
possible (having the `SameOperandAndResultTypes` trait or
`TypeMatchesWith` without transform field) are a small minority, they
also require the result type to make the builder syntax more consistent.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D91190
2020-11-12 11:29:23 +01:00
Mehdi Amini a62d38a90d Disable implicit nesting on parsing textual pass pipeline
Previous the textual form of the pass pipeline would implicitly nest,
instead we opt for the explicit form here: this has less surprise.

This also avoids asserting in the bindings when passing a pass pipeline
with incorrect nesting.

Differential Revision: https://reviews.llvm.org/D91233
2020-11-11 19:21:51 +00:00
Stella Laurenzo 5fef6ce0cc [mlir][Python] Allow PassManager to interop with the capsule APIs.
* Used in npcomp to cast Python objects via the C-API.

Differential Revision: https://reviews.llvm.org/D91232
2020-11-11 10:37:21 -08:00
Mehdi Amini 6cb1c0cae0 Add Python binding to run a PassManager on a MLIR Module
Reviewed By: ftynse, stellaraccident

Differential Revision: https://reviews.llvm.org/D90823
2020-11-10 20:06:23 +00:00
Mehdi Amini dc43f78565 Add basic Python bindings for the PassManager and bind libTransforms
This only exposes the ability to round-trip a textual pipeline at the
moment.
To exercise it, we also bind the libTransforms in a new Python extension. This
does not include any interesting bindings, but it includes all the
mechanism to add separate native extensions and load them dynamically.
As such passes in libTransforms are only registered after `import
mlir.transforms`.
To support this global registration, the TableGen backend is also
extended to bind to the C API the group registration for passes.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D90819
2020-11-10 19:55:21 +00:00
Alex Zinenko fd407e1f1e [mlir] ODS-backed python binding generator for custom op classes
Introduce an ODS/Tablegen backend producing Op wrappers for Python bindings
based on the ODS operation definition. Usage:

  mlir-tblgen -gen-python-op-bindings -Iinclude <path/to/Ops.td> \
              -bind-dialect=<dialect-name>

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D90960
2020-11-10 10:58:29 +01:00
Alex Zinenko 6c7e6b2c9a [mlir] Support slicing for operands in results in Python bindings
Slicing, that is element access with `[being🔚step]` structure, is
a common Python idiom for sequence-like containers. It is also necessary
to support custom accessor for operations with variadic operands and
results (an operation an return a slice of its operands that correspond
to the given variadic group).

Add generic utility to support slicing in Python bindings and use it
for operation operands and results.

Depends On D90923

Reviewed By: stellaraccident, mehdi_amini

Differential Revision: https://reviews.llvm.org/D90936
2020-11-10 10:46:21 +01:00