forked from OSchip/llvm-project
[mlir:Pass] Add support for op-agnostic pass managers
This commit refactors the current pass manager support to allow for operation agnostic pass managers. This allows for a series of passes to be executed on any viable pass manager root operation, instead of one specific operation type. Op-agnostic/generic pass managers only allow for adding op-agnostic passes. These types of pass managers are extremely useful when constructing pass pipelines that can apply to many different types of operations, e.g., the default inliner simplification pipeline. With the advent of interface/trait passes, this support can be used to define FunctionOpInterface pass managers, or other pass managers that effectively operate on specific interfaces/traits/etc (see #52916 for an example). Differential Revision: https://reviews.llvm.org/D123536
This commit is contained in:
River Riddle
parent
6a22b185d6
commit
c2fb9c29b4
|
|
@ -24,9 +24,9 @@ following restrictions; any noncompliance will lead to problematic behavior in
|
|||
multithreaded and other advanced scenarios:
|
||||
|
||||
* Must not modify any state referenced or relied upon outside the current
|
||||
being operated on. This includes adding or removing operations from the
|
||||
parent block, changing the attributes(depending on the contract of the
|
||||
current operation)/operands/results/successors of the current operation.
|
||||
operation being operated on. This includes adding or removing operations
|
||||
from the parent block, changing the attributes(depending on the contract
|
||||
of the current operation)/operands/results/successors of the current operation.
|
||||
* Must not modify the state of another operation not nested within the current
|
||||
operation being operated on.
|
||||
* Other threads may be operating on these operations simultaneously.
|
||||
|
|
@ -46,62 +46,16 @@ multithreaded and other advanced scenarios:
|
|||
* Multiple instances of the pass may be created by the pass manager to
|
||||
process operations in parallel.
|
||||
|
||||
When creating an operation pass, there are two different types to choose from
|
||||
depending on the usage scenario:
|
||||
### Op-Agnostic Operation Passes
|
||||
|
||||
### OperationPass : Op-Specific
|
||||
By default, an operation pass is `op-agnostic`, meaning that it operates on the
|
||||
operation type of the pass manager that it is added to. This means a pass may operate
|
||||
on many different types of operations. Agnostic passes should be written such that
|
||||
they do not make assumptions on the operation they run on. Examples of this type of pass are
|
||||
[Canonicalization](Pass.md/-canonicalize-canonicalize-operations)
|
||||
[Common Sub-Expression Elimination](Passes.md/#-cse-eliminate-common-sub-expressions).
|
||||
|
||||
An `op-specific` operation pass operates explicitly on a given operation type.
|
||||
This operation type must adhere to the restrictions set by the pass manager for
|
||||
pass execution.
|
||||
|
||||
To define an op-specific operation pass, a derived class must adhere to the
|
||||
following:
|
||||
|
||||
* Inherit from the CRTP class `OperationPass` and provide the operation type
|
||||
as an additional template parameter.
|
||||
* Override the virtual `void runOnOperation()` method.
|
||||
|
||||
A simple pass may look like:
|
||||
|
||||
```c++
|
||||
namespace {
|
||||
/// Here we utilize the CRTP `PassWrapper` utility class to provide some
|
||||
/// necessary utility hooks. This is only necessary for passes defined directly
|
||||
/// in C++. Passes defined declaratively use a cleaner mechanism for providing
|
||||
/// these utilities.
|
||||
struct MyFunctionPass : public PassWrapper<MyFunctionPass,
|
||||
OperationPass<func::FuncOp>> {
|
||||
void runOnOperation() override {
|
||||
// Get the current func::FuncOp operation being operated on.
|
||||
func::FuncOp f = getOperation();
|
||||
|
||||
// Walk the operations within the function.
|
||||
f.walk([](Operation *inst) {
|
||||
....
|
||||
});
|
||||
}
|
||||
};
|
||||
} // namespace
|
||||
|
||||
/// Register this pass so that it can be built via from a textual pass pipeline.
|
||||
/// (Pass registration is discussed more below)
|
||||
void registerMyPass() {
|
||||
PassRegistration<MyFunctionPass>();
|
||||
}
|
||||
```
|
||||
|
||||
### OperationPass : Op-Agnostic
|
||||
|
||||
An `op-agnostic` pass operates on the operation type of the pass manager that it
|
||||
is added to. This means that passes of this type may operate on several
|
||||
different operation types. Passes of this type are generally written generically
|
||||
using operation [interfaces](Interfaces.md) and [traits](Traits.md). Examples of
|
||||
this type of pass are
|
||||
[Common Sub-Expression Elimination](Passes.md/#-cse-eliminate-common-sub-expressions)
|
||||
and [Inlining](Passes.md/#-inline-inline-function-calls).
|
||||
|
||||
To create an operation pass, a derived class must adhere to the following:
|
||||
To create an agnostic operation pass, a derived class must adhere to the following:
|
||||
|
||||
* Inherit from the CRTP class `OperationPass`.
|
||||
* Override the virtual `void runOnOperation()` method.
|
||||
|
|
@ -122,6 +76,108 @@ struct MyOperationPass : public PassWrapper<MyOperationPass, OperationPass<>> {
|
|||
};
|
||||
```
|
||||
|
||||
### Filtered Operation Pass
|
||||
|
||||
If a pass needs to constrain its execution to specific types or classes of operations,
|
||||
additional filtering may be applied on top. This transforms a once `agnostic` pass into
|
||||
one more specific to a certain context. There are various ways in which to filter the
|
||||
execution of a pass, and different contexts in which filtering may apply:
|
||||
|
||||
### Operation Pass: Static Schedule Filtering
|
||||
|
||||
Static filtering allows for applying additional constraints on the operation types a
|
||||
pass may be scheduled on. This type of filtering generally allows for building more
|
||||
constrained passes that can only be scheduled on operations that satisfy the necessary
|
||||
constraints. For example, this allows for specifying passes that only run on operations
|
||||
of a certain, those that provide a certain interface, trait, or some other constraint that
|
||||
applies to all instances of that operation type. Below is an example of a pass that only
|
||||
permits scheduling on operations that implement `FunctionOpInterface`:
|
||||
|
||||
```c++
|
||||
struct MyFunctionPass : ... {
|
||||
/// This method is used to provide additional static filtering, and returns if the
|
||||
/// pass may be scheduled on the given operation type.
|
||||
bool canScheduleOn(RegisteredOperationName opInfo) const override {
|
||||
return opInfo.hasInterface<FunctionOpInterface>();
|
||||
}
|
||||
|
||||
void runOnOperation() {
|
||||
// Here we can freely cast to FunctionOpInterface, because our `canScheduleOn` ensures
|
||||
// that our pass is only executed on operations implementing that interface.
|
||||
FunctionOpInterface op = cast<FunctionOpInterface>(getOperation());
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
When a pass with static filtering is added to an [`op-specific` pass manager](#oppassmanager),
|
||||
it asserts that the operation type of the pass manager satisfies the static constraints of the
|
||||
pass. When added to an [`op-agnostic` pass manager](#oppassmanager), that pass manager, and all
|
||||
passes contained within, inherits the static constraints of the pass. For example, if the pass
|
||||
filters on `FunctionOpInterface`, as in the `MyFunctionPass` example above, only operations that
|
||||
implement `FunctionOpInterface` will be considered when executing **any** passes within the pass
|
||||
manager. This invariant is important to keep in mind, as each pass added to an `op-agnostic` pass
|
||||
manager further constrains the operations that may be scheduled on it. Consider the following example:
|
||||
|
||||
```mlir
|
||||
func.func @foo() {
|
||||
// ...
|
||||
return
|
||||
}
|
||||
|
||||
module @someModule {
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
If we were to apply the op-agnostic pipeline, `any(cse,my-function-pass)`, to the above MLIR snippet
|
||||
it would only run on the `foo` function operation. This is because the `my-function-pass` has a
|
||||
static filtering constraint to only schedule on operations implementing `FunctionOpInterface`. Remember
|
||||
that this constraint is inherited by the entire pass manager, so we never consider `someModule` for
|
||||
any of the passes, including `cse` which normally can be scheduled on any operation.
|
||||
|
||||
#### Operation Pass: Static Filtering By Op Type
|
||||
|
||||
In the above section, we detailed a general mechanism for statically filtering the types of operations
|
||||
that a pass may be scheduled on. Sugar is provided on top of that mechanism to simplify the definition
|
||||
of passes that are restricted to scheduling on a single operation type. In these cases, a pass simply
|
||||
needs to provide the type of operation to the `OperationPass` base class. This will automatically
|
||||
instill filtering on that operation type:
|
||||
|
||||
```c++
|
||||
/// Here we utilize the CRTP `PassWrapper` utility class to provide some
|
||||
/// necessary utility hooks. This is only necessary for passes defined directly
|
||||
/// in C++. Passes defined declaratively use a cleaner mechanism for providing
|
||||
/// these utilities.
|
||||
struct MyFunctionPass : public PassWrapper<MyOperationPass, OperationPass<func::FuncOp>> {
|
||||
void runOnOperation() {
|
||||
// Get the current operation being operated on.
|
||||
func::FuncOp op = getOperation();
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
#### Operation Pass: Static Filtering By Interface
|
||||
|
||||
In the above section, we detailed a general mechanism for statically filtering the types of operations
|
||||
that a pass may be scheduled on. Sugar is provided on top of that mechanism to simplify the definition
|
||||
of passes that are restricted to scheduling on a specific operation interface. In these cases, a pass
|
||||
simply needs to inherit from the `InterfacePass` base class. This class is similar to `OperationPass`,
|
||||
but expects the type of interface to operate on. This will automatically instill filtering on that
|
||||
interface type:
|
||||
|
||||
```c++
|
||||
/// Here we utilize the CRTP `PassWrapper` utility class to provide some
|
||||
/// necessary utility hooks. This is only necessary for passes defined directly
|
||||
/// in C++. Passes defined declaratively use a cleaner mechanism for providing
|
||||
/// these utilities.
|
||||
struct MyFunctionPass : public PassWrapper<MyOperationPass, InterfacePass<FunctionOpInterface>> {
|
||||
void runOnOperation() {
|
||||
// Get the current operation being operated on.
|
||||
FunctionOpInterface op = getOperation();
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
### Dependent Dialects
|
||||
|
||||
Dialects must be loaded in the MLIRContext before entities from these dialects
|
||||
|
|
@ -293,27 +349,28 @@ used to schedule passes to run at a specific level of nesting. The top-level
|
|||
|
||||
### OpPassManager
|
||||
|
||||
An `OpPassManager` is essentially a collection of passes to execute on an
|
||||
operation of a specific type. This operation type must adhere to the following
|
||||
requirement:
|
||||
An `OpPassManager` is essentially a collection of passes anchored to execute on
|
||||
operations at a given level of nesting. A pass manager may be `op-specific`
|
||||
(anchored on a specific operation type), or `op-agnostic` (not restricted to any
|
||||
specific operation, and executed on any viable operation type). Operation types that
|
||||
anchor pass managers must adhere to the following requirement:
|
||||
|
||||
* Must be registered and marked
|
||||
[`IsolatedFromAbove`](Traits.md/#isolatedfromabove).
|
||||
|
||||
* Passes are expected to not modify operations at or above the current
|
||||
* Passes are expected not to modify operations at or above the current
|
||||
operation being processed. If the operation is not isolated, it may
|
||||
inadvertently modify or traverse the SSA use-list of an operation it is
|
||||
not supposed to.
|
||||
|
||||
Passes can be added to a pass manager via `addPass`. The pass must either be an
|
||||
`op-specific` pass operating on the same operation type as `OpPassManager`, or
|
||||
an `op-agnostic` pass.
|
||||
Passes can be added to a pass manager via `addPass`.
|
||||
|
||||
An `OpPassManager` is generally created by explicitly nesting a pipeline within
|
||||
another existing `OpPassManager` via the `nest<>` method. This method takes the
|
||||
operation type that the nested pass manager will operate on. At the top-level, a
|
||||
`PassManager` acts as an `OpPassManager`. Nesting in this sense, corresponds to
|
||||
the [structural](Tutorials/UnderstandingTheIRStructure.md) nesting within
|
||||
another existing `OpPassManager` via the `nest<OpT>` or `nestAny` methods. The
|
||||
former method takes the operation type that the nested pass manager will operate on.
|
||||
The latter method nests an `op-agnostic` pass manager, that may run on any viable
|
||||
operation type. Nesting in this sense, corresponds to the
|
||||
[structural](Tutorials/UnderstandingTheIRStructure.md) nesting within
|
||||
[Regions](LangRef.md/#regions) of the IR.
|
||||
|
||||
For example, the following `.mlir`:
|
||||
|
|
@ -331,9 +388,9 @@ module {
|
|||
Has the nesting structure of:
|
||||
|
||||
```
|
||||
`module`
|
||||
`builtin.module`
|
||||
`spv.module`
|
||||
`function`
|
||||
`spv.func`
|
||||
```
|
||||
|
||||
Below is an example of constructing a pipeline that operates on the above
|
||||
|
|
@ -359,6 +416,12 @@ nestedModulePM.addPass(std::make_unique<MySPIRVModulePass>());
|
|||
OpPassManager &nestedFunctionPM = nestedModulePM.nest<func::FuncOp>();
|
||||
nestedFunctionPM.addPass(std::make_unique<MyFunctionPass>());
|
||||
|
||||
// Nest an op-agnostic pass manager. This will operate on any viable
|
||||
// operation, e.g. func.func, spv.func, spv.module, builtin.module, etc.
|
||||
OpPassManager &nestedAnyPM = nestedModulePM.nestAny();
|
||||
nestedFunctionPM.addPass(createCanonicalizePass());
|
||||
nestedFunctionPM.addPass(createCSEPass());
|
||||
|
||||
// Run the pass manager on the top-level module.
|
||||
ModuleOp m = ...;
|
||||
if (failed(pm.run(m)))
|
||||
|
|
@ -374,6 +437,9 @@ OpPassManager<ModuleOp>
|
|||
MySPIRVModulePass
|
||||
OpPassManager<func::FuncOp>
|
||||
MyFunctionPass
|
||||
OpPassManager<>
|
||||
Canonicalizer
|
||||
CSE
|
||||
```
|
||||
|
||||
These pipelines are then run over a single operation at a time. This means that,
|
||||
|
|
@ -652,14 +718,17 @@ defined as a series of names, each of which may in itself recursively contain a
|
|||
nested pipeline description. The syntax for this specification is as follows:
|
||||
|
||||
```ebnf
|
||||
pipeline ::= op-name `(` pipeline-element (`,` pipeline-element)* `)`
|
||||
pipeline ::= op-anchor `(` pipeline-element (`,` pipeline-element)* `)`
|
||||
pipeline-element ::= pipeline | (pass-name | pass-pipeline-name) options?
|
||||
options ::= '{' (key ('=' value)?)+ '}'
|
||||
```
|
||||
|
||||
* `op-name`
|
||||
* This corresponds to the mnemonic name of an operation to run passes on,
|
||||
e.g. `func.func` or `builtin.module`.
|
||||
* `op-anchor`
|
||||
* This corresponds to the mnemonic name that anchors the execution of the
|
||||
pass manager. This is either the name of an operation to run passes on,
|
||||
e.g. `func.func` or `builtin.module`, or `any`, for op-agnostic pass
|
||||
managers that execute on any viable operation (i.e. any operation that
|
||||
can be used to anchor a pass manager).
|
||||
* `pass-name` | `pass-pipeline-name`
|
||||
* This corresponds to the argument of a registered pass or pass pipeline,
|
||||
e.g. `cse` or `canonicalize`.
|
||||
|
|
@ -678,7 +747,11 @@ $ mlir-opt foo.mlir -cse -canonicalize -convert-func-to-llvm='use-bare-ptr-memre
|
|||
Can also be specified as (via the `-pass-pipeline` flag):
|
||||
|
||||
```shell
|
||||
# Anchor the cse and canonicalize passes on the `func.func` operation.
|
||||
$ mlir-opt foo.mlir -pass-pipeline='func.func(cse,canonicalize),convert-func-to-llvm{use-bare-ptr-memref-call-conv=1}'
|
||||
|
||||
# Anchor the cse and canonicalize passes on "any" viable root operation.
|
||||
$ mlir-opt foo.mlir -pass-pipeline='any(cse,canonicalize),convert-func-to-llvm{use-bare-ptr-memref-call-conv=1}'
|
||||
```
|
||||
|
||||
In order to support round-tripping a pass to the textual representation using
|
||||
|
|
|
|||
|
|
@ -9,11 +9,11 @@
|
|||
#ifndef MLIR_PASS_PASSINSTRUMENTATION_H_
|
||||
#define MLIR_PASS_PASSINSTRUMENTATION_H_
|
||||
|
||||
#include "mlir/IR/BuiltinAttributes.h"
|
||||
#include "mlir/Support/LLVM.h"
|
||||
#include "mlir/Support/TypeID.h"
|
||||
|
||||
namespace mlir {
|
||||
class OperationName;
|
||||
class Operation;
|
||||
class Pass;
|
||||
|
||||
|
|
@ -41,16 +41,18 @@ public:
|
|||
virtual ~PassInstrumentation() = 0;
|
||||
|
||||
/// A callback to run before a pass pipeline is executed. This function takes
|
||||
/// the name of the operation type being operated on, and information related
|
||||
/// to the parent that spawned this pipeline.
|
||||
virtual void runBeforePipeline(StringAttr name,
|
||||
const PipelineParentInfo &parentInfo) {}
|
||||
/// the name of the operation type being operated on, or None if the pipeline
|
||||
/// is op-agnostic, and information related to the parent that spawned this
|
||||
/// pipeline.
|
||||
virtual void runBeforePipeline(Optional<OperationName> name,
|
||||
const PipelineParentInfo &parentInfo);
|
||||
|
||||
/// A callback to run after a pass pipeline has executed. This function takes
|
||||
/// the name of the operation type being operated on, and information related
|
||||
/// to the parent that spawned this pipeline.
|
||||
virtual void runAfterPipeline(StringAttr name,
|
||||
const PipelineParentInfo &parentInfo) {}
|
||||
/// the name of the operation type being operated on, or None if the pipeline
|
||||
/// is op-agnostic, and information related to the parent that spawned this
|
||||
/// pipeline.
|
||||
virtual void runAfterPipeline(Optional<OperationName> name,
|
||||
const PipelineParentInfo &parentInfo);
|
||||
|
||||
/// A callback to run before a pass is executed. This function takes a pointer
|
||||
/// to the pass to be executed, as well as the current operation being
|
||||
|
|
@ -90,12 +92,12 @@ public:
|
|||
|
||||
/// See PassInstrumentation::runBeforePipeline for details.
|
||||
void
|
||||
runBeforePipeline(StringAttr name,
|
||||
runBeforePipeline(Optional<OperationName> name,
|
||||
const PassInstrumentation::PipelineParentInfo &parentInfo);
|
||||
|
||||
/// See PassInstrumentation::runAfterPipeline for details.
|
||||
void
|
||||
runAfterPipeline(StringAttr name,
|
||||
runAfterPipeline(Optional<OperationName> name,
|
||||
const PassInstrumentation::PipelineParentInfo &parentInfo);
|
||||
|
||||
/// See PassInstrumentation::runBeforePass for details.
|
||||
|
|
|
|||
|
|
@ -32,7 +32,6 @@ class Operation;
|
|||
class Pass;
|
||||
class PassInstrumentation;
|
||||
class PassInstrumentor;
|
||||
class StringAttr;
|
||||
|
||||
namespace detail {
|
||||
struct OpPassManagerImpl;
|
||||
|
|
@ -45,14 +44,33 @@ struct PassExecutionState;
|
|||
// OpPassManager
|
||||
//===----------------------------------------------------------------------===//
|
||||
|
||||
/// This class represents a pass manager that runs passes on a specific
|
||||
/// operation type. This class is not constructed directly, but nested within
|
||||
/// other OpPassManagers or the top-level PassManager.
|
||||
/// This class represents a pass manager that runs passes on either a specific
|
||||
/// operation type, or any isolated operation. This pass manager can not be run
|
||||
/// on an operation directly, but must be run either as part of a top-level
|
||||
/// `PassManager`(e.g. when constructed via `nest` calls), or dynamically within
|
||||
/// a pass by using the `Pass::runPipeline` API.
|
||||
class OpPassManager {
|
||||
public:
|
||||
enum class Nesting { Implicit, Explicit };
|
||||
OpPassManager(StringAttr name, Nesting nesting = Nesting::Explicit);
|
||||
/// This enum represents the nesting behavior of the pass manager.
|
||||
enum class Nesting {
|
||||
/// Implicit nesting behavior. This allows for adding passes operating on
|
||||
/// operations different from this pass manager, in which case a new pass
|
||||
/// manager is implicitly nested for the operation type of the new pass.
|
||||
Implicit,
|
||||
/// Explicit nesting behavior. This requires that any passes added to this
|
||||
/// pass manager support its operation type.
|
||||
Explicit
|
||||
};
|
||||
|
||||
/// Construct a new op-agnostic ("any") pass manager with the given operation
|
||||
/// type and nesting behavior. This is the same as invoking:
|
||||
/// `OpPassManager(getAnyOpAnchorName(), nesting)`.
|
||||
OpPassManager(Nesting nesting = Nesting::Explicit);
|
||||
|
||||
/// Construct a new pass manager with the given anchor operation type and
|
||||
/// nesting behavior.
|
||||
OpPassManager(StringRef name, Nesting nesting = Nesting::Explicit);
|
||||
OpPassManager(OperationName name, Nesting nesting = Nesting::Explicit);
|
||||
OpPassManager(OpPassManager &&rhs);
|
||||
OpPassManager(const OpPassManager &rhs);
|
||||
~OpPassManager();
|
||||
|
|
@ -78,12 +96,16 @@ public:
|
|||
|
||||
/// Nest a new operation pass manager for the given operation kind under this
|
||||
/// pass manager.
|
||||
OpPassManager &nest(StringAttr nestedName);
|
||||
OpPassManager &nest(OperationName nestedName);
|
||||
OpPassManager &nest(StringRef nestedName);
|
||||
template <typename OpT> OpPassManager &nest() {
|
||||
return nest(OpT::getOperationName());
|
||||
}
|
||||
|
||||
/// Nest a new op-agnostic ("any") pass manager under this pass manager.
|
||||
/// Note: This is the same as invoking `nest(getAnyOpAnchorName())`.
|
||||
OpPassManager &nestAny();
|
||||
|
||||
/// Add the given pass to this pass manager. If this pass has a concrete
|
||||
/// operation type, it must be the same type as this pass manager.
|
||||
void addPass(std::unique_ptr<Pass> pass);
|
||||
|
|
@ -100,11 +122,22 @@ public:
|
|||
/// Returns the number of passes held by this manager.
|
||||
size_t size() const;
|
||||
|
||||
/// Return the operation name that this pass manager operates on.
|
||||
OperationName getOpName(MLIRContext &context) const;
|
||||
/// Return the operation name that this pass manager operates on, or None if
|
||||
/// this is an op-agnostic pass manager.
|
||||
Optional<OperationName> getOpName(MLIRContext &context) const;
|
||||
|
||||
/// Return the operation name that this pass manager operates on.
|
||||
StringRef getOpName() const;
|
||||
/// Return the operation name that this pass manager operates on, or None if
|
||||
/// this is an op-agnostic pass manager.
|
||||
Optional<StringRef> getOpName() const;
|
||||
|
||||
/// Return the name used to anchor this pass manager. This is either the name
|
||||
/// of an operation, or the result of `getAnyOpAnchorName()` in the case of an
|
||||
/// op-agnostic pass manager.
|
||||
StringRef getOpAnchorName() const;
|
||||
|
||||
/// Return the string name used to anchor op-agnostic pass managers that
|
||||
/// operate generically on any viable operation.
|
||||
static StringRef getAnyOpAnchorName() { return "any"; }
|
||||
|
||||
/// Returns the internal implementation instance.
|
||||
detail::OpPassManagerImpl &getImpl();
|
||||
|
|
@ -177,6 +210,8 @@ public:
|
|||
/// Create a new pass manager under the given context with a specific nesting
|
||||
/// style. The created pass manager can schedule operations that match
|
||||
/// `operationName`.
|
||||
/// FIXME: We should make the specification of `builtin.module` explicit here,
|
||||
/// so that we can have top-level op-agnostic pass managers.
|
||||
PassManager(MLIRContext *ctx, Nesting nesting = Nesting::Explicit,
|
||||
StringRef operationName = "builtin.module");
|
||||
PassManager(MLIRContext *ctx, StringRef operationName)
|
||||
|
|
|
|||
|
|
@ -58,7 +58,7 @@ void Pass::printAsTextualPipeline(raw_ostream &os) {
|
|||
llvm::interleave(
|
||||
adaptor->getPassManagers(),
|
||||
[&](OpPassManager &pm) {
|
||||
os << pm.getOpName() << "(";
|
||||
os << pm.getOpAnchorName() << "(";
|
||||
pm.printAsTextualPipeline(os);
|
||||
os << ")";
|
||||
},
|
||||
|
|
@ -84,18 +84,39 @@ namespace mlir {
|
|||
namespace detail {
|
||||
struct OpPassManagerImpl {
|
||||
OpPassManagerImpl(OperationName opName, OpPassManager::Nesting nesting)
|
||||
: name(opName.getStringRef()), opName(opName),
|
||||
: name(opName.getStringRef().str()), opName(opName),
|
||||
initializationGeneration(0), nesting(nesting) {}
|
||||
OpPassManagerImpl(StringRef name, OpPassManager::Nesting nesting)
|
||||
: name(name), initializationGeneration(0), nesting(nesting) {}
|
||||
: name(name == OpPassManager::getAnyOpAnchorName() ? "" : name.str()),
|
||||
initializationGeneration(0), nesting(nesting) {}
|
||||
OpPassManagerImpl(OpPassManager::Nesting nesting)
|
||||
: name(""), initializationGeneration(0), nesting(nesting) {}
|
||||
OpPassManagerImpl(const OpPassManagerImpl &rhs)
|
||||
: name(rhs.name), opName(rhs.opName),
|
||||
initializationGeneration(rhs.initializationGeneration),
|
||||
nesting(rhs.nesting) {
|
||||
for (const std::unique_ptr<Pass> &pass : rhs.passes) {
|
||||
std::unique_ptr<Pass> newPass = pass->clone();
|
||||
newPass->threadingSibling = pass.get();
|
||||
passes.push_back(std::move(newPass));
|
||||
}
|
||||
}
|
||||
|
||||
/// Merge the passes of this pass manager into the one provided.
|
||||
void mergeInto(OpPassManagerImpl &rhs);
|
||||
|
||||
/// Nest a new operation pass manager for the given operation kind under this
|
||||
/// pass manager.
|
||||
OpPassManager &nest(StringAttr nestedName);
|
||||
OpPassManager &nest(StringRef nestedName);
|
||||
OpPassManager &nest(OperationName nestedName) {
|
||||
return nest(OpPassManager(nestedName, nesting));
|
||||
}
|
||||
OpPassManager &nest(StringRef nestedName) {
|
||||
return nest(OpPassManager(nestedName, nesting));
|
||||
}
|
||||
OpPassManager &nestAny() { return nest(OpPassManager(nesting)); }
|
||||
|
||||
/// Nest the given pass manager under this pass manager.
|
||||
OpPassManager &nest(OpPassManager &&nested);
|
||||
|
||||
/// Add the given pass to this pass manager. If this pass has a concrete
|
||||
/// operation type, it must be the same type as this pass manager.
|
||||
|
|
@ -111,11 +132,25 @@ struct OpPassManagerImpl {
|
|||
LogicalResult finalizePassList(MLIRContext *ctx);
|
||||
|
||||
/// Return the operation name of this pass manager.
|
||||
OperationName getOpName(MLIRContext &context) {
|
||||
if (!opName)
|
||||
Optional<OperationName> getOpName(MLIRContext &context) {
|
||||
if (!name.empty() && !opName)
|
||||
opName = OperationName(name, &context);
|
||||
return *opName;
|
||||
return opName;
|
||||
}
|
||||
Optional<StringRef> getOpName() const {
|
||||
return name.empty() ? Optional<StringRef>() : Optional<StringRef>(name);
|
||||
}
|
||||
|
||||
/// Return the name used to anchor this pass manager. This is either the name
|
||||
/// of an operation, or the result of `getAnyOpAnchorName()` in the case of an
|
||||
/// op-agnostic pass manager.
|
||||
StringRef getOpAnchorName() const {
|
||||
return getOpName().getValueOr(OpPassManager::getAnyOpAnchorName());
|
||||
}
|
||||
|
||||
/// Indicate if the current pass manager can be scheduled on the given
|
||||
/// operation type.
|
||||
bool canScheduleOn(MLIRContext &context, OperationName opName);
|
||||
|
||||
/// The name of the operation that passes of this pass manager operate on.
|
||||
std::string name;
|
||||
|
|
@ -145,15 +180,7 @@ void OpPassManagerImpl::mergeInto(OpPassManagerImpl &rhs) {
|
|||
passes.clear();
|
||||
}
|
||||
|
||||
OpPassManager &OpPassManagerImpl::nest(StringAttr nestedName) {
|
||||
OpPassManager nested(nestedName, nesting);
|
||||
auto *adaptor = new OpToOpPassAdaptor(std::move(nested));
|
||||
addPass(std::unique_ptr<Pass>(adaptor));
|
||||
return adaptor->getPassManagers().front();
|
||||
}
|
||||
|
||||
OpPassManager &OpPassManagerImpl::nest(StringRef nestedName) {
|
||||
OpPassManager nested(nestedName, nesting);
|
||||
OpPassManager &OpPassManagerImpl::nest(OpPassManager &&nested) {
|
||||
auto *adaptor = new OpToOpPassAdaptor(std::move(nested));
|
||||
addPass(std::unique_ptr<Pass>(adaptor));
|
||||
return adaptor->getPassManagers().front();
|
||||
|
|
@ -162,14 +189,15 @@ OpPassManager &OpPassManagerImpl::nest(StringRef nestedName) {
|
|||
void OpPassManagerImpl::addPass(std::unique_ptr<Pass> pass) {
|
||||
// If this pass runs on a different operation than this pass manager, then
|
||||
// implicitly nest a pass manager for this operation if enabled.
|
||||
auto passOpName = pass->getOpName();
|
||||
if (passOpName && passOpName->str() != name) {
|
||||
Optional<StringRef> pmOpName = getOpName();
|
||||
Optional<StringRef> passOpName = pass->getOpName();
|
||||
if (pmOpName && passOpName && *pmOpName != *passOpName) {
|
||||
if (nesting == OpPassManager::Nesting::Implicit)
|
||||
return nest(*passOpName).addPass(std::move(pass));
|
||||
llvm::report_fatal_error(llvm::Twine("Can't add pass '") + pass->getName() +
|
||||
"' restricted to '" + *passOpName +
|
||||
"' on a PassManager intended to run on '" + name +
|
||||
"', did you intend to nest?");
|
||||
"' on a PassManager intended to run on '" +
|
||||
getOpAnchorName() + "', did you intend to nest?");
|
||||
}
|
||||
|
||||
passes.emplace_back(std::move(pass));
|
||||
|
|
@ -178,6 +206,13 @@ void OpPassManagerImpl::addPass(std::unique_ptr<Pass> pass) {
|
|||
void OpPassManagerImpl::clear() { passes.clear(); }
|
||||
|
||||
LogicalResult OpPassManagerImpl::finalizePassList(MLIRContext *ctx) {
|
||||
auto finalizeAdaptor = [ctx](OpToOpPassAdaptor *adaptor) {
|
||||
for (auto &pm : adaptor->getPassManagers())
|
||||
if (failed(pm.getImpl().finalizePassList(ctx)))
|
||||
return failure();
|
||||
return success();
|
||||
};
|
||||
|
||||
// Walk the pass list and merge adjacent adaptors.
|
||||
OpToOpPassAdaptor *lastAdaptor = nullptr;
|
||||
for (auto &pass : passes) {
|
||||
|
|
@ -190,61 +225,80 @@ LogicalResult OpPassManagerImpl::finalizePassList(MLIRContext *ctx) {
|
|||
continue;
|
||||
}
|
||||
|
||||
// Otherwise, merge into the existing adaptor and delete the current one.
|
||||
currentAdaptor->mergeInto(*lastAdaptor);
|
||||
pass.reset();
|
||||
// Otherwise, try to merge into the existing adaptor and delete the
|
||||
// current one. If merging fails, just remember this as the last adaptor.
|
||||
if (succeeded(currentAdaptor->tryMergeInto(ctx, *lastAdaptor)))
|
||||
pass.reset();
|
||||
else
|
||||
lastAdaptor = currentAdaptor;
|
||||
} else if (lastAdaptor) {
|
||||
// If this pass is not an adaptor, then finalize and forget any existing
|
||||
// adaptor.
|
||||
for (auto &pm : lastAdaptor->getPassManagers())
|
||||
if (failed(pm.getImpl().finalizePassList(ctx)))
|
||||
return failure();
|
||||
// If this pass isn't an adaptor, finalize it and forget the last adaptor.
|
||||
if (failed(finalizeAdaptor(lastAdaptor)))
|
||||
return failure();
|
||||
lastAdaptor = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
// If there was an adaptor at the end of the manager, finalize it as well.
|
||||
if (lastAdaptor) {
|
||||
for (auto &pm : lastAdaptor->getPassManagers())
|
||||
if (failed(pm.getImpl().finalizePassList(ctx)))
|
||||
return failure();
|
||||
}
|
||||
if (lastAdaptor && failed(finalizeAdaptor(lastAdaptor)))
|
||||
return failure();
|
||||
|
||||
// Now that the adaptors have been merged, erase any empty slots corresponding
|
||||
// to the merged adaptors that were nulled-out in the loop above.
|
||||
Optional<RegisteredOperationName> opName =
|
||||
getOpName(*ctx).getRegisteredInfo();
|
||||
llvm::erase_if(passes, std::logical_not<std::unique_ptr<Pass>>());
|
||||
|
||||
// Verify that all of the passes are valid for the operation.
|
||||
// If this is a op-agnostic pass manager, there is nothing left to do.
|
||||
Optional<OperationName> rawOpName = getOpName(*ctx);
|
||||
if (!rawOpName)
|
||||
return success();
|
||||
|
||||
// Otherwise, verify that all of the passes are valid for the current
|
||||
// operation anchor.
|
||||
Optional<RegisteredOperationName> opName = rawOpName->getRegisteredInfo();
|
||||
for (std::unique_ptr<Pass> &pass : passes) {
|
||||
if (opName && !pass->canScheduleOn(*opName)) {
|
||||
return emitError(UnknownLoc::get(ctx))
|
||||
<< "unable to schedule pass '" << pass->getName()
|
||||
<< "' on a PassManager intended to run on '" << name << "'!";
|
||||
<< "' on a PassManager intended to run on '" << getOpAnchorName()
|
||||
<< "'!";
|
||||
}
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
bool OpPassManagerImpl::canScheduleOn(MLIRContext &context,
|
||||
OperationName opName) {
|
||||
// If this pass manager is op-specific, we simply check if the provided
|
||||
// operation name is the same as this one.
|
||||
Optional<OperationName> pmOpName = getOpName(context);
|
||||
if (pmOpName)
|
||||
return pmOpName == opName;
|
||||
|
||||
// Otherwise, this is an op-agnostic pass manager. Check that the operation
|
||||
// can be scheduled on all passes within the manager.
|
||||
Optional<RegisteredOperationName> registeredInfo = opName.getRegisteredInfo();
|
||||
if (!registeredInfo ||
|
||||
!registeredInfo->hasTrait<OpTrait::IsIsolatedFromAbove>())
|
||||
return false;
|
||||
return llvm::all_of(passes, [&](const std::unique_ptr<Pass> &pass) {
|
||||
return pass->canScheduleOn(*registeredInfo);
|
||||
});
|
||||
}
|
||||
|
||||
//===----------------------------------------------------------------------===//
|
||||
// OpPassManager
|
||||
//===----------------------------------------------------------------------===//
|
||||
|
||||
OpPassManager::OpPassManager(StringAttr name, Nesting nesting)
|
||||
: impl(new OpPassManagerImpl(name, nesting)) {}
|
||||
OpPassManager::OpPassManager(Nesting nesting)
|
||||
: impl(new OpPassManagerImpl(nesting)) {}
|
||||
OpPassManager::OpPassManager(StringRef name, Nesting nesting)
|
||||
: impl(new OpPassManagerImpl(name, nesting)) {}
|
||||
OpPassManager::OpPassManager(OperationName name, Nesting nesting)
|
||||
: impl(new OpPassManagerImpl(name, nesting)) {}
|
||||
OpPassManager::OpPassManager(OpPassManager &&rhs) : impl(std::move(rhs.impl)) {}
|
||||
OpPassManager::OpPassManager(const OpPassManager &rhs) { *this = rhs; }
|
||||
OpPassManager &OpPassManager::operator=(const OpPassManager &rhs) {
|
||||
impl = std::make_unique<OpPassManagerImpl>(rhs.impl->name, rhs.impl->nesting);
|
||||
impl->initializationGeneration = rhs.impl->initializationGeneration;
|
||||
for (auto &pass : rhs.impl->passes) {
|
||||
auto newPass = pass->clone();
|
||||
newPass->threadingSibling = pass.get();
|
||||
impl->passes.push_back(std::move(newPass));
|
||||
}
|
||||
impl = std::make_unique<OpPassManagerImpl>(*rhs.impl);
|
||||
return *this;
|
||||
}
|
||||
|
||||
|
|
@ -266,12 +320,13 @@ OpPassManager::const_pass_iterator OpPassManager::end() const {
|
|||
|
||||
/// Nest a new operation pass manager for the given operation kind under this
|
||||
/// pass manager.
|
||||
OpPassManager &OpPassManager::nest(StringAttr nestedName) {
|
||||
OpPassManager &OpPassManager::nest(OperationName nestedName) {
|
||||
return impl->nest(nestedName);
|
||||
}
|
||||
OpPassManager &OpPassManager::nest(StringRef nestedName) {
|
||||
return impl->nest(nestedName);
|
||||
}
|
||||
OpPassManager &OpPassManager::nestAny() { return impl->nestAny(); }
|
||||
|
||||
/// Add the given pass to this pass manager. If this pass has a concrete
|
||||
/// operation type, it must be the same type as this pass manager.
|
||||
|
|
@ -288,13 +343,19 @@ size_t OpPassManager::size() const { return impl->passes.size(); }
|
|||
OpPassManagerImpl &OpPassManager::getImpl() { return *impl; }
|
||||
|
||||
/// Return the operation name that this pass manager operates on.
|
||||
StringRef OpPassManager::getOpName() const { return impl->name; }
|
||||
Optional<StringRef> OpPassManager::getOpName() const {
|
||||
return impl->getOpName();
|
||||
}
|
||||
|
||||
/// Return the operation name that this pass manager operates on.
|
||||
OperationName OpPassManager::getOpName(MLIRContext &context) const {
|
||||
Optional<OperationName> OpPassManager::getOpName(MLIRContext &context) const {
|
||||
return impl->getOpName(context);
|
||||
}
|
||||
|
||||
StringRef OpPassManager::getOpAnchorName() const {
|
||||
return impl->getOpAnchorName();
|
||||
}
|
||||
|
||||
/// Prints out the given passes as the textual representation of a pipeline.
|
||||
static void printAsTextualPipeline(ArrayRef<std::unique_ptr<Pass>> passes,
|
||||
raw_ostream &os) {
|
||||
|
|
@ -361,10 +422,11 @@ LogicalResult OpPassManager::initialize(MLIRContext *context,
|
|||
LogicalResult OpToOpPassAdaptor::run(Pass *pass, Operation *op,
|
||||
AnalysisManager am, bool verifyPasses,
|
||||
unsigned parentInitGeneration) {
|
||||
if (!op->isRegistered())
|
||||
Optional<RegisteredOperationName> opInfo = op->getRegisteredInfo();
|
||||
if (!opInfo)
|
||||
return op->emitOpError()
|
||||
<< "trying to schedule a pass on an unregistered operation";
|
||||
if (!op->hasTrait<OpTrait::IsIsolatedFromAbove>())
|
||||
if (!opInfo->hasTrait<OpTrait::IsIsolatedFromAbove>())
|
||||
return op->emitOpError() << "trying to schedule a pass on an operation not "
|
||||
"marked as 'IsolatedFromAbove'";
|
||||
|
||||
|
|
@ -380,7 +442,8 @@ LogicalResult OpToOpPassAdaptor::run(Pass *pass, Operation *op,
|
|||
<< "Trying to schedule a dynamic pipeline on an "
|
||||
"operation that isn't "
|
||||
"nested under the current operation the pass is processing";
|
||||
assert(pipeline.getOpName() == root->getName().getStringRef());
|
||||
assert(
|
||||
pipeline.getImpl().canScheduleOn(*op->getContext(), root->getName()));
|
||||
|
||||
// Before running, finalize the passes held by the pipeline.
|
||||
if (failed(pipeline.getImpl().finalizePassList(root->getContext())))
|
||||
|
|
@ -390,7 +453,7 @@ LogicalResult OpToOpPassAdaptor::run(Pass *pass, Operation *op,
|
|||
if (failed(pipeline.initialize(root->getContext(), parentInitGeneration)))
|
||||
return failure();
|
||||
AnalysisManager nestedAm = root == op ? am : am.nest(root);
|
||||
return OpToOpPassAdaptor::runPipeline(pipeline.getPasses(), root, nestedAm,
|
||||
return OpToOpPassAdaptor::runPipeline(pipeline, root, nestedAm,
|
||||
verifyPasses, parentInitGeneration,
|
||||
pi, &parentInfo);
|
||||
};
|
||||
|
|
@ -448,9 +511,8 @@ LogicalResult OpToOpPassAdaptor::run(Pass *pass, Operation *op,
|
|||
|
||||
/// Run the given operation and analysis manager on a provided op pass manager.
|
||||
LogicalResult OpToOpPassAdaptor::runPipeline(
|
||||
iterator_range<OpPassManager::pass_iterator> passes, Operation *op,
|
||||
AnalysisManager am, bool verifyPasses, unsigned parentInitGeneration,
|
||||
PassInstrumentor *instrumentor,
|
||||
OpPassManager &pm, Operation *op, AnalysisManager am, bool verifyPasses,
|
||||
unsigned parentInitGeneration, PassInstrumentor *instrumentor,
|
||||
const PassInstrumentation::PipelineParentInfo *parentInfo) {
|
||||
assert((!instrumentor || parentInfo) &&
|
||||
"expected parent info if instrumentor is provided");
|
||||
|
|
@ -463,22 +525,28 @@ LogicalResult OpToOpPassAdaptor::runPipeline(
|
|||
});
|
||||
|
||||
// Run the pipeline over the provided operation.
|
||||
if (instrumentor)
|
||||
instrumentor->runBeforePipeline(op->getName().getIdentifier(), *parentInfo);
|
||||
for (Pass &pass : passes)
|
||||
if (instrumentor) {
|
||||
instrumentor->runBeforePipeline(pm.getOpName(*op->getContext()),
|
||||
*parentInfo);
|
||||
}
|
||||
|
||||
for (Pass &pass : pm.getPasses())
|
||||
if (failed(run(&pass, op, am, verifyPasses, parentInitGeneration)))
|
||||
return failure();
|
||||
if (instrumentor)
|
||||
instrumentor->runAfterPipeline(op->getName().getIdentifier(), *parentInfo);
|
||||
|
||||
if (instrumentor) {
|
||||
instrumentor->runAfterPipeline(pm.getOpName(*op->getContext()),
|
||||
*parentInfo);
|
||||
}
|
||||
return success();
|
||||
}
|
||||
|
||||
/// Find an operation pass manager that can operate on an operation of the given
|
||||
/// type, or nullptr if one does not exist.
|
||||
static OpPassManager *findPassManagerFor(MutableArrayRef<OpPassManager> mgrs,
|
||||
StringRef name) {
|
||||
/// Find an operation pass manager with the given anchor name, or nullptr if one
|
||||
/// does not exist.
|
||||
static OpPassManager *
|
||||
findPassManagerWithAnchor(MutableArrayRef<OpPassManager> mgrs, StringRef name) {
|
||||
auto *it = llvm::find_if(
|
||||
mgrs, [&](OpPassManager &mgr) { return mgr.getOpName() == name; });
|
||||
mgrs, [&](OpPassManager &mgr) { return mgr.getOpAnchorName() == name; });
|
||||
return it == mgrs.end() ? nullptr : &*it;
|
||||
}
|
||||
|
||||
|
|
@ -487,8 +555,9 @@ static OpPassManager *findPassManagerFor(MutableArrayRef<OpPassManager> mgrs,
|
|||
static OpPassManager *findPassManagerFor(MutableArrayRef<OpPassManager> mgrs,
|
||||
OperationName name,
|
||||
MLIRContext &context) {
|
||||
auto *it = llvm::find_if(
|
||||
mgrs, [&](OpPassManager &mgr) { return mgr.getOpName(context) == name; });
|
||||
auto *it = llvm::find_if(mgrs, [&](OpPassManager &mgr) {
|
||||
return mgr.getImpl().canScheduleOn(context, name);
|
||||
});
|
||||
return it == mgrs.end() ? nullptr : &*it;
|
||||
}
|
||||
|
||||
|
|
@ -501,12 +570,47 @@ void OpToOpPassAdaptor::getDependentDialects(DialectRegistry &dialects) const {
|
|||
pm.getDependentDialects(dialects);
|
||||
}
|
||||
|
||||
/// Merge the current pass adaptor into given 'rhs'.
|
||||
void OpToOpPassAdaptor::mergeInto(OpToOpPassAdaptor &rhs) {
|
||||
LogicalResult OpToOpPassAdaptor::tryMergeInto(MLIRContext *ctx,
|
||||
OpToOpPassAdaptor &rhs) {
|
||||
// Functor used to check if a pass manager is generic, i.e. op-agnostic.
|
||||
auto isGenericPM = [&](OpPassManager &pm) { return !pm.getOpName(); };
|
||||
|
||||
// Functor used to detect if the given generic pass manager will have a
|
||||
// potential schedule conflict with the given `otherPMs`.
|
||||
auto hasScheduleConflictWith = [&](OpPassManager &genericPM,
|
||||
MutableArrayRef<OpPassManager> otherPMs) {
|
||||
return llvm::any_of(otherPMs, [&](OpPassManager &pm) {
|
||||
// If this is a non-generic pass manager, a conflict will arise if a
|
||||
// non-generic pass manager's operation name can be scheduled on the
|
||||
// generic passmanager.
|
||||
if (Optional<OperationName> pmOpName = pm.getOpName(*ctx))
|
||||
return genericPM.getImpl().canScheduleOn(*ctx, *pmOpName);
|
||||
// Otherwise, this is a generic pass manager. We current can't determine
|
||||
// when generic pass managers can be merged, so conservatively assume they
|
||||
// conflict.
|
||||
return true;
|
||||
});
|
||||
};
|
||||
|
||||
// Check that if either adaptor has a generic pass manager, that pm is
|
||||
// compatible within any non-generic pass managers.
|
||||
//
|
||||
// Check the current adaptor.
|
||||
auto *lhsGenericPMIt = llvm::find_if(mgrs, isGenericPM);
|
||||
if (lhsGenericPMIt != mgrs.end() &&
|
||||
hasScheduleConflictWith(*lhsGenericPMIt, rhs.mgrs))
|
||||
return failure();
|
||||
// Check the rhs adaptor.
|
||||
auto *rhsGenericPMIt = llvm::find_if(rhs.mgrs, isGenericPM);
|
||||
if (rhsGenericPMIt != rhs.mgrs.end() &&
|
||||
hasScheduleConflictWith(*rhsGenericPMIt, mgrs))
|
||||
return failure();
|
||||
|
||||
for (auto &pm : mgrs) {
|
||||
// If an existing pass manager exists, then merge the given pass manager
|
||||
// into it.
|
||||
if (auto *existingPM = findPassManagerFor(rhs.mgrs, pm.getOpName())) {
|
||||
if (auto *existingPM =
|
||||
findPassManagerWithAnchor(rhs.mgrs, pm.getOpAnchorName())) {
|
||||
pm.getImpl().mergeInto(existingPM->getImpl());
|
||||
} else {
|
||||
// Otherwise, add the given pass manager to the list.
|
||||
|
|
@ -516,10 +620,17 @@ void OpToOpPassAdaptor::mergeInto(OpToOpPassAdaptor &rhs) {
|
|||
mgrs.clear();
|
||||
|
||||
// After coalescing, sort the pass managers within rhs by name.
|
||||
llvm::array_pod_sort(rhs.mgrs.begin(), rhs.mgrs.end(),
|
||||
[](const OpPassManager *lhs, const OpPassManager *rhs) {
|
||||
return lhs->getOpName().compare(rhs->getOpName());
|
||||
});
|
||||
auto compareFn = [](const OpPassManager *lhs, const OpPassManager *rhs) {
|
||||
// Order op-specific pass managers first and op-agnostic pass managers last.
|
||||
if (Optional<StringRef> lhsName = lhs->getOpName()) {
|
||||
if (Optional<StringRef> rhsName = rhs->getOpName())
|
||||
return lhsName->compare(*rhsName);
|
||||
return -1; // lhs(op-specific) < rhs(op-agnostic)
|
||||
}
|
||||
return 1; // lhs(op-agnostic) > rhs(op-specific)
|
||||
};
|
||||
llvm::array_pod_sort(rhs.mgrs.begin(), rhs.mgrs.end(), compareFn);
|
||||
return success();
|
||||
}
|
||||
|
||||
/// Returns the adaptor pass name.
|
||||
|
|
@ -527,7 +638,7 @@ std::string OpToOpPassAdaptor::getAdaptorName() {
|
|||
std::string name = "Pipeline Collection : [";
|
||||
llvm::raw_string_ostream os(name);
|
||||
llvm::interleaveComma(getPassManagers(), os, [&](OpPassManager &pm) {
|
||||
os << '\'' << pm.getOpName() << '\'';
|
||||
os << '\'' << pm.getOpAnchorName() << '\'';
|
||||
});
|
||||
os << ']';
|
||||
return os.str();
|
||||
|
|
@ -561,9 +672,8 @@ void OpToOpPassAdaptor::runOnOperationImpl(bool verifyPasses) {
|
|||
|
||||
// Run the held pipeline over the current operation.
|
||||
unsigned initGeneration = mgr->impl->initializationGeneration;
|
||||
if (failed(runPipeline(mgr->getPasses(), &op, am.nest(&op),
|
||||
verifyPasses, initGeneration, instrumentor,
|
||||
&parentInfo)))
|
||||
if (failed(runPipeline(*mgr, &op, am.nest(&op), verifyPasses,
|
||||
initGeneration, instrumentor, &parentInfo)))
|
||||
return signalPassFailure();
|
||||
}
|
||||
}
|
||||
|
|
@ -589,17 +699,37 @@ void OpToOpPassAdaptor::runOnOperationAsyncImpl(bool verifyPasses) {
|
|||
if (asyncExecutors.empty() || hasSizeMismatch(asyncExecutors.front(), mgrs))
|
||||
asyncExecutors.assign(context->getThreadPool().getThreadCount(), mgrs);
|
||||
|
||||
// This struct represents the information for a single operation to be
|
||||
// scheduled on a pass manager.
|
||||
struct OpPMInfo {
|
||||
OpPMInfo(unsigned passManagerIdx, Operation *op, AnalysisManager am)
|
||||
: passManagerIdx(passManagerIdx), op(op), am(am) {}
|
||||
|
||||
/// The index of the pass manager to schedule the operation on.
|
||||
unsigned passManagerIdx;
|
||||
/// The operation to schedule.
|
||||
Operation *op;
|
||||
/// The analysis manager for the operation.
|
||||
AnalysisManager am;
|
||||
};
|
||||
|
||||
// Run a prepass over the operation to collect the nested operations to
|
||||
// execute over. This ensures that an analysis manager exists for each
|
||||
// operation, as well as providing a queue of operations to execute over.
|
||||
std::vector<std::pair<Operation *, AnalysisManager>> opAMPairs;
|
||||
std::vector<OpPMInfo> opInfos;
|
||||
DenseMap<OperationName, Optional<unsigned>> knownOpPMIdx;
|
||||
for (auto ®ion : getOperation()->getRegions()) {
|
||||
for (auto &block : region) {
|
||||
for (auto &op : block) {
|
||||
// Add this operation iff the name matches any of the pass managers.
|
||||
if (findPassManagerFor(mgrs, op.getName(), *context))
|
||||
opAMPairs.emplace_back(&op, am.nest(&op));
|
||||
for (Operation &op : region.getOps()) {
|
||||
// Get the pass manager index for this operation type.
|
||||
auto pmIdxIt = knownOpPMIdx.try_emplace(op.getName(), llvm::None);
|
||||
if (pmIdxIt.second) {
|
||||
if (auto *mgr = findPassManagerFor(mgrs, op.getName(), *context))
|
||||
pmIdxIt.first->second = std::distance(mgrs.begin(), mgr);
|
||||
}
|
||||
|
||||
// If this operation can be scheduled, add it to the list.
|
||||
if (pmIdxIt.first->second)
|
||||
opInfos.emplace_back(*pmIdxIt.first->second, &op, am.nest(&op));
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -611,8 +741,8 @@ void OpToOpPassAdaptor::runOnOperationAsyncImpl(bool verifyPasses) {
|
|||
// An atomic failure variable for the async executors.
|
||||
std::vector<std::atomic<bool>> activePMs(asyncExecutors.size());
|
||||
std::fill(activePMs.begin(), activePMs.end(), false);
|
||||
auto processFn = [&](auto &opPMPair) {
|
||||
// Find a pass manager for this operation.
|
||||
auto processFn = [&](OpPMInfo &opInfo) {
|
||||
// Find an executor for this operation.
|
||||
auto it = llvm::find_if(activePMs, [](std::atomic<bool> &isActive) {
|
||||
bool expectedInactive = false;
|
||||
return isActive.compare_exchange_strong(expectedInactive, true);
|
||||
|
|
@ -620,14 +750,10 @@ void OpToOpPassAdaptor::runOnOperationAsyncImpl(bool verifyPasses) {
|
|||
unsigned pmIndex = it - activePMs.begin();
|
||||
|
||||
// Get the pass manager for this operation and execute it.
|
||||
auto *pm = findPassManagerFor(asyncExecutors[pmIndex],
|
||||
opPMPair.first->getName(), *context);
|
||||
assert(pm && "expected valid pass manager for operation");
|
||||
|
||||
unsigned initGeneration = pm->impl->initializationGeneration;
|
||||
LogicalResult pipelineResult =
|
||||
runPipeline(pm->getPasses(), opPMPair.first, opPMPair.second,
|
||||
verifyPasses, initGeneration, instrumentor, &parentInfo);
|
||||
OpPassManager &pm = asyncExecutors[pmIndex][opInfo.passManagerIdx];
|
||||
LogicalResult pipelineResult = runPipeline(
|
||||
pm, opInfo.op, opInfo.am, verifyPasses,
|
||||
pm.impl->initializationGeneration, instrumentor, &parentInfo);
|
||||
|
||||
// Reset the active bit for this pass manager.
|
||||
activePMs[pmIndex].store(false);
|
||||
|
|
@ -635,7 +761,7 @@ void OpToOpPassAdaptor::runOnOperationAsyncImpl(bool verifyPasses) {
|
|||
};
|
||||
|
||||
// Signal a failure if any of the executors failed.
|
||||
if (failed(failableParallelForEach(context, opAMPairs, processFn)))
|
||||
if (failed(failableParallelForEach(context, opInfos, processFn)))
|
||||
signalPassFailure();
|
||||
}
|
||||
|
||||
|
|
@ -645,7 +771,7 @@ void OpToOpPassAdaptor::runOnOperationAsyncImpl(bool verifyPasses) {
|
|||
|
||||
PassManager::PassManager(MLIRContext *ctx, Nesting nesting,
|
||||
StringRef operationName)
|
||||
: OpPassManager(StringAttr::get(ctx, operationName), nesting), context(ctx),
|
||||
: OpPassManager(OperationName(operationName, ctx), nesting), context(ctx),
|
||||
initializationKey(DenseMapInfo<llvm::hash_code>::getTombstoneKey()),
|
||||
passTiming(false), verifyPasses(true) {}
|
||||
|
||||
|
|
@ -708,7 +834,7 @@ void PassManager::addInstrumentation(std::unique_ptr<PassInstrumentation> pi) {
|
|||
}
|
||||
|
||||
LogicalResult PassManager::runPasses(Operation *op, AnalysisManager am) {
|
||||
return OpToOpPassAdaptor::runPipeline(getPasses(), op, am, verifyPasses,
|
||||
return OpToOpPassAdaptor::runPipeline(*this, op, am, verifyPasses,
|
||||
impl->initializationGeneration);
|
||||
}
|
||||
|
||||
|
|
@ -788,6 +914,12 @@ void detail::NestedAnalysisMap::invalidate(
|
|||
|
||||
PassInstrumentation::~PassInstrumentation() = default;
|
||||
|
||||
void PassInstrumentation::runBeforePipeline(
|
||||
Optional<OperationName> name, const PipelineParentInfo &parentInfo) {}
|
||||
|
||||
void PassInstrumentation::runAfterPipeline(
|
||||
Optional<OperationName> name, const PipelineParentInfo &parentInfo) {}
|
||||
|
||||
//===----------------------------------------------------------------------===//
|
||||
// PassInstrumentor
|
||||
//===----------------------------------------------------------------------===//
|
||||
|
|
@ -809,7 +941,7 @@ PassInstrumentor::~PassInstrumentor() = default;
|
|||
|
||||
/// See PassInstrumentation::runBeforePipeline for details.
|
||||
void PassInstrumentor::runBeforePipeline(
|
||||
StringAttr name,
|
||||
Optional<OperationName> name,
|
||||
const PassInstrumentation::PipelineParentInfo &parentInfo) {
|
||||
llvm::sys::SmartScopedLock<true> instrumentationLock(impl->mutex);
|
||||
for (auto &instr : impl->instrumentations)
|
||||
|
|
@ -818,7 +950,7 @@ void PassInstrumentor::runBeforePipeline(
|
|||
|
||||
/// See PassInstrumentation::runAfterPipeline for details.
|
||||
void PassInstrumentor::runAfterPipeline(
|
||||
StringAttr name,
|
||||
Optional<OperationName> name,
|
||||
const PassInstrumentation::PipelineParentInfo &parentInfo) {
|
||||
llvm::sys::SmartScopedLock<true> instrumentationLock(impl->mutex);
|
||||
for (auto &instr : llvm::reverse(impl->instrumentations))
|
||||
|
|
|
|||
|
|
@ -29,8 +29,16 @@ public:
|
|||
void runOnOperation(bool verifyPasses);
|
||||
void runOnOperation() override;
|
||||
|
||||
/// Merge the current pass adaptor into given 'rhs'.
|
||||
void mergeInto(OpToOpPassAdaptor &rhs);
|
||||
/// Try to merge the current pass adaptor into 'rhs'. This will try to append
|
||||
/// the pass managers of this adaptor into those within `rhs`, or return
|
||||
/// failure if merging isn't possible. The main situation in which merging is
|
||||
/// not possible is if one of the adaptors has an `any` pipeline that is not
|
||||
/// compatible with a pass manager in the other adaptor. For example, if this
|
||||
/// adaptor has a `func.func` pipeline and `rhs` has an `any` pipeline that
|
||||
/// operates on FunctionOpInterface. In this situation the pipelines have a
|
||||
/// conflict (they both want to run on the same operations), so we can't
|
||||
/// merge.
|
||||
LogicalResult tryMergeInto(MLIRContext *ctx, OpToOpPassAdaptor &rhs);
|
||||
|
||||
/// Returns the pass managers held by this adaptor.
|
||||
MutableArrayRef<OpPassManager> getPassManagers() { return mgrs; }
|
||||
|
|
@ -66,9 +74,8 @@ private:
|
|||
/// parent pass manager, and is used to initialize any dynamic pass pipelines
|
||||
/// run by the given passes.
|
||||
static LogicalResult runPipeline(
|
||||
iterator_range<OpPassManager::pass_iterator> passes, Operation *op,
|
||||
AnalysisManager am, bool verifyPasses, unsigned parentInitGeneration,
|
||||
PassInstrumentor *instrumentor = nullptr,
|
||||
OpPassManager &pm, Operation *op, AnalysisManager am, bool verifyPasses,
|
||||
unsigned parentInitGeneration, PassInstrumentor *instrumentor = nullptr,
|
||||
const PassInstrumentation::PipelineParentInfo *parentInfo = nullptr);
|
||||
|
||||
/// A set of adaptors to run.
|
||||
|
|
|
|||
|
|
@ -38,12 +38,16 @@ buildDefaultRegistryFn(const PassAllocatorFunction &allocator) {
|
|||
function_ref<LogicalResult(const Twine &)> errorHandler) {
|
||||
std::unique_ptr<Pass> pass = allocator();
|
||||
LogicalResult result = pass->initializeOptions(options);
|
||||
if ((pm.getNesting() == OpPassManager::Nesting::Explicit) &&
|
||||
pass->getOpName() && *pass->getOpName() != pm.getOpName())
|
||||
|
||||
Optional<StringRef> pmOpName = pm.getOpName();
|
||||
Optional<StringRef> passOpName = pass->getOpName();
|
||||
if ((pm.getNesting() == OpPassManager::Nesting::Explicit) && pmOpName &&
|
||||
passOpName && *pmOpName != *passOpName) {
|
||||
return errorHandler(llvm::Twine("Can't add pass '") + pass->getName() +
|
||||
"' restricted to '" + *pass->getOpName() +
|
||||
"' on a PassManager intended to run on '" +
|
||||
pm.getOpName() + "', did you intend to nest?");
|
||||
pm.getOpAnchorName() + "', did you intend to nest?");
|
||||
}
|
||||
pm.addPass(std::move(pass));
|
||||
return result;
|
||||
};
|
||||
|
|
|
|||
|
|
@ -120,7 +120,7 @@ static void printResultsAsPipeline(raw_ostream &os, OpPassManager &pm) {
|
|||
|
||||
// Print each of the children passes.
|
||||
for (OpPassManager &mgr : mgrs) {
|
||||
auto name = ("'" + mgr.getOpName() + "' Pipeline").str();
|
||||
auto name = ("'" + mgr.getOpAnchorName() + "' Pipeline").str();
|
||||
printPassEntry(os, indent, name);
|
||||
for (Pass &pass : mgr.getPasses())
|
||||
printPass(indent + 2, &pass);
|
||||
|
|
|
|||
|
|
@ -52,7 +52,7 @@ struct PassTiming : public PassInstrumentation {
|
|||
// Pipeline
|
||||
//===--------------------------------------------------------------------===//
|
||||
|
||||
void runBeforePipeline(StringAttr name,
|
||||
void runBeforePipeline(Optional<OperationName> name,
|
||||
const PipelineParentInfo &parentInfo) override {
|
||||
auto tid = llvm::get_threadid();
|
||||
auto &activeTimers = activeThreadTimers[tid];
|
||||
|
|
@ -68,12 +68,17 @@ struct PassTiming : public PassInstrumentation {
|
|||
} else {
|
||||
parentScope = &activeTimers.back();
|
||||
}
|
||||
activeTimers.push_back(parentScope->nest(name.getAsOpaquePointer(), [name] {
|
||||
return ("'" + name.strref() + "' Pipeline").str();
|
||||
|
||||
// Use nullptr to anchor op-agnostic pipelines, otherwise use the name of
|
||||
// the operation.
|
||||
const void *timerId = name ? name->getAsOpaquePointer() : nullptr;
|
||||
activeTimers.push_back(parentScope->nest(timerId, [name] {
|
||||
return ("'" + (name ? name->getStringRef() : "any") + "' Pipeline").str();
|
||||
}));
|
||||
}
|
||||
|
||||
void runAfterPipeline(StringAttr, const PipelineParentInfo &) override {
|
||||
void runAfterPipeline(Optional<OperationName>,
|
||||
const PipelineParentInfo &) override {
|
||||
auto &activeTimers = activeThreadTimers[llvm::get_threadid()];
|
||||
assert(!activeTimers.empty() && "expected active timer");
|
||||
activeTimers.pop_back();
|
||||
|
|
|
|||
|
|
@ -734,6 +734,7 @@ LogicalResult InlinerPass::initializeOptions(StringRef options) {
|
|||
return failure();
|
||||
|
||||
// Initialize the default pipeline builder to use the option string.
|
||||
// TODO: Use a generic pass manager for default pipelines, and remove this.
|
||||
if (!defaultPipelineStr.empty()) {
|
||||
std::string defaultPipelineCopy = defaultPipelineStr;
|
||||
defaultPipeline = [=](OpPassManager &pm) {
|
||||
|
|
@ -747,7 +748,7 @@ LogicalResult InlinerPass::initializeOptions(StringRef options) {
|
|||
llvm::StringMap<OpPassManager> pipelines;
|
||||
for (OpPassManager pipeline : opPipelineList)
|
||||
if (!pipeline.empty())
|
||||
pipelines.try_emplace(pipeline.getOpName(), pipeline);
|
||||
pipelines.try_emplace(pipeline.getOpAnchorName(), pipeline);
|
||||
opPipelines.assign({std::move(pipelines)});
|
||||
|
||||
return success();
|
||||
|
|
|
|||
|
|
@ -0,0 +1,24 @@
|
|||
// RUN: mlir-opt %s -verify-diagnostics -pass-pipeline='any(cse, test-interface-pass)' -allow-unregistered-dialect -o /dev/null
|
||||
|
||||
// Test that we execute generic pipelines correctly. The `cse` pass is fully generic and should execute
|
||||
// on both the module and the func. The `test-interface-pass` filters based on FunctionOpInterface and
|
||||
// should only execute on the func.
|
||||
|
||||
// expected-remark@below {{Executing interface pass on operation}}
|
||||
func.func @main() -> (i1, i1) {
|
||||
// CHECK-LABEL: func @main
|
||||
// CHECK-NEXT: arith.constant true
|
||||
// CHECK-NEXT: return
|
||||
%true = arith.constant true
|
||||
%true1 = arith.constant true
|
||||
return %true, %true1 : i1, i1
|
||||
}
|
||||
|
||||
module @module {
|
||||
// CHECK-LABEL: module @main
|
||||
// CHECK-NEXT: arith.constant true
|
||||
// CHECK-NEXT: foo.op
|
||||
%true = arith.constant true
|
||||
%true1 = arith.constant true
|
||||
"foo.op"(%true, %true1) : (i1, i1) -> ()
|
||||
}
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
// RUN: mlir-opt %s -mlir-disable-threading -pass-pipeline='builtin.module(test-module-pass,func.func(test-function-pass)),func.func(test-function-pass)' -pass-pipeline="func.func(cse,canonicalize)" -verify-each=false -mlir-timing -mlir-timing-display=tree 2>&1 | FileCheck %s
|
||||
// RUN: mlir-opt %s -mlir-disable-threading -test-textual-pm-nested-pipeline -verify-each=false -mlir-timing -mlir-timing-display=tree 2>&1 | FileCheck %s --check-prefix=TEXTUAL_CHECK
|
||||
// RUN: mlir-opt %s -mlir-disable-threading -pass-pipeline='builtin.module(test-module-pass),any(test-interface-pass),any(test-interface-pass),func.func(test-function-pass),any(canonicalize),func.func(cse)' -verify-each=false -mlir-timing -mlir-timing-display=tree 2>&1 | FileCheck %s --check-prefix=GENERIC_MERGE_CHECK
|
||||
// RUN: not mlir-opt %s -pass-pipeline='builtin.module(test-module-pass' 2>&1 | FileCheck --check-prefix=CHECK_ERROR_1 %s
|
||||
// RUN: not mlir-opt %s -pass-pipeline='builtin.module(test-module-pass))' 2>&1 | FileCheck --check-prefix=CHECK_ERROR_2 %s
|
||||
// RUN: not mlir-opt %s -pass-pipeline='builtin.module()(' 2>&1 | FileCheck --check-prefix=CHECK_ERROR_3 %s
|
||||
|
|
@ -11,6 +12,7 @@
|
|||
// CHECK_ERROR_3: expected ',' after parsing pipeline
|
||||
// CHECK_ERROR_4: does not refer to a registered pass or pass pipeline
|
||||
// CHECK_ERROR_5: Can't add pass '{{.*}}TestModulePass' restricted to 'builtin.module' on a PassManager intended to run on 'func.func', did you intend to nest?
|
||||
|
||||
func.func @foo() {
|
||||
return
|
||||
}
|
||||
|
|
@ -39,3 +41,20 @@ module {
|
|||
// TEXTUAL_CHECK-NEXT: TestModulePass
|
||||
// TEXTUAL_CHECK-NEXT: 'func.func' Pipeline
|
||||
// TEXTUAL_CHECK-NEXT: TestFunctionPass
|
||||
|
||||
// Check that generic pass pipelines are only merged when they aren't
|
||||
// going to overlap with op-specific pipelines.
|
||||
// GENERIC_MERGE_CHECK: Pipeline Collection : ['builtin.module', 'any']
|
||||
// GENERIC_MERGE_CHECK-NEXT: 'any' Pipeline
|
||||
// GENERIC_MERGE_CHECK-NEXT: (anonymous namespace)::TestInterfacePass
|
||||
// GENERIC_MERGE_CHECK-NEXT: 'builtin.module' Pipeline
|
||||
// GENERIC_MERGE_CHECK-NEXT: (anonymous namespace)::TestModulePass
|
||||
// GENERIC_MERGE_CHECK-NEXT: 'any' Pipeline
|
||||
// GENERIC_MERGE_CHECK-NEXT: (anonymous namespace)::TestInterfacePass
|
||||
// GENERIC_MERGE_CHECK-NEXT: 'func.func' Pipeline
|
||||
// GENERIC_MERGE_CHECK-NEXT: (anonymous namespace)::TestFunctionPass
|
||||
// GENERIC_MERGE_CHECK-NEXT: 'any' Pipeline
|
||||
// GENERIC_MERGE_CHECK-NEXT: Canonicalizer
|
||||
// GENERIC_MERGE_CHECK-NEXT: 'func.func' Pipeline
|
||||
// GENERIC_MERGE_CHECK-NEXT: CSE
|
||||
// GENERIC_MERGE_CHECK-NEXT: (A) DominanceInfo
|
||||
|
|
|
|||
Loading…
Reference in New Issue