maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/mlir/test/Dialect/Linalg/transform-patterns.mlir

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

469 lines
26 KiB
MLIR
Raw Normal View History

Add basic support for declarative Linalg transformations Linalg ops provide a good anchor for pattern matching/rewriting transformations. This CL adds a simple example of how multi-level tiling may be specified by attaching a simple StringAttr to ops as they are transformed so we can easily specify partial lowering to control transformation application. This is a first stab at taking advantage of higher-level information contained in Linalg ops and will evolve in the future. PiperOrigin-RevId: 277497958
2019-10-30 22:12:07 +08:00
// RUN: mlir-opt %s -test-linalg-transform-patterns | FileCheck %s
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-DAG: #[[STRIDED_1D:.*]] = affine_map<(d0)[s0, s1] -> (d0 * s1 + s0)>
// Map corresponding to a 2D memory access where the stride along the last dim is known to be 1.
// CHECK-DAG: #[[STRIDED_2D_u_1:.*]] = affine_map<(d0, d1)[s0, s1] -> (d0 * s1 + s0 + d1)>
// Map corresponding to a 2D memory access where the stride along all dims are unknown.
// CHECK-DAG: #[[STRIDED_2D:.*]] = affine_map<(d0, d1)[s0, s1, s2] -> (d0 * s1 + s0 + d1 * s2)>
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
// CHECK-DAG: #[[mk:.*]] = affine_map<(d0, d1, d2) -> (d0, d2)>
// CHECK-DAG: #[[kn:.*]] = affine_map<(d0, d1, d2) -> (d2, d1)>
// CHECK-DAG: #[[mn:.*]] = affine_map<(d0, d1, d2) -> (d0, d1)>
// CHECK-DAG: #[[nm:.*]] = affine_map<(d0, d1, d2) -> (d1, d0)>
// CHECK-DAG: #[[km:.*]] = affine_map<(d0, d1, d2) -> (d2, d0)>
Add basic support for declarative Linalg transformations Linalg ops provide a good anchor for pattern matching/rewriting transformations. This CL adds a simple example of how multi-level tiling may be specified by attaching a simple StringAttr to ops as they are transformed so we can easily specify partial lowering to control transformation application. This is a first stab at taking advantage of higher-level information contained in Linalg ops and will evolve in the future. PiperOrigin-RevId: 277497958
2019-10-30 22:12:07 +08:00
func @dot(%x: memref<?xf32, offset: ?, strides: [1]>,
%y: memref<?xf32, offset: ?, strides: [1]>,
%v: memref<f32>) {
linalg.dot(%x, %y, %v) : memref<?xf32, offset: ?, strides: [1]>,
memref<?xf32, offset: ?, strides: [1]>,
memref<f32>
return
}
// CHECK-LABEL: func @dot
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-DAG: %[[c0:.*]] = constant 0 : index
// CHECK-DAG: %[[c1:.*]] = constant 1 : index
// CHECK-DAG: %[[c8:.*]] = constant 8 : index
// CHECK-DAG: %[[c8000:.*]] = constant 8000 : index
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c8000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c8]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c1]] {
// CHECK: load
// CHECK: load
// CHECK: mulf
// CHECK: load
// CHECK: addf
// CHECK: store
Add basic support for declarative Linalg transformations Linalg ops provide a good anchor for pattern matching/rewriting transformations. This CL adds a simple example of how multi-level tiling may be specified by attaching a simple StringAttr to ops as they are transformed so we can easily specify partial lowering to control transformation application. This is a first stab at taking advantage of higher-level information contained in Linalg ops and will evolve in the future. PiperOrigin-RevId: 277497958
2019-10-30 22:12:07 +08:00
func @matvec(%A: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%x: memref<?xf32, offset: ?, strides: [1]>,
%y: memref<?xf32, offset: ?, strides: [1]>) {
linalg.matvec(%A, %x, %y) : memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?xf32, offset: ?, strides: [1]>,
memref<?xf32, offset: ?, strides: [1]>
return
}
// CHECK-LABEL: func @matvec
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-DAG: %[[c0:.*]] = constant 0 : index
// CHECK-DAG: %[[c5:.*]] = constant 5 : index
// CHECK-DAG: %[[c6:.*]] = constant 6 : index
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c5]]
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c6]]
// CHECK: linalg.matvec({{.*}}, {{.*}}, {{.*}}) : memref<?x?xf32, #[[STRIDED_2D]]>, memref<?xf32, #[[STRIDED_1D]]>, memref<?xf32, #[[STRIDED_1D]]>
Add basic support for declarative Linalg transformations Linalg ops provide a good anchor for pattern matching/rewriting transformations. This CL adds a simple example of how multi-level tiling may be specified by attaching a simple StringAttr to ops as they are transformed so we can easily specify partial lowering to control transformation application. This is a first stab at taking advantage of higher-level information contained in Linalg ops and will evolve in the future. PiperOrigin-RevId: 277497958
2019-10-30 22:12:07 +08:00
func @matmul(%A: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%B: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%C: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
linalg.matmul(%A, %B, %C) : memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>
return
}
// CHECK-LABEL: func @matmul
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-DAG: %[[c0:.*]] = constant 0 : index
// CHECK-DAG: %[[c2:.*]] = constant 2 : index
// CHECK-DAG: %[[c3:.*]] = constant 3 : index
// CHECK-DAG: %[[c4:.*]] = constant 4 : index
// CHECK-DAG: %[[c20:.*]] = constant 20 : index
// CHECK-DAG: %[[c30:.*]] = constant 30 : index
// CHECK-DAG: %[[c40:.*]] = constant 40 : index
// CHECK-DAG: %[[c200:.*]] = constant 200 : index
// CHECK-DAG: %[[c300:.*]] = constant 300 : index
// CHECK-DAG: %[[c400:.*]] = constant 400 : index
// CHECK-DAG: %[[c2000:.*]] = constant 2000 : index
// CHECK-DAG: %[[c3000:.*]] = constant 3000 : index
// CHECK-DAG: %[[c4000:.*]] = constant 4000 : index
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c2000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c3000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c4000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c200]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c300]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c400]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c20]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c30]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c40]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c2]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c3]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c4]] {
// CHECK: linalg.matmul({{.*}}, {{.*}}, {{.*}}) : memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>
Add Linalg pattern for producer-consumer fusion This CL adds a simple pattern for specifying producer-consumer fusion on Linalg operations. Implementing such an extension reveals some interesting properties. Since Linalg operates on a buffer abstraction, the output buffers are specified as in/out parameters to the ops. As a consequence, there are no SSA use-def chains and one cannot specify complex dag input patterns with the current infrastructure. Instead this CL uses constraints based on the existing linalg dependence analysis to focus the pattern and refine patterns based on the type of op that last wrote in a buffer. This is a very local property and is less powerful than the generic dag specification based on SSA use-def chains. This will be generalized in the future. PiperOrigin-RevId: 277931503
2019-11-01 23:29:42 +08:00
#some_generic_trait = {
Continue refactoring StructuredOps utilities This CL adds more common information to StructuredOpsUtils.h The n_view attribute is retired in favor of args_in + args_out but the CL is otherwise NFC. PiperOrigin-RevId: 285000621
2019-12-12 01:26:51 +08:00
args_in = 1,
args_out = 1,
Add Linalg pattern for producer-consumer fusion This CL adds a simple pattern for specifying producer-consumer fusion on Linalg operations. Implementing such an extension reveals some interesting properties. Since Linalg operates on a buffer abstraction, the output buffers are specified as in/out parameters to the ops. As a consequence, there are no SSA use-def chains and one cannot specify complex dag input patterns with the current infrastructure. Instead this CL uses constraints based on the existing linalg dependence analysis to focus the pattern and refine patterns based on the type of op that last wrote in a buffer. This is a very local property and is less powerful than the generic dag specification based on SSA use-def chains. This will be generalized in the future. PiperOrigin-RevId: 277931503
2019-11-01 23:29:42 +08:00
indexing_maps = [
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
affine_map<(i, j) -> (i, j)>,
affine_map<(i, j) -> (i, j)>
Add Linalg pattern for producer-consumer fusion This CL adds a simple pattern for specifying producer-consumer fusion on Linalg operations. Implementing such an extension reveals some interesting properties. Since Linalg operates on a buffer abstraction, the output buffers are specified as in/out parameters to the ops. As a consequence, there are no SSA use-def chains and one cannot specify complex dag input patterns with the current infrastructure. Instead this CL uses constraints based on the existing linalg dependence analysis to focus the pattern and refine patterns based on the type of op that last wrote in a buffer. This is a very local property and is less powerful than the generic dag specification based on SSA use-def chains. This will be generalized in the future. PiperOrigin-RevId: 277931503
2019-11-01 23:29:42 +08:00
],
[Linalg] Change attribute n_loop_types to iterator This addresses issue tensorflow/mlir#270. Linalg is updated to take the same form of iterator_types than vector contraction. Closes tensorflow/mlir#280 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/280 from tetuante:PRissue270 d26d88d090d3765d3b9884bfabdd023143f27287 PiperOrigin-RevId: 282905396
2019-11-28 17:59:22 +08:00
iterator_types = ["parallel", "parallel"]
Add Linalg pattern for producer-consumer fusion This CL adds a simple pattern for specifying producer-consumer fusion on Linalg operations. Implementing such an extension reveals some interesting properties. Since Linalg operates on a buffer abstraction, the output buffers are specified as in/out parameters to the ops. As a consequence, there are no SSA use-def chains and one cannot specify complex dag input patterns with the current infrastructure. Instead this CL uses constraints based on the existing linalg dependence analysis to focus the pattern and refine patterns based on the type of op that last wrote in a buffer. This is a very local property and is less powerful than the generic dag specification based on SSA use-def chains. This will be generalized in the future. PiperOrigin-RevId: 277931503
2019-11-01 23:29:42 +08:00
}
func @fusion_test(%A: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%B: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%C: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%D: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%E: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
// This should not be fused as it would violate dependencies. It will get
// tiled for all levels of the memory hierarchy.
linalg.matmul(%A, %A, %C) : memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>
// This should be fused.
linalg.matmul(%A, %B, %C) : memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>
// This should not be fused or transformed at all since there are no patterns
// on it. However it will be reordered because there are no dependencies.
linalg.generic #some_generic_trait %A, %D {
^bb(%a: f32, %b: f32) :
linalg.yield %a : f32
} : memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>
linalg.matmul(%C, %D, %E) : memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>
return
}
// CHECK-LABEL: func @fusion_test
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-DAG: %[[c0:.*]] = constant 0 : index
// CHECK-DAG: %[[c2:.*]] = constant 2 : index
// CHECK-DAG: %[[c3:.*]] = constant 3 : index
// CHECK-DAG: %[[c4:.*]] = constant 4 : index
// CHECK-DAG: %[[c20:.*]] = constant 20 : index
// CHECK-DAG: %[[c30:.*]] = constant 30 : index
// CHECK-DAG: %[[c40:.*]] = constant 40 : index
// CHECK-DAG: %[[c100:.*]] = constant 100 : index
// CHECK-DAG: %[[c150:.*]] = constant 150 : index
// CHECK-DAG: %[[c200:.*]] = constant 200 : index
// CHECK-DAG: %[[c300:.*]] = constant 300 : index
// CHECK-DAG: %[[c400:.*]] = constant 400 : index
// CHECK-DAG: %[[c2000:.*]] = constant 2000 : index
// CHECK-DAG: %[[c3000:.*]] = constant 3000 : index
// CHECK-DAG: %[[c4000:.*]] = constant 4000 : index
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c2000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c3000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c4000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c200]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c300]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c400]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c20]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c30]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c40]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c2]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c3]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c4]] {
// CHECK: linalg.matmul({{.*}}, {{.*}}, {{.*}}) : memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>
Add Linalg pattern for producer-consumer fusion This CL adds a simple pattern for specifying producer-consumer fusion on Linalg operations. Implementing such an extension reveals some interesting properties. Since Linalg operates on a buffer abstraction, the output buffers are specified as in/out parameters to the ops. As a consequence, there are no SSA use-def chains and one cannot specify complex dag input patterns with the current infrastructure. Instead this CL uses constraints based on the existing linalg dependence analysis to focus the pattern and refine patterns based on the type of op that last wrote in a buffer. This is a very local property and is less powerful than the generic dag specification based on SSA use-def chains. This will be generalized in the future. PiperOrigin-RevId: 277931503
2019-11-01 23:29:42 +08:00
//
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK: linalg.generic
Add Linalg pattern for producer-consumer fusion This CL adds a simple pattern for specifying producer-consumer fusion on Linalg operations. Implementing such an extension reveals some interesting properties. Since Linalg operates on a buffer abstraction, the output buffers are specified as in/out parameters to the ops. As a consequence, there are no SSA use-def chains and one cannot specify complex dag input patterns with the current infrastructure. Instead this CL uses constraints based on the existing linalg dependence analysis to focus the pattern and refine patterns based on the type of op that last wrote in a buffer. This is a very local property and is less powerful than the generic dag specification based on SSA use-def chains. This will be generalized in the future. PiperOrigin-RevId: 277931503
2019-11-01 23:29:42 +08:00
//
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK: loop.for %{{.*}} = %[[c0]] to %{{.*}} step %[[c100]] {
// CHECK: loop.for %{{.*}} = %[[c0]] to %{{.*}} step %[[c150]] {
// CHECK: loop.for %{{.*}} = %[[c0]] to %{{.*}} step %[[c2]] {
// CHECK: loop.for %{{.*}} = %[[c0]] to %{{.*}} step %[[c3]] {
// CHECK: loop.for %{{.*}} = %[[c0]] to %{{.*}} step %[[c4]] {
// CHECK: linalg.matmul(%{{.*}}, %{{.*}}, %{{.*}}) : memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>
// CHECK: loop.for %{{.*}} = %[[c0]] to %{{.*}} step %[[c2]] {
// CHECK: loop.for %{{.*}} = %[[c0]] to %{{.*}} step %[[c3]] {
// CHECK: loop.for %{{.*}} = %[[c0]] to %{{.*}} step %[[c4]] {
// CHECK: linalg.matmul(%{{.*}}, %{{.*}}, %{{.*}}) : memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>
Add Linalg pattern for producer-consumer fusion This CL adds a simple pattern for specifying producer-consumer fusion on Linalg operations. Implementing such an extension reveals some interesting properties. Since Linalg operates on a buffer abstraction, the output buffers are specified as in/out parameters to the ops. As a consequence, there are no SSA use-def chains and one cannot specify complex dag input patterns with the current infrastructure. Instead this CL uses constraints based on the existing linalg dependence analysis to focus the pattern and refine patterns based on the type of op that last wrote in a buffer. This is a very local property and is less powerful than the generic dag specification based on SSA use-def chains. This will be generalized in the future. PiperOrigin-RevId: 277931503
2019-11-01 23:29:42 +08:00
[StructuredOps][Linalg] Add a primitive pattern to rewrite the linalg.generic form of matmul to vector form. This CL uses the newly expanded matcher support to easily detect when a linalg.generic has a multiply-accumulate body. A linalg.generic with such a body is rewritten as a vector contraction. This CL additionally limits the rewrite to the case of matrix multiplication on contiguous and statically shaped memrefs for now. Before expanding further, we should harden the infrastructure for expressing custom ops with the structured ops abstraction. PiperOrigin-RevId: 284566659
2019-12-10 01:14:05 +08:00
#matmul_trait = {
Continue refactoring StructuredOps utilities This CL adds more common information to StructuredOpsUtils.h The n_view attribute is retired in favor of args_in + args_out but the CL is otherwise NFC. PiperOrigin-RevId: 285000621
2019-12-12 01:26:51 +08:00
args_in = 2,
args_out = 1,
[StructuredOps][Linalg] Add a primitive pattern to rewrite the linalg.generic form of matmul to vector form. This CL uses the newly expanded matcher support to easily detect when a linalg.generic has a multiply-accumulate body. A linalg.generic with such a body is rewritten as a vector contraction. This CL additionally limits the rewrite to the case of matrix multiplication on contiguous and statically shaped memrefs for now. Before expanding further, we should harden the infrastructure for expressing custom ops with the structured ops abstraction. PiperOrigin-RevId: 284566659
2019-12-10 01:14:05 +08:00
indexing_maps = [
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
affine_map<(m, n, k) -> (m, k)>,
affine_map<(m, n, k) -> (k, n)>,
affine_map<(m, n, k) -> (m, n)>
[StructuredOps][Linalg] Add a primitive pattern to rewrite the linalg.generic form of matmul to vector form. This CL uses the newly expanded matcher support to easily detect when a linalg.generic has a multiply-accumulate body. A linalg.generic with such a body is rewritten as a vector contraction. This CL additionally limits the rewrite to the case of matrix multiplication on contiguous and statically shaped memrefs for now. Before expanding further, we should harden the infrastructure for expressing custom ops with the structured ops abstraction. PiperOrigin-RevId: 284566659
2019-12-10 01:14:05 +08:00
],
iterator_types = ["parallel", "parallel", "reduction"],
[mlir][Linalg] Extend linalg vectorization to MatmulOp Summary: This is a simple extension to allow vectorization to work not only on GenericLinalgOp but more generally across named ops too. For now, this still only vectorizes matmul-like ops but is a step towards more generic vectorization of Linalg ops. Reviewers: ftynse Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D72942
2020-01-18 03:26:50 +08:00
__internal_linalg_transform__ = "VECTORIZE"
[StructuredOps][Linalg] Add a primitive pattern to rewrite the linalg.generic form of matmul to vector form. This CL uses the newly expanded matcher support to easily detect when a linalg.generic has a multiply-accumulate body. A linalg.generic with such a body is rewritten as a vector contraction. This CL additionally limits the rewrite to the case of matrix multiplication on contiguous and statically shaped memrefs for now. Before expanding further, we should harden the infrastructure for expressing custom ops with the structured ops abstraction. PiperOrigin-RevId: 284566659
2019-12-10 01:14:05 +08:00
}
func @vectorization_test(%A: memref<8x16xf32>, %B: memref<16x32xf32>,
%C: memref<8x32xf32>) {
linalg.generic #matmul_trait %A, %B, %C {
^bb(%a: f32, %b: f32, %c: f32) :
%d = mulf %a, %b: f32
%e = addf %c, %d: f32
linalg.yield %e : f32
} : memref<8x16xf32>, memref<16x32xf32>, memref<8x32xf32>
return
}
// CHECK-LABEL: func @vectorization_test
// CHECK: vector.type_cast %{{.*}} : memref<8x16xf32> to memref<vector<8x16xf32>>
// CHECK: load %{{.*}}[] : memref<vector<8x16xf32>>
// CHECK: vector.type_cast %{{.*}} : memref<16x32xf32> to memref<vector<16x32xf32>>
// CHECK: load %{{.*}}[] : memref<vector<16x32xf32>>
// CHECK: vector.type_cast %{{.*}} : memref<8x32xf32> to memref<vector<8x32xf32>>
// CHECK: load %{{.*}}[] : memref<vector<8x32xf32>>
// CHECK: vector.contract {indexing_maps = [#[[mk]], #[[kn]], #[[mn]]], iterator_types = ["parallel", "parallel", "reduction"]} %{{.*}}, %{{.*}}, %{{.*}} : vector<8x16xf32>, vector<16x32xf32> into vector<8x32xf32>
// CHECK: store %{{.*}}, %{{.*}}[] : memref<vector<8x32xf32>>
[mlir][Linalg] Extend linalg vectorization to MatmulOp Summary: This is a simple extension to allow vectorization to work not only on GenericLinalgOp but more generally across named ops too. For now, this still only vectorizes matmul-like ops but is a step towards more generic vectorization of Linalg ops. Reviewers: ftynse Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D72942
2020-01-18 03:26:50 +08:00
func @vectorization_test_2(%A: memref<8x16xf32>, %B: memref<16x32xf32>,
%C: memref<8x32xf32>) {
linalg.matmul(%A, %B, %C) { __internal_linalg_transform__ = "VECTORIZE"} :
memref<8x16xf32>, memref<16x32xf32>, memref<8x32xf32>
return
}
// CHECK-LABEL: func @vectorization_test_2
// CHECK: vector.contract {{.*}} :
// vector<8x16xf32>, vector<16x32xf32> into vector<8x32xf32>
[mlir] Add primitive transform pattern to rewrite linalg.fill into vector.broadcast form. Summary: This diff adds a transformation patter to rewrite linalg.fill as broadcasting a scaler into a vector. It uses the same preconditioning as matmul (memory is contiguous). Reviewers: nicolasvasilache Subscribers: mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D73391
2020-01-25 08:23:16 +08:00
func @test_vectorize_fill(%A : memref<8x16xf32>, %arg0 : f32) {
linalg.fill(%A, %arg0) { __internal_linalg_transform__ = "VECTORIZE"} : memref<8x16xf32>, f32
return
}
// CHECK-LABEL: func @test_vectorize_fill
// CHECK: vector.broadcast {{.*}} : f32 to vector<8x16xf32>
[Linalg] Add a Linalg iterator permutation transformation This patch closes issue tensorflow/mlir#272 We add a standalone iterator permutation transformation to Linalg. This transformation composes a permutation map with the maps in the "indexing_maps" attribute. It also permutes "iterator_types" accordingly. Change-Id: I7c1e693b8203aeecc595a7c012e738ca1100c857 Closes tensorflow/mlir#307 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/307 from tetuante:issue272 f7908d58792f4111119721885e247045104f1131 PiperOrigin-RevId: 284824102
2019-12-11 04:25:10 +08:00
#matmul_accesses = [
[mlir][Linalg] Drop function attribute from generic ops. The function attribute in generic ops is not paying for itself. A region is the more standardized way of specifying a custom computation. If needed this region can call a function directly. This is deemed more natural than managing a dedicated function attribute. This also simplifies named ops generation by trimming unnecessary complexity. Differential Revision: https://reviews.llvm.org/D78266
2020-04-16 21:46:00 +08:00
affine_map<(m, n, k) -> (m, k)>,
affine_map<(m, n, k) -> (k, n)>,
affine_map<(m, n, k) -> (m, n)>
[Linalg] Add a Linalg iterator permutation transformation This patch closes issue tensorflow/mlir#272 We add a standalone iterator permutation transformation to Linalg. This transformation composes a permutation map with the maps in the "indexing_maps" attribute. It also permutes "iterator_types" accordingly. Change-Id: I7c1e693b8203aeecc595a7c012e738ca1100c857 Closes tensorflow/mlir#307 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/307 from tetuante:issue272 f7908d58792f4111119721885e247045104f1131 PiperOrigin-RevId: 284824102
2019-12-11 04:25:10 +08:00
]
#generic_matmul_trait = {
[mlir][Linalg] Drop function attribute from generic ops. The function attribute in generic ops is not paying for itself. A region is the more standardized way of specifying a custom computation. If needed this region can call a function directly. This is deemed more natural than managing a dedicated function attribute. This also simplifies named ops generation by trimming unnecessary complexity. Differential Revision: https://reviews.llvm.org/D78266
2020-04-16 21:46:00 +08:00
args_in = 2,
args_out = 1,
indexing_maps = #matmul_accesses,
library_call = "linalg_matmul",
iterator_types = ["parallel", "parallel", "reduction"]
}
[Linalg] Add a Linalg iterator permutation transformation This patch closes issue tensorflow/mlir#272 We add a standalone iterator permutation transformation to Linalg. This transformation composes a permutation map with the maps in the "indexing_maps" attribute. It also permutes "iterator_types" accordingly. Change-Id: I7c1e693b8203aeecc595a7c012e738ca1100c857 Closes tensorflow/mlir#307 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/307 from tetuante:issue272 f7908d58792f4111119721885e247045104f1131 PiperOrigin-RevId: 284824102
2019-12-11 04:25:10 +08:00
func @permute_generic(%A: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%B: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%C: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
[mlir][Linalg] Drop function attribute from generic ops. The function attribute in generic ops is not paying for itself. A region is the more standardized way of specifying a custom computation. If needed this region can call a function directly. This is deemed more natural than managing a dedicated function attribute. This also simplifies named ops generation by trimming unnecessary complexity. Differential Revision: https://reviews.llvm.org/D78266
2020-04-16 21:46:00 +08:00
linalg.generic #generic_matmul_trait %A, %B, %C {
^bb(%a: f32, %b: f32, %c: f32):
%d = mulf %a, %b: f32
%e = addf %c, %d: f32
linalg.yield %e: f32
}: memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>
[Linalg] Add a Linalg iterator permutation transformation This patch closes issue tensorflow/mlir#272 We add a standalone iterator permutation transformation to Linalg. This transformation composes a permutation map with the maps in the "indexing_maps" attribute. It also permutes "iterator_types" accordingly. Change-Id: I7c1e693b8203aeecc595a7c012e738ca1100c857 Closes tensorflow/mlir#307 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/307 from tetuante:issue272 f7908d58792f4111119721885e247045104f1131 PiperOrigin-RevId: 284824102
2019-12-11 04:25:10 +08:00
return
}
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-LABEL: func @permute_generic
// CHECK: linalg.generic {args_in = 2 : i64, args_out = 1 : i64,
// CHECK-SAME: indexing_maps = [#[[kn]], #[[nm]], #[[km]]],
// CHECK-SAME: iterator_types = ["parallel", "reduction", "parallel"],
// CHECK-SAME: library_call = "linalg_matmul"} %{{.*}}, %{{.*}}, %{{.*}}
// CHECK: memref<?x?xf32, #[[STRIDED_2D_u_1]]>,
// CHECK-SAME: memref<?x?xf32, #[[STRIDED_2D_u_1]]>,
// CHECK-SAME: memref<?x?xf32, #[[STRIDED_2D_u_1]]>
[Linalg] Add a Linalg iterator permutation transformation This patch closes issue tensorflow/mlir#272 We add a standalone iterator permutation transformation to Linalg. This transformation composes a permutation map with the maps in the "indexing_maps" attribute. It also permutes "iterator_types" accordingly. Change-Id: I7c1e693b8203aeecc595a7c012e738ca1100c857 Closes tensorflow/mlir#307 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/307 from tetuante:issue272 f7908d58792f4111119721885e247045104f1131 PiperOrigin-RevId: 284824102
2019-12-11 04:25:10 +08:00
#indexed_matmul_trait = {
[mlir][Linalg] Drop function attribute from generic ops. The function attribute in generic ops is not paying for itself. A region is the more standardized way of specifying a custom computation. If needed this region can call a function directly. This is deemed more natural than managing a dedicated function attribute. This also simplifies named ops generation by trimming unnecessary complexity. Differential Revision: https://reviews.llvm.org/D78266
2020-04-16 21:46:00 +08:00
args_in = 2,
args_out = 1,
indexing_maps = #matmul_accesses,
library_call = "linalg_matmul_indexed",
iterator_types = ["parallel", "parallel", "reduction"]
[Linalg] Add a Linalg iterator permutation transformation This patch closes issue tensorflow/mlir#272 We add a standalone iterator permutation transformation to Linalg. This transformation composes a permutation map with the maps in the "indexing_maps" attribute. It also permutes "iterator_types" accordingly. Change-Id: I7c1e693b8203aeecc595a7c012e738ca1100c857 Closes tensorflow/mlir#307 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/307 from tetuante:issue272 f7908d58792f4111119721885e247045104f1131 PiperOrigin-RevId: 284824102
2019-12-11 04:25:10 +08:00
}
[mlir][Linalg] Drop function attribute from generic ops. The function attribute in generic ops is not paying for itself. A region is the more standardized way of specifying a custom computation. If needed this region can call a function directly. This is deemed more natural than managing a dedicated function attribute. This also simplifies named ops generation by trimming unnecessary complexity. Differential Revision: https://reviews.llvm.org/D78266
2020-04-16 21:46:00 +08:00
func @permute_generic_indexed(
%A: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%B: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%C: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
linalg.indexed_generic #indexed_matmul_trait %A, %B, %C {
^bb(%i: index, %j: index, %k: index, %a: f32, %b: f32, %c: f32):
%d = mulf %a, %b: f32
%e = addf %c, %d: f32
linalg.yield %e: f32
} : memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>
[Linalg] Add a Linalg iterator permutation transformation This patch closes issue tensorflow/mlir#272 We add a standalone iterator permutation transformation to Linalg. This transformation composes a permutation map with the maps in the "indexing_maps" attribute. It also permutes "iterator_types" accordingly. Change-Id: I7c1e693b8203aeecc595a7c012e738ca1100c857 Closes tensorflow/mlir#307 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/307 from tetuante:issue272 f7908d58792f4111119721885e247045104f1131 PiperOrigin-RevId: 284824102
2019-12-11 04:25:10 +08:00
return
}
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-LABEL: func @permute_generic_indexed
// CHECK: linalg.indexed_generic {args_in = 2 : i64, args_out = 1 : i64,
// CHECK-SAME: indexing_maps = [#[[kn]], #[[nm]], #[[km]]],
// CHECK-SAME: iterator_types = ["parallel", "reduction", "parallel"],
// CHECK-SAME: library_call = "linalg_matmul_indexed"} %{{.*}}, %{{.*}}, %{{.*}}
// CHECK: memref<?x?xf32, #[[STRIDED_2D_u_1]]>,
// CHECK-SAME: memref<?x?xf32, #[[STRIDED_2D_u_1]]>,
// CHECK-SAME: memref<?x?xf32, #[[STRIDED_2D_u_1]]>
Fold TestLinalgTilePermutePatterns into TestLinalgTransformPatterns - NFC Centralize all patterns that test Linalg transforms in a single pass. PiperOrigin-RevId: 284835938
2019-12-11 05:25:41 +08:00
func @dot_perm(%x: memref<?xf32, offset: ?, strides: [1]>,
%y: memref<?xf32, offset: ?, strides: [1]>,
%v: memref<f32>) {
linalg.dot(%x, %y, %v) {__internal_linalg_transform__ = "__with_perm__"} :
memref<?xf32, offset: ?, strides: [1]>,
memref<?xf32, offset: ?, strides: [1]>,
memref<f32>
return
}
// CHECK-LABEL: func @dot_perm
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-DAG: %[[c0:.*]] = constant 0 : index
// CHECK-DAG: %[[c8:.*]] = constant 8 : index
// CHECK-DAG: %[[c8000:.*]] = constant 8000 : index
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c8000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c8]] {
// CHECK: linalg.dot({{.*}}, {{.*}}, {{.*}}) : memref<?xf32, #[[STRIDED_1D]]>, memref<?xf32, #[[STRIDED_1D]]>, memref<f32>
Fold TestLinalgTilePermutePatterns into TestLinalgTransformPatterns - NFC Centralize all patterns that test Linalg transforms in a single pass. PiperOrigin-RevId: 284835938
2019-12-11 05:25:41 +08:00
func @matvec_perm(%A: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%x: memref<?xf32, offset: ?, strides: [1]>,
%y: memref<?xf32, offset: ?, strides: [1]>) {
linalg.matvec(%A, %x, %y) {__internal_linalg_transform__ = "__with_perm__"} :
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?xf32, offset: ?, strides: [1]>,
memref<?xf32, offset: ?, strides: [1]>
return
}
// CHECK-LABEL: func @matvec_perm
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-DAG: %[[c0:.*]] = constant 0 : index
// CHECK-DAG: %[[c5:.*]] = constant 5 : index
// CHECK-DAG: %[[c6:.*]] = constant 6 : index
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c6]]
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c5]]
// CHECK: linalg.matvec({{.*}}, {{.*}}, {{.*}}) : memref<?x?xf32, #[[STRIDED_2D]]>, memref<?xf32, #[[STRIDED_1D]]>, memref<?xf32, #[[STRIDED_1D]]>
Fold TestLinalgTilePermutePatterns into TestLinalgTransformPatterns - NFC Centralize all patterns that test Linalg transforms in a single pass. PiperOrigin-RevId: 284835938
2019-12-11 05:25:41 +08:00
func @matmul_perm(%A: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%B: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%C: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
linalg.matmul(%A, %B, %C) {__internal_linalg_transform__ = "__with_perm__"} :
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>,
memref<?x?xf32, offset: ?, strides: [?, 1]>
return
}
// CHECK-LABEL: func @matmul_perm
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-DAG: %[[c0:.*]] = constant 0 : index
// CHECK-DAG: %[[c20:.*]] = constant 20 : index
// CHECK-DAG: %[[c30:.*]] = constant 30 : index
// CHECK-DAG: %[[c40:.*]] = constant 40 : index
// CHECK-DAG: %[[c200:.*]] = constant 200 : index
// CHECK-DAG: %[[c300:.*]] = constant 300 : index
// CHECK-DAG: %[[c400:.*]] = constant 400 : index
// CHECK-DAG: %[[c2000:.*]] = constant 2000 : index
// CHECK-DAG: %[[c3000:.*]] = constant 3000 : index
// CHECK-DAG: %[[c4000:.*]] = constant 4000 : index
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c3000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c4000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c2000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c300]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c200]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c400]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c20]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c30]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c40]] {
// CHECK: linalg.matmul({{.*}}, {{.*}}, {{.*}}) : memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>
[Linalg] Expose subview promotion as a declarative pattern This PR targest issue tensorflow/mlir#295. It exposes the already existing subiew promotion pass as a declarative pattern Change-Id: If901ebef9fb53fcd0b12ecc536f6b174ce320b92 Closes tensorflow/mlir#315 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/315 from tetuante:issue295 8e5f268b6d85f31015c33505329dbd7a4db97ac5 PiperOrigin-RevId: 285801463
2019-12-17 02:36:06 +08:00
func @promote_subview_matmul(%arg0: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%arg1: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%arg2: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
%c2000 = constant 2000 : index
%c3000 = constant 3000 : index
%c4000 = constant 4000 : index
%c0 = constant 0 : index
%c1 = constant 1 : index
%0 = dim %arg0, 0 : memref<?x?xf32, offset: ?, strides: [?, 1]>
%1 = dim %arg0, 1 : memref<?x?xf32, offset: ?, strides: [?, 1]>
%2 = dim %arg1, 1 : memref<?x?xf32, offset: ?, strides: [?, 1]>
loop.for %arg3 = %c0 to %0 step %c2000 {
loop.for %arg4 = %c0 to %2 step %c3000 {
loop.for %arg5 = %c0 to %1 step %c4000 {
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
%3 = subview %arg0[%arg3, %arg5][%c2000, %c4000][%c1, %c1] :
[Linalg] Expose subview promotion as a declarative pattern This PR targest issue tensorflow/mlir#295. It exposes the already existing subiew promotion pass as a declarative pattern Change-Id: If901ebef9fb53fcd0b12ecc536f6b174ce320b92 Closes tensorflow/mlir#315 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/315 from tetuante:issue295 8e5f268b6d85f31015c33505329dbd7a4db97ac5 PiperOrigin-RevId: 285801463
2019-12-17 02:36:06 +08:00
memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]>
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
%4 = subview %arg1[%arg5, %arg4][%c4000, %c3000][%c1, %c1] :
[Linalg] Expose subview promotion as a declarative pattern This PR targest issue tensorflow/mlir#295. It exposes the already existing subiew promotion pass as a declarative pattern Change-Id: If901ebef9fb53fcd0b12ecc536f6b174ce320b92 Closes tensorflow/mlir#315 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/315 from tetuante:issue295 8e5f268b6d85f31015c33505329dbd7a4db97ac5 PiperOrigin-RevId: 285801463
2019-12-17 02:36:06 +08:00
memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]>
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
%5 = subview %arg2[%arg3, %arg4][%c2000, %c3000][%c1, %c1] :
[Linalg] Expose subview promotion as a declarative pattern This PR targest issue tensorflow/mlir#295. It exposes the already existing subiew promotion pass as a declarative pattern Change-Id: If901ebef9fb53fcd0b12ecc536f6b174ce320b92 Closes tensorflow/mlir#315 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/315 from tetuante:issue295 8e5f268b6d85f31015c33505329dbd7a4db97ac5 PiperOrigin-RevId: 285801463
2019-12-17 02:36:06 +08:00
memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]>
linalg.matmul(%3, %4, %5) {__internal_linalg_transform__ = "_promote_views_"} :
memref<?x?xf32, offset: ?, strides: [?, ?]>,
memref<?x?xf32, offset: ?, strides: [?, ?]>,
memref<?x?xf32, offset: ?, strides: [?, ?]>
}
}
}
return
}
// CHECK-LABEL: func @promote_subview_matmul
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c2000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c3000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c4000]] {
// CHECK: %[[s0:.*]] = subview {{%.*}}[{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}>
// CHECK: %[[s1:.*]] = subview {{%.*}}[{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}>
// CHECK: %[[s2:.*]] = subview {{%.*}}[{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}>
// CHECK: %[[a0:.*]] = alloc({{%.*}}) : memref<?xi8>
// CHECK: %[[v0:.*]] = std.view %[[a0]][][{{%.*}}, {{%.*}}] : memref<?xi8> to memref<?x?xf32>
// CHECK: %[[l0:.*]] = subview %[[v0]][{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32> to memref<?x?xf32, #[[STRIDED_2D]]>
// CHECK: %[[a1:.*]] = alloc({{%.*}}) : memref<?xi8>
// CHECK: %[[v1:.*]] = std.view %[[a1]][][{{%.*}}, {{%.*}}] : memref<?xi8> to memref<?x?xf32>
// CHECK: %[[l1:.*]] = subview %[[v1]][{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32> to memref<?x?xf32, #[[STRIDED_2D]]>
// CHECK: %[[a2:.*]] = alloc({{%.*}}) : memref<?xi8>
// CHECK: %[[v2:.*]] = std.view %[[a2]][][{{%.*}}, {{%.*}}] : memref<?xi8> to memref<?x?xf32>
// CHECK: %[[l2:.*]] = subview %[[v2]][{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32> to memref<?x?xf32, #[[STRIDED_2D]]>
// CHECK: linalg.copy(%[[s0]], %[[l0]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}>
// CHECK: linalg.copy(%[[s1]], %[[l1]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}>
// CHECK: linalg.copy(%[[s2]], %[[l2]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}>
// CHECK: linalg.matmul(%[[v0]], %[[v1]], %[[v2]]) : memref<?x?xf32>, memref<?x?xf32>, memref<?x?xf32>
[mlir] [linalg] Only promote selected buffers. The promotion transformation is promoting all input and output buffers of the transformed op. The user might want to only promote some of these buffers. Differential Revision: https://reviews.llvm.org/D78498
2020-04-21 17:43:28 +08:00
func @promote_first_subview_matmul(%arg0: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%arg1: memref<?x?xf32, offset: ?, strides: [?, 1]>,
%arg2: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
%c2000 = constant 2000 : index
%c3000 = constant 3000 : index
%c4000 = constant 4000 : index
%c0 = constant 0 : index
%c1 = constant 1 : index
%0 = dim %arg0, 0 : memref<?x?xf32, offset: ?, strides: [?, 1]>
%1 = dim %arg0, 1 : memref<?x?xf32, offset: ?, strides: [?, 1]>
%2 = dim %arg1, 1 : memref<?x?xf32, offset: ?, strides: [?, 1]>
loop.for %arg3 = %c0 to %0 step %c2000 {
loop.for %arg4 = %c0 to %2 step %c3000 {
loop.for %arg5 = %c0 to %1 step %c4000 {
%3 = std.subview %arg0[%arg3, %arg5][%c2000, %c4000][%c1, %c1] :
memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]>
%4 = std.subview %arg1[%arg5, %arg4][%c4000, %c3000][%c1, %c1] :
memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]>
%5 = std.subview %arg2[%arg3, %arg4][%c2000, %c3000][%c1, %c1] :
memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]>
linalg.matmul(%3, %4, %5) {__internal_linalg_transform__ = "_promote_first_view_"} :
memref<?x?xf32, offset: ?, strides: [?, ?]>,
memref<?x?xf32, offset: ?, strides: [?, ?]>,
memref<?x?xf32, offset: ?, strides: [?, ?]>
}
}
}
return
}
// CHECK-LABEL: func @promote_first_subview_matmul
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c2000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c3000]] {
// CHECK: loop.for {{.*}} = %[[c0]] to {{.*}} step %[[c4000]] {
[mlir] [linalg] Only promote selected buffers. The promotion transformation is promoting all input and output buffers of the transformed op. The user might want to only promote some of these buffers. Differential Revision: https://reviews.llvm.org/D78498
2020-04-21 17:43:28 +08:00
// CHECK: %[[s0:.*]] = subview {{%.*}}[{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}>
// CHECK: %[[s1:.*]] = subview {{%.*}}[{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}>
// CHECK: %[[s2:.*]] = subview {{%.*}}[{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}>
// CHECK: %[[a0:.*]] = alloc({{%.*}}) : memref<?xi8>
// CHECK: %[[v0:.*]] = std.view %[[a0]][][{{%.*}}, {{%.*}}] : memref<?xi8> to memref<?x?xf32>
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK: %[[l0:.*]] = subview %[[v0]][{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32> to memref<?x?xf32, #[[STRIDED_2D]]>
[mlir] [linalg] Only promote selected buffers. The promotion transformation is promoting all input and output buffers of the transformed op. The user might want to only promote some of these buffers. Differential Revision: https://reviews.llvm.org/D78498
2020-04-21 17:43:28 +08:00
// CHECK-NOT: %[[a1:.*]] = alloc({{%.*}}) : memref<?xi8>
// CHECK-NOT: %[[v1:.*]] = std.view %[[a1]][][{{%.*}}, {{%.*}}] : memref<?xi8> to memref<?x?xf32>
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-NOT: %[[l0:.*]] = subview %[[v1]][{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32> to memref<?x?xf32, #[[STRIDED_2D]]>
[mlir] [linalg] Only promote selected buffers. The promotion transformation is promoting all input and output buffers of the transformed op. The user might want to only promote some of these buffers. Differential Revision: https://reviews.llvm.org/D78498
2020-04-21 17:43:28 +08:00
// CHECK-NOT: %[[a2:.*]] = alloc({{%.*}}) : memref<?xi8>
// CHECK-NOT: %[[v2:.*]] = std.view %[[a2]][][{{%.*}}, {{%.*}}] : memref<?xi8> to memref<?x?xf32>
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK-NOT: %[[l0:.*]] = subview %[[v2]][{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32> to memref<?x?xf32, #[[STRIDED_2D]]>
[mlir] [linalg] Only promote selected buffers. The promotion transformation is promoting all input and output buffers of the transformed op. The user might want to only promote some of these buffers. Differential Revision: https://reviews.llvm.org/D78498
2020-04-21 17:43:28 +08:00
// CHECK: linalg.copy(%[[s0]], %[[l0]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}>
// CHECK-NOT: linalg.copy(%[[s1]], %[[l1]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}>
// CHECK-NOT: linalg.copy(%[[s2]], %[[l2]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}>^
[mlir] [linalg] Fix transform-patterns test. Unfortunately FileCheck ignores directives with whitespace between the directive and the colon (`CHECK :` for example), thus most of the directives of this test were ignored. Differential Revision: https://reviews.llvm.org/D78548
2020-04-21 18:51:35 +08:00
// CHECK: linalg.matmul(%[[v0]], %[[s1]], %[[s2]]) : memref<?x?xf32>, memref<?x?xf32, #[[STRIDED_2D]]>, memref<?x?xf32, #[[STRIDED_2D]]>
[mlir] [linalg] Specify alignment during promotion. The buffer allocated by a promotion can be subject to other transformations afterward. For example it could be vectorized, in which case it is needed to ensure that this buffer is memory-aligned. Differential Revision: https://reviews.llvm.org/D78556
2020-04-22 00:12:11 +08:00
func @aligned_promote_fill(%arg0: memref<?x?xf32, offset: ?, strides: [?, 1]>) {
%c2000 = constant 2000 : index
%c4000 = constant 4000 : index
%c0 = constant 0 : index
%c1 = constant 1 : index
%cf = constant 1.0 : f32
%3 = std.subview %arg0[%c0, %c0][%c2000, %c4000][%c1, %c1] :
memref<?x?xf32, offset: ?, strides: [?, 1]> to memref<?x?xf32, offset: ?, strides: [?, ?]>
linalg.fill(%3, %cf) { __internal_linalg_transform__ = "_promote_views_aligned_"}
: memref<?x?xf32, offset: ?, strides: [?, ?]>, f32
return
}
// CHECK-LABEL: func @aligned_promote_fill
// CHECK: %[[cf:.*]] = constant {{.*}} : f32
// CHECK: %[[s0:.*]] = subview {{%.*}}[{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32, #map{{.*}}> to memref<?x?xf32, #map{{.*}}>
// CHECK: %[[a0:.*]] = alloc({{%.*}}) {alignment = 32 : i64} : memref<?xi8>
// CHECK: %[[v0:.*]] = std.view %[[a0]][][{{%.*}}, {{%.*}}] : memref<?xi8> to memref<?x?xf32>
// CHECK: %[[l0:.*]] = subview %[[v0]][{{%.*}}, {{%.*}}] [{{%.*}}, {{%.*}}] : memref<?x?xf32> to memref<?x?xf32, #[[STRIDED_2D]]>
// CHECK: linalg.fill(%[[v0]], {{%.*}}) : memref<?x?xf32>, f32
// CHECK: linalg.copy(%[[s0]], %[[l0]]) : memref<?x?xf32, #map{{.*}}>, memref<?x?xf32, #map{{.*}}>
// CHECK: linalg.fill(%[[v0]], %[[cf]]) : memref<?x?xf32>, f32