maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/mlir/test/Transforms/loop-fusion.mlir

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2809 lines
99 KiB
MLIR
Raw Normal View History

Add a flag on the context to protect against creation of operations in unregistered dialects Differential Revision: https://reviews.llvm.org/D76903
2020-03-30 06:35:38 +08:00
// RUN: mlir-opt -allow-unregistered-dialect %s -affine-loop-fusion -split-input-file | FileCheck %s
[mlir] Eliminate the remaining usages of cl::opt instead of PassOption. Summary: Pass options are a better choice for various reasons and avoid the need for static constructors. Differential Revision: https://reviews.llvm.org/D77707
2020-04-09 03:57:02 +08:00
// RUN: mlir-opt -allow-unregistered-dialect %s -affine-loop-fusion="fusion-maximal" -split-input-file | FileCheck %s --check-prefix=MAXIMAL
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
[mlir][NFC] Remove usernames and google bug numbers from TODO comments. These were largely leftover from when MLIR was a google project, and don't really follow LLVM guidelines.
2020-07-07 16:35:23 +08:00
// TODO: Add more tests:
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// *) Add nested fusion test cases when non-constant loop bound support is
// added to iteration domain in dependence check.
// *) Add a test w/ floordiv/ceildiv/mod when supported in dependence check.
// *) Add tests which check fused computation slice indexing and loop bounds.
[mlir][NFC] Remove usernames and google bug numbers from TODO comments. These were largely leftover from when MLIR was a google project, and don't really follow LLVM guidelines.
2020-07-07 16:35:23 +08:00
// TODO: Test clean up: move memref allocs to func args.
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// -----
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_fuse_raw_dep_for_locality() {
func @should_fuse_raw_dep_for_locality() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %m[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_fuse_reduction_to_pointwise() {
func @should_fuse_reduction_to_pointwise() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%a = alloc() : memref<10x10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %b[%i0] : memref<10xf32>
%v1 = affine.load %a[%i0, %i1] : memref<10x10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%v3 = addf %v0, %v1 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %v3, %b[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v4 = affine.load %b[%i2] : memref<10xf32>
affine.store %v4, %c[%i2] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
// Should fuse in entire inner loop on %i1 from source loop nest, as %i1
// is not used in the access function of the store/load on %b.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x10xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
Improve loop fusion algorithm by using a memref dependence graph. Fixed TODO for reduction fusion unit test. PiperOrigin-RevId: 226277226
2018-12-20 12:42:55 +08:00
// CHECK-NEXT: }
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-DAG: [[$MAP_SHIFT_MINUS_ONE_R1:#map[0-9]+]] = affine_map<(d0) -> (d0 - 1)>
// CHECK-DAG: [[$MAP_SHIFT_D0_BY_ONE:#map[0-9]+]] = affine_map<(d0, d1) -> (d0 + 1)>
// CHECK-DAG: [[$MAP_SHIFT_D1_BY_ONE:#map[0-9]+]] = affine_map<(d0, d1) -> (d1 + 1)>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_fuse_loop_nests_with_shifts() {
func @should_fuse_loop_nests_with_shifts() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%a = alloc() : memref<10x10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 9 {
affine.for %i1 = 0 to 9 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
affine.store %cf7, %a[%i0 + 1, %i1 + 1] : memref<10x10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 1 to 10 {
affine.for %i3 = 1 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i2, %i3] : memref<10x10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
}
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// Source slice affine apply sequence:
// *) First two affine apply's map from the dst to src iteration space.
// *) Third affine apply is access function around src store.
// *) Fourth affine apply shifts the stores access function by '-1', because
// of the offset induced by reducing the memref shape from 10x10 to 9x9.
// *) Fifth affine apply shifts the loads access function by '-1', because
// of the offset induced by reducing the memref shape from 10x10 to 9x9.
// NOTE: Should create a private memref with reduced shape 9x9xf32.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 1 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 1 to 10 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: %[[I:.*]] = affine.apply [[$MAP_SHIFT_MINUS_ONE_R1]](%{{.*}})
// CHECK-NEXT: %[[J:.*]] = affine.apply [[$MAP_SHIFT_MINUS_ONE_R1]](%{{.*}})
// CHECK-NEXT: affine.apply [[$MAP_SHIFT_D0_BY_ONE]](%[[I]], %[[J]])
// CHECK-NEXT: affine.apply [[$MAP_SHIFT_D1_BY_ONE]](%[[I]], %[[J]])
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0] : memref<1x1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0] : memref<1x1xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
Adds a dependence check to test whether two accesses to the same memref access the same element. - Builds access functions and iterations domains for each access. - Builds dependence polyhedron constraint system which has equality constraints for equated access functions and inequality constraints for iteration domain loop bounds. - Runs elimination on the dependence polyhedron to test if no dependence exists between the accesses. - Adds a trivial LoopFusion transformation pass with a simple test policy to test dependence between accesses to the same memref in adjacent loops. - The LoopFusion pass will be extended in subsequent CLs. PiperOrigin-RevId: 219630898
2018-11-01 22:26:00 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
return
}
// -----
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_fuse_loop_nest() {
func @should_fuse_loop_nest() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%a = alloc() : memref<10x10xf32>
%b = alloc() : memref<10x10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%i0, %i1] : memref<10x10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
affine.for %i3 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i3, %i2] : memref<10x10xf32>
affine.store %v0, %b[%i2, %i3] : memref<10x10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i4 = 0 to 10 {
affine.for %i5 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %b[%i4, %i5] : memref<10x10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
}
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// Expecting private memref for '%a' first, then private memref for '%b'.
// CHECK-DAG: [[NEWA:%[0-9]+]] = alloc() : memref<1x1xf32>
// CHECK-DAG: [[NEWB:%[0-9]+]] = alloc() : memref<1x1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, [[NEWA]][0, 0] : memref<1x1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load [[NEWA]][0, 0] : memref<1x1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, [[NEWB]][0, 0] : memref<1x1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load [[NEWB]][0, 0] : memref<1x1xf32>
Adds a dependence check to test whether two accesses to the same memref access the same element. - Builds access functions and iterations domains for each access. - Builds dependence polyhedron constraint system which has equality constraints for equated access functions and inequality constraints for iteration domain loop bounds. - Runs elimination on the dependence polyhedron to test if no dependence exists between the accesses. - Adds a trivial LoopFusion transformation pass with a simple test policy to test dependence between accesses to the same memref in adjacent loops. - The LoopFusion pass will be extended in subsequent CLs. PiperOrigin-RevId: 219630898
2018-11-01 22:26:00 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
return
}
// -----
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_fuse_across_intermediate_loop_with_no_deps() {
func @should_fuse_across_intermediate_loop_with_no_deps() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i0] : memref<10xf32>
affine.store %v0, %b[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %c[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %b[%i2] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
// Should fuse first loop (past second loop with no dependences) into third.
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// Note that fusion creates a private memref '%2' for the fused loop nest.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_fuse_all_loops() {
func @should_fuse_all_loops() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
// Set up flow dependences from first and second loops to third.
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %b[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i2] : memref<10xf32>
%v1 = affine.load %b[%i2] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
// Should fuse first and second loops into third.
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// Expecting private memref for '%a' first, then private memref for '%b'.
// CHECK-DAG: [[NEWA:%[0-9]+]] = alloc() : memref<1xf32>
// CHECK-DAG: [[NEWB:%[0-9]+]] = alloc() : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, [[NEWA]][0] : memref<1xf32>
// CHECK-NEXT: affine.store %{{.*}}, [[NEWB]][0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load [[NEWA]][0] : memref<1xf32>
// CHECK-NEXT: affine.load [[NEWB]][0] : memref<1xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_fuse_first_and_second_loops() {
func @should_fuse_first_and_second_loops() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i1] : memref<10xf32>
affine.store %cf7, %b[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %c[%i2] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
// Should fuse first loop into the second (last loop should not be fused).
[mlir] Change dialect namespace loop->scf All ops of the SCF dialect now use the `scf.` prefix instead of `loop.`. This is a part of dialect renaming. Differential Revision: https://reviews.llvm.org/D79844
2020-05-13 18:12:30 +08:00
// Should create private memref '%2' for fused scf.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_not_fuse_would_create_cycle() {
func @should_not_fuse_would_create_cycle() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
// Set up the following dependences:
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// 1) loop0 -> loop1 on memref '%{{.*}}'
// 2) loop0 -> loop2 on memref '%{{.*}}'
// 3) loop1 -> loop2 on memref '%{{.*}}'
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i0] : memref<10xf32>
affine.store %cf7, %b[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%i1] : memref<10xf32>
%v1 = affine.load %c[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v2 = affine.load %b[%i2] : memref<10xf32>
affine.store %cf7, %c[%i2] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
// Should not fuse: fusing loop first loop into last would create a cycle.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
// CHECK-LABEL: func @should_fuse_producer_consumer() {
func @should_fuse_producer_consumer() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %m[%i2] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
// Fusing loop %i0 to %i2 would violate the WAW dependence between %i0 and
// %i1, but OK to fuse %i1 into %i2.
[mlir][NFC] Remove usernames and google bug numbers from TODO comments. These were largely leftover from when MLIR was a google project, and don't really follow LLVM guidelines.
2020-07-07 16:35:23 +08:00
// TODO: When the fusion pass is run to a fixed-point, it should
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
// fuse all three of these loop nests.
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK: alloc() : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
Fix Affine Loop Fusion test case reported on github. This CL utilizies the more robust fusion feasibility analysis being built out in LoopFusionUtils, which will eventually be used to replace the current affine loop fusion pass. PiperOrigin-RevId: 281112340
2019-11-19 03:20:03 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-LABEL: func @should_fuse_and_move_to_preserve_war_dep() {
func @should_fuse_and_move_to_preserve_war_dep() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i0] : memref<10xf32>
affine.store %v0, %b[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %b[%i2] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// Loops '%i1' and '%i2' have no dependences. We can fuse a slice of '%i0'
// into '%i2' if we move the fused loop nest before '%i1', which preserves
// the WAR dependence from load '%a' in '%i0' to the store '%a' in loop '%i1'.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
return
}
// -----
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-LABEL: func @should_fuse_if_top_level_access() {
func @should_fuse_if_top_level_access() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %m[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
%c0 = constant 4 : index
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %m[%c0] : memref<10xf32>
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// Top-level load to '%m' should prevent creating a private memref but
// loop nests should be fused and '%i0' should be removed.
// CHECK: %[[m:.*]] = alloc() : memref<10xf32>
// CHECK-NOT: alloc
// CHECK: affine.for %[[i1:.*]] = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %[[m]][%[[i1]]] : memref<10xf32>
// CHECK-NEXT: affine.load %[[m]][%[[i1]]] : memref<10xf32>
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-20 09:24:57 +08:00
// CHECK-NEXT: }
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK: affine.load %[[m]][%{{.*}}] : memref<10xf32>
return
}
// -----
// CHECK-LABEL: func @should_fuse_but_not_remove_src() {
func @should_fuse_but_not_remove_src() {
%m = alloc() : memref<100xf32>
%cf7 = constant 7.0 : f32
affine.for %i0 = 0 to 100 {
affine.store %cf7, %m[%i0] : memref<100xf32>
}
affine.for %i1 = 0 to 17 {
%v0 = affine.load %m[%i1] : memref<100xf32>
}
%v1 = affine.load %m[99] : memref<100xf32>
// Loop '%i0' and '%i1' should be fused but '%i0' shouldn't be removed to
// preserve the dependence with the top-level access.
// CHECK: affine.for %{{.*}} = 0 to 100 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<100xf32>
// CHECK-NEXT: }
// CHECK-NEXT: affine.for %{{.*}} = 0 to 17 {
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-NEXT: affine.load %{{.*}}[99] : memref<100xf32>
// CHECK-NEXT: return
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
return
}
// -----
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_fuse_no_top_level_access() {
func @should_fuse_no_top_level_access() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %m[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
#set0 = affine_set<(d0) : (1 == 0)>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_not_fuse_if_inst_at_top_level() {
func @should_not_fuse_if_inst_at_top_level() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %m[%i1] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
%c0 = constant 4 : index
NFC: Rename the 'if' operation in the AffineOps dialect to 'affine.if'. PiperOrigin-RevId: 240071154
2019-03-25 11:35:07 +08:00
affine.if #set0(%c0) {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
Recommit: Define a AffineOps dialect as well as an AffineIfOp operation. Replace all instances of IfInst with AffineIfOp and delete IfInst. PiperOrigin-RevId: 231342063
2019-01-29 13:23:53 +08:00
// Top-level IfOp should prevent fusion.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
return
}
// -----
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
#set0 = affine_set<(d0) : (1 == 0)>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
Eliminate extfunc/cfgfunc/mlfunc as a concept, and just use 'func' instead. The entire compiler now looks at structural properties of the function (e.g. does it have one block, does it contain an if/for stmt, etc) so the only thing holding up this difference is round tripping through the parser/printer syntax. Removing this shrinks the compile by ~140LOC. This is step 31/n towards merging instructions and statements. The last step is updating the docs, which I will do as a separate patch in order to split it from this mostly mechanical patch. PiperOrigin-RevId: 227540453
2019-01-03 02:20:00 +08:00
// CHECK-LABEL: func @should_not_fuse_if_inst_in_loop_nest() {
func @should_not_fuse_if_inst_in_loop_nest() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
%c4 = constant 4 : index
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
NFC: Rename the 'if' operation in the AffineOps dialect to 'affine.if'. PiperOrigin-RevId: 240071154
2019-03-25 11:35:07 +08:00
affine.if #set0(%c4) {
Adds a dependence check to test whether two accesses to the same memref access the same element. - Builds access functions and iterations domains for each access. - Builds dependence polyhedron constraint system which has equality constraints for equated access functions and inequality constraints for iteration domain loop bounds. - Runs elimination on the dependence polyhedron to test if no dependence exists between the accesses. - Adds a trivial LoopFusion transformation pass with a simple test policy to test dependence between accesses to the same memref in adjacent loops. - The LoopFusion pass will be extended in subsequent CLs. PiperOrigin-RevId: 219630898
2018-11-01 22:26:00 +08:00
}
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %m[%i1] : memref<10xf32>
Adds a dependence check to test whether two accesses to the same memref access the same element. - Builds access functions and iterations domains for each access. - Builds dependence polyhedron constraint system which has equality constraints for equated access functions and inequality constraints for iteration domain loop bounds. - Runs elimination on the dependence polyhedron to test if no dependence exists between the accesses. - Adds a trivial LoopFusion transformation pass with a simple test policy to test dependence between accesses to the same memref in adjacent loops. - The LoopFusion pass will be extended in subsequent CLs. PiperOrigin-RevId: 219630898
2018-11-01 22:26:00 +08:00
}
Recommit: Define a AffineOps dialect as well as an AffineIfOp operation. Replace all instances of IfInst with AffineIfOp and delete IfInst. PiperOrigin-RevId: 231342063
2019-01-29 13:23:53 +08:00
// IfOp in ForInst should prevent fusion.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[mlir][ASM] Refactor how attribute/type aliases are specified. Previously they were separated into "instance" and "kind" aliases, and also required that the dialect know ahead of time all of the instances that would have a corresponding alias. This approach was very clunky and not ergonomic to interact with. The new approach is to provide the dialect with an instance of an attribute/type to provide an alias for, fully replacing the original split approach. Differential Revision: https://reviews.llvm.org/D89354
2020-10-30 15:30:59 +08:00
// CHECK-NEXT: affine.if #set(%{{.*}}) {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
return
}
// -----
Update replaceAllMemRefUsesWith to generate single result affine_apply's for index remapping - generate a sequence of single result affine_apply's for the index remapping (instead of one multi result affine_apply) - update dma-generate and loop-fusion test cases; while on this, change test cases to use single result affine apply ops - some fusion comment fix/cleanup PiperOrigin-RevId: 230985830
2019-01-26 08:00:50 +08:00
// CHECK-LABEL: func @permute_and_fuse() {
func @permute_and_fuse() {
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
%m = alloc() : memref<10x20x30xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
affine.for %i1 = 0 to 20 {
affine.for %i2 = 0 to 30 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0, %i1, %i2] : memref<10x20x30xf32>
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
Adds a dependence check to test whether two accesses to the same memref access the same element. - Builds access functions and iterations domains for each access. - Builds dependence polyhedron constraint system which has equality constraints for equated access functions and inequality constraints for iteration domain loop bounds. - Runs elimination on the dependence polyhedron to test if no dependence exists between the accesses. - Adds a trivial LoopFusion transformation pass with a simple test policy to test dependence between accesses to the same memref in adjacent loops. - The LoopFusion pass will be extended in subsequent CLs. PiperOrigin-RevId: 219630898
2018-11-01 22:26:00 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i3 = 0 to 30 {
affine.for %i4 = 0 to 10 {
affine.for %i5 = 0 to 20 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %m[%i4, %i5, %i3] : memref<10x20x30xf32>
Update replaceAllMemRefUsesWith to generate single result affine_apply's for index remapping - generate a sequence of single result affine_apply's for the index remapping (instead of one multi result affine_apply) - update dma-generate and loop-fusion test cases; while on this, change test cases to use single result affine apply ops - some fusion comment fix/cleanup PiperOrigin-RevId: 230985830
2019-01-26 08:00:50 +08:00
"foo"(%v0) : (f32) -> ()
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
}
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 30 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 20 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0, 0] : memref<1x1x1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0, 0] : memref<1x1x1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: "foo"(%{{.*}}) : (f32) -> ()
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
Adds a dependence check to test whether two accesses to the same memref access the same element. - Builds access functions and iterations domains for each access. - Builds dependence polyhedron constraint system which has equality constraints for equated access functions and inequality constraints for iteration domain loop bounds. - Runs elimination on the dependence polyhedron to test if no dependence exists between the accesses. - Adds a trivial LoopFusion transformation pass with a simple test policy to test dependence between accesses to the same memref in adjacent loops. - The LoopFusion pass will be extended in subsequent CLs. PiperOrigin-RevId: 219630898
2018-11-01 22:26:00 +08:00
return
Loop Fusion pass update: introduce utilities to perform generalized loop fusion based on slicing; encompasses standard loop fusion. *) Adds simple greedy fusion algorithm to drive experimentation. This algorithm greedily fuses loop nests with single-writer/single-reader memref dependences to improve locality. *) Adds support for fusing slices of a loop nest computation: fusing one loop nest into another by adjusting the source loop nest's iteration bounds (after it is fused into the destination loop nest). This is accomplished by solving for the source loop nest's IVs in terms of the destination loop nests IVs and symbols using the dependece polyhedron, then creating AffineMaps of these functions for the loop bounds of the fused source loop. *) Adds utility function 'insertMemRefComputationSlice' which computes and inserts computation slice from loop nest surrounding a source memref access into the loop nest surrounding the destingation memref access. *) Adds FlatAffineConstraints::toAffineMap function which returns and AffineMap which represents an equality contraint where one dimension identifier is represented as a function of all others in the equality constraint. *) Adds multiple fusion unit tests. PiperOrigin-RevId: 225842944
2018-12-18 01:57:14 +08:00
}
Computation slice update: adds parameters to insertBackwardComputationSlice which specify the source loop nest depth at which to perform iteration space slicing, and the destination loop nest depth at which to insert the compution slice. Updates LoopFusion pass to take these parameters as command line flags for experimentation. PiperOrigin-RevId: 226514297
2018-12-22 03:06:23 +08:00
// -----
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-DAG: [[$MAP0:#map[0-9]+]] = affine_map<(d0, d1) -> (d0 * 4 + d1)>
// CHECK-DAG: [[$MAP1:#map[0-9]+]] = affine_map<(d0) -> (d0 floordiv 4)>
// CHECK-DAG: [[$MAP2:#map[0-9]+]] = affine_map<(d0) -> (d0 mod 4)>
Complete TODOs / cleanup for loop-fusion utility - this is CL 1/2 that does a clean up and gets rid of one limitation in an underlying method - as a result, fusion works for more cases. - fix bugs/incomplete impl. in toAffineMapFromEq - fusing across rank changing reshapes for example now just works For eg. given a rank 1 memref to rank 2 memref reshape (64 -> 8 x 8) like this, -loop-fusion -memref-dataflow-opt now completely fuses and inlines/store-forward to get rid of the temporary: INPUT // Rank 1 -> Rank 2 reshape for %i0 = 0 to 64 { %v = load %A[%i0] store %v, %B[%i0 floordiv 8, i0 mod 8] } for %i1 = 0 to 8 for %i2 = 0 to 8 %w = load %B[%i1, i2] "foo"(%w) : (f32) -> () OUTPUT $ mlir-opt -loop-fusion -memref-dataflow-opt fuse_reshape.mlir #map0 = (d0, d1) -> (d0 * 8 + d1) mlfunc @fuse_reshape(%arg0: memref<64xf32>) { for %i0 = 0 to 8 { for %i1 = 0 to 8 { %0 = affine_apply #map0(%i0, %i1) %1 = load %arg0[%0] : memref<64xf32> "foo"(%1) : (f32) -> () } } } AFAIK, there is no polyhedral tool / compiler that can perform such fusion - because it's not really standard loop fusion, but possible through a generalized slicing-based approach such as ours. PiperOrigin-RevId: 227918338
2019-01-05 06:52:21 +08:00
// Reshape from a 64 x f32 to 16 x 4 x f32.
// CHECK-LABEL: func @fuse_reshape_64_16_4
func @fuse_reshape_64_16_4(%in : memref<64xf32>) {
%out = alloc() : memref<16x4xf32>
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 64 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v = affine.load %in[%i0] : memref<64xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
affine.store %v, %out[%i0 floordiv 4, %i0 mod 4] : memref<16x4xf32>
Complete TODOs / cleanup for loop-fusion utility - this is CL 1/2 that does a clean up and gets rid of one limitation in an underlying method - as a result, fusion works for more cases. - fix bugs/incomplete impl. in toAffineMapFromEq - fusing across rank changing reshapes for example now just works For eg. given a rank 1 memref to rank 2 memref reshape (64 -> 8 x 8) like this, -loop-fusion -memref-dataflow-opt now completely fuses and inlines/store-forward to get rid of the temporary: INPUT // Rank 1 -> Rank 2 reshape for %i0 = 0 to 64 { %v = load %A[%i0] store %v, %B[%i0 floordiv 8, i0 mod 8] } for %i1 = 0 to 8 for %i2 = 0 to 8 %w = load %B[%i1, i2] "foo"(%w) : (f32) -> () OUTPUT $ mlir-opt -loop-fusion -memref-dataflow-opt fuse_reshape.mlir #map0 = (d0, d1) -> (d0 * 8 + d1) mlfunc @fuse_reshape(%arg0: memref<64xf32>) { for %i0 = 0 to 8 { for %i1 = 0 to 8 { %0 = affine_apply #map0(%i0, %i1) %1 = load %arg0[%0] : memref<64xf32> "foo"(%1) : (f32) -> () } } } AFAIK, there is no polyhedral tool / compiler that can perform such fusion - because it's not really standard loop fusion, but possible through a generalized slicing-based approach such as ours. PiperOrigin-RevId: 227918338
2019-01-05 06:52:21 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 16 {
affine.for %i2 = 0 to 4 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%w = affine.load %out[%i1, %i2] : memref<16x4xf32>
Complete TODOs / cleanup for loop-fusion utility - this is CL 1/2 that does a clean up and gets rid of one limitation in an underlying method - as a result, fusion works for more cases. - fix bugs/incomplete impl. in toAffineMapFromEq - fusing across rank changing reshapes for example now just works For eg. given a rank 1 memref to rank 2 memref reshape (64 -> 8 x 8) like this, -loop-fusion -memref-dataflow-opt now completely fuses and inlines/store-forward to get rid of the temporary: INPUT // Rank 1 -> Rank 2 reshape for %i0 = 0 to 64 { %v = load %A[%i0] store %v, %B[%i0 floordiv 8, i0 mod 8] } for %i1 = 0 to 8 for %i2 = 0 to 8 %w = load %B[%i1, i2] "foo"(%w) : (f32) -> () OUTPUT $ mlir-opt -loop-fusion -memref-dataflow-opt fuse_reshape.mlir #map0 = (d0, d1) -> (d0 * 8 + d1) mlfunc @fuse_reshape(%arg0: memref<64xf32>) { for %i0 = 0 to 8 { for %i1 = 0 to 8 { %0 = affine_apply #map0(%i0, %i1) %1 = load %arg0[%0] : memref<64xf32> "foo"(%1) : (f32) -> () } } } AFAIK, there is no polyhedral tool / compiler that can perform such fusion - because it's not really standard loop fusion, but possible through a generalized slicing-based approach such as ours. PiperOrigin-RevId: 227918338
2019-01-05 06:52:21 +08:00
"foo"(%w) : (f32) -> ()
}
}
return
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} =
// CHECK-NEXT: affine.for %{{.*}} =
Complete TODOs / cleanup for loop-fusion utility - this is CL 1/2 that does a clean up and gets rid of one limitation in an underlying method - as a result, fusion works for more cases. - fix bugs/incomplete impl. in toAffineMapFromEq - fusing across rank changing reshapes for example now just works For eg. given a rank 1 memref to rank 2 memref reshape (64 -> 8 x 8) like this, -loop-fusion -memref-dataflow-opt now completely fuses and inlines/store-forward to get rid of the temporary: INPUT // Rank 1 -> Rank 2 reshape for %i0 = 0 to 64 { %v = load %A[%i0] store %v, %B[%i0 floordiv 8, i0 mod 8] } for %i1 = 0 to 8 for %i2 = 0 to 8 %w = load %B[%i1, i2] "foo"(%w) : (f32) -> () OUTPUT $ mlir-opt -loop-fusion -memref-dataflow-opt fuse_reshape.mlir #map0 = (d0, d1) -> (d0 * 8 + d1) mlfunc @fuse_reshape(%arg0: memref<64xf32>) { for %i0 = 0 to 8 { for %i1 = 0 to 8 { %0 = affine_apply #map0(%i0, %i1) %1 = load %arg0[%0] : memref<64xf32> "foo"(%1) : (f32) -> () } } } AFAIK, there is no polyhedral tool / compiler that can perform such fusion - because it's not really standard loop fusion, but possible through a generalized slicing-based approach such as ours. PiperOrigin-RevId: 227918338
2019-01-05 06:52:21 +08:00
// CHECK-NOT: for
// CHECK: }
// CHECK-NEXT: }
// CHECK-NEXT: return
}
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
// -----
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-DAG: [[$MAP0:#map[0-9]+]] = affine_map<(d0) -> (d0 floordiv 4)>
// CHECK-DAG: [[$MAP1:#map[0-9]+]] = affine_map<(d0) -> (d0 mod 4)>
// CHECK-DAG: [[$MAP2:#map[0-9]+]] = affine_map<(d0, d1) -> (d0 * 4 + d1)>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
// Reshape a 16x4xf32 to 64xf32.
// CHECK-LABEL: func @fuse_reshape_16_4_64
func @fuse_reshape_16_4_64() {
%in = alloc() : memref<16x4xf32>
%out = alloc() : memref<64xf32>
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 16 {
affine.for %i1 = 0 to 4 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v = affine.load %in[%i0, %i1] : memref<16x4xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
affine.store %v, %out[4*%i0 + %i1] : memref<64xf32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 64 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%w = affine.load %out[%i2] : memref<64xf32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
"foo"(%w) : (f32) -> ()
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 64 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.apply [[$MAP0]](%{{.*}})
// CHECK-NEXT: affine.apply [[$MAP1]](%{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<16x4xf32>
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.apply [[$MAP2]](%{{.*}}, %{{.*}})
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: "foo"(%{{.*}}) : (f32) -> ()
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// All three loop nests below (6-d one, 2-d one, 2-d one is fused into a single
// 2-d loop nest).
func @R6_to_R2_reshape_square() -> memref<64x9xi32> {
%in = alloc() : memref<2x2x3x3x16x1xi32>
%out = alloc() : memref<64x9xi32>
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
%live_out = alloc() : memref<64x9xi32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
Update stale / target-specific information in comments - NFC PiperOrigin-RevId: 229800834
2019-01-18 04:51:48 +08:00
// Initialize input.
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 2 {
affine.for %i1 = 0 to 2 {
affine.for %i2 = 0 to 3 {
affine.for %i3 = 0 to 3 {
affine.for %i4 = 0 to 16 {
affine.for %i5 = 0 to 1 {
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
%val = "foo"(%i0, %i1, %i2, %i3, %i4, %i5) : (index, index, index, index, index, index) -> i32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %val, %in[%i0, %i1, %i2, %i3, %i4, %i5] : memref<2x2x3x3x16x1xi32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
}
}
}
}
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %ii = 0 to 64 {
affine.for %jj = 0 to 9 {
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
// Convert output coordinates to linear index.
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
%a0 = affine.apply affine_map<(d0, d1) -> (d0 * 9 + d1)> (%ii, %jj)
%0 = affine.apply affine_map<(d0) -> (d0 floordiv (2 * 3 * 3 * 16 * 1))>(%a0)
%1 = affine.apply affine_map<(d0) -> ((d0 mod 288) floordiv (3 * 3 * 16 * 1))>(%a0)
%2 = affine.apply affine_map<(d0) -> (((d0 mod 288) mod 144) floordiv (3 * 16 * 1))>(%a0)
%3 = affine.apply affine_map<(d0) -> ((((d0 mod 288) mod 144) mod 48) floordiv (16 * 1))>(%a0)
%4 = affine.apply affine_map<(d0) -> ((((d0 mod 288) mod 144) mod 48) mod 16)>(%a0)
%5 = affine.apply affine_map<(d0) -> (((((d0 mod 144) mod 144) mod 48) mod 16) mod 1)>(%a0)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v = affine.load %in[%0, %1, %2, %3, %4, %5] : memref<2x2x3x3x16x1xi32>
affine.store %v, %out[%ii, %jj] : memref<64x9xi32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i = 0 to 64 {
affine.for %j = 0 to 9 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%a = affine.load %out[%i, %j] : memref<64x9xi32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
%b = muli %a, %a : i32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %b, %live_out[%i, %j] : memref<64x9xi32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
}
}
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
return %live_out : memref<64x9xi32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
}
// Everything above is fused to a single 2-d loop nest, and the 6-d tensor %in
// is eliminated if -memref-dataflow-opt is also supplied.
//
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-DAG: [[$MAP0:#map[0-9]+]] = affine_map<(d0, d1) -> ((d0 * 9 + d1) floordiv 288)>
// CHECK-DAG: [[$MAP1:#map[0-9]+]] = affine_map<(d0, d1) -> (((d0 * 9 + d1) mod 288) floordiv 144)>
// CHECK-DAG: [[$MAP2:#map[0-9]+]] = affine_map<(d0, d1) -> ((((d0 * 9 + d1) mod 288) mod 144) floordiv 48)>
// CHECK-DAG: [[$MAP3:#map[0-9]+]] = affine_map<(d0, d1) -> (((((d0 * 9 + d1) mod 288) mod 144) mod 48) floordiv 16)>
// CHECK-DAG: [[$MAP4:#map[0-9]+]] = affine_map<(d0, d1) -> (((((d0 * 9 + d1) mod 288) mod 144) mod 48) mod 16)>
// CHECK-DAG: [[$MAP11:#map[0-9]+]] = affine_map<(d0, d1) -> (d0 * 9 + d1)>
// CHECK-DAG: [[$MAP12:#map[0-9]+]] = affine_map<(d0) -> (d0 floordiv 288)>
// CHECK-DAG: [[$MAP13:#map[0-9]+]] = affine_map<(d0) -> ((d0 mod 288) floordiv 144)>
// CHECK-DAG: [[$MAP14:#map[0-9]+]] = affine_map<(d0) -> (((d0 mod 288) mod 144) floordiv 48)>
// CHECK-DAG: [[$MAP15:#map[0-9]+]] = affine_map<(d0) -> ((((d0 mod 288) mod 144) mod 48) floordiv 16)>
// CHECK-DAG: [[$MAP16:#map[0-9]+]] = affine_map<(d0) -> ((((d0 mod 288) mod 144) mod 48) mod 16)>
// CHECK-DAG: [[$MAP17:#map[0-9]+]] = affine_map<(d0) -> (0)>
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
Update tests using affine maps to not rely on specific map numbers in the output IR. This is necessary to remove the dependency on ForInst not numbering the AffineMap bounds it has custom formatting for. PiperOrigin-RevId: 231634812
2019-01-31 03:33:32 +08:00
//
// CHECK-LABEL: func @R6_to_R2_reshape
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK: alloc() : memref<1x2x3x3x16x1xi32>
// CHECK: alloc() : memref<1x1xi32>
// CHECK: alloc() : memref<64x9xi32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 64 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 9 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.apply [[$MAP0]](%{{.*}}, %{{.*}})
// CHECK-NEXT: affine.apply [[$MAP1]](%{{.*}}, %{{.*}})
// CHECK-NEXT: affine.apply [[$MAP2]](%{{.*}}, %{{.*}})
// CHECK-NEXT: affine.apply [[$MAP3]](%{{.*}}, %{{.*}})
// CHECK-NEXT: affine.apply [[$MAP4]](%{{.*}}, %{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: "foo"(%{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}) : (index, index, index, index, index, index) -> i32
Extend map canonicalization to propagate constant operands - extend canonicalizeMapAndOperands to propagate constant operands into the map's expressions (and thus drop those operands). - canonicalizeMapAndOperands previously only dropped duplicate and unused operands; however, operands that were constants were retained. This change makes IR maps/expressions generated by various utilities/passes even simpler; also makes some of the test checks more accurate and simpler -- for eg., 0' instead of symbol(%{{.*}}). Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#107 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/107 from bondhugula:canonicalize-maps c889a51486d14fbf7db489f224f881e7e1ff7d72 PiperOrigin-RevId: 266085289
2019-08-29 16:13:01 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, ((%{{.*}} * 9 + %{{.*}}) mod 288) floordiv 144, (((%{{.*}} * 9 + %{{.*}}) mod 288) mod 144) floordiv 48, ((((%{{.*}} * 9 + %{{.*}}) mod 288) mod 144) mod 48) floordiv 16, ((((%{{.*}} * 9 + %{{.*}}) mod 288) mod 144) mod 48) mod 16, 0] : memref<1x2x3x3x16x1xi32>
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.apply [[$MAP11]](%{{.*}}, %{{.*}})
// CHECK-NEXT: affine.apply [[$MAP12]](%{{.*}})
// CHECK-NEXT: affine.apply [[$MAP13]](%{{.*}})
// CHECK-NEXT: affine.apply [[$MAP14]](%{{.*}})
// CHECK-NEXT: affine.apply [[$MAP15]](%{{.*}})
// CHECK-NEXT: affine.apply [[$MAP16]](%{{.*}})
// CHECK-NEXT: affine.apply [[$MAP17]](%{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, ((%{{.*}} * 9 + %{{.*}}) mod 288) floordiv 144, (((%{{.*}} * 9 + %{{.*}}) mod 288) mod 144) floordiv 48, ((((%{{.*}} * 9 + %{{.*}}) mod 288) mod 144) mod 48) floordiv 16, ((((%{{.*}} * 9 + %{{.*}}) mod 288) mod 144) mod 48) mod 16, 0] : memref<1x2x3x3x16x1xi32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0] : memref<1x1xi32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0] : memref<1x1xi32>
// CHECK-NEXT: muli %{{.*}}, %{{.*}} : i32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<64x9xi32>
Fix test cases that were accessing out of bounds to start with (b/123072438) - detected with memref-bound-check - fixes b/123072438; while on this, fix another test case which was reported out of bounds PiperOrigin-RevId: 229978187
2019-01-19 04:55:46 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: return %{{.*}} : memref<64x9xi32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
// -----
// CHECK-LABEL: func @fuse_symbolic_bounds
func @fuse_symbolic_bounds(%M : index, %N : index) {
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
%N_plus_5 = affine.apply affine_map<(d0) -> (d0 + 5)>(%N)
Fix test cases that were accessing out of bounds to start with (b/123072438) - detected with memref-bound-check - fixes b/123072438; while on this, fix another test case which was reported out of bounds PiperOrigin-RevId: 229978187
2019-01-19 04:55:46 +08:00
%m = alloc(%M, %N_plus_5) : memref<? x ? x f32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
%c0 = constant 0.0 : f32
%s = constant 5 : index
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to %M {
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
affine.for %i1 = 0 to affine_map<(d0) -> (d0 + 5)> (%N) {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %c0, %m[%i0, %i1] : memref<? x ? x f32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to %M {
affine.for %i3 = 0 to %N {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
%v = affine.load %m[%i2, %i3 + symbol(%s)] : memref<? x ? x f32>
Extend loop-fusion's slicing utility + other fixes / updates - refactor toAffineFromEq and the code surrounding it; refactor code into FlatAffineConstraints::getSliceBounds - add FlatAffineConstraints methods to detect identifiers as mod's and div's of other identifiers - add FlatAffineConstraints::getConstantLower/UpperBound - Address b/122118218 (don't assert on invalid fusion depths cmdline flags - instead, don't do anything; change cmdline flags src-loop-depth -> fusion-src-loop-depth - AffineExpr/Map print method update: don't fail on null instances (since we have a wrapper around a pointer, it's avoidable); rationale: dump/print methods should never fail if possible. - Update memref-dataflow-opt to add an optimization to avoid a unnecessary call to IsRangeOneToOne when it's trivially going to be true. - Add additional test cases to exercise the new support - update a few existing test cases since the maps are now generated uniformly with all destination loop operands appearing for the backward slice - Fix projectOut - fix wrong range for getBestElimCandidate. - Fix for getConstantBoundOnDimSize() - didn't show up in any test cases since we didn't have any non-hyperrectangular ones. PiperOrigin-RevId: 228265152
2019-01-08 09:34:26 +08:00
}
}
return
}
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// -----
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-LABEL: func @should_fuse_reduction_at_depth_of_one
func @should_fuse_reduction_at_depth_of_one() {
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
%a = alloc() : memref<10x100xf32>
%b = alloc() : memref<10xf32>
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
affine.for %i1 = 0 to 100 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %b[%i0] : memref<10xf32>
%v1 = affine.load %a[%i0, %i1] : memref<10x100xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
%v2 = "maxf"(%v0, %v1) : (f32, f32) -> f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %v2, %b[%i0] : memref<10xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
affine.for %i3 = 0 to 100 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v3 = affine.load %b[%i2] : memref<10xf32>
%v4 = affine.load %a[%i2, %i3] : memref<10x100xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
%v5 = subf %v4, %v3 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %v5, %b[%i2] : memref<10xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
}
}
// This test should fuse the src reduction loop at depth 1 in the destination
// loop nest, which improves locality and enables subsequence passes to
// decrease the reduction memref size and possibly place it in a faster
// memory space.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 100 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x100xf32>
// CHECK-NEXT: "maxf"(%{{.*}}, %{{.*}}) : (f32, f32) -> f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 100 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x100xf32>
// CHECK-NEXT: subf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @should_fuse_at_src_depth1_and_dst_depth1
func @should_fuse_at_src_depth1_and_dst_depth1() {
%a = alloc() : memref<100x16xf32>
%b = alloc() : memref<100x16xf32>
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 100 {
affine.for %i1 = 0 to 16 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i0, %i1] : memref<100x16xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
"op0"(%v0) : (f32) -> ()
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 16 {
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
%v1 = "op1"() : () -> (f32)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %v1, %b[%i0, %i2] : memref<100x16xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i3 = 0 to 100 {
affine.for %i4 = 0 to 16 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v2 = affine.load %b[%i3, %i4] : memref<100x16xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
"op2"(%v2) : (f32) -> ()
}
}
Fix 'the the' typo. PiperOrigin-RevId: 281501234
2019-11-20 21:37:49 +08:00
// We can slice iterations of the '%i0' and '%i1' loops in the source
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// loop nest, but slicing at depth 2 and inserting the slice in the
// destination loop nest at depth2 causes extra computation. Instead,
// the fusion algorithm should detect that the source loop should be sliced
// at depth 1 and the slice should be inserted at depth 1.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 100 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<100x16xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: "op0"(%{{.*}}) : (f32) -> ()
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
// CHECK-NEXT: %{{.*}} = "op1"() : () -> f32
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, %{{.*}}] : memref<1x16xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, %{{.*}}] : memref<1x16xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: "op2"(%{{.*}}) : (f32) -> ()
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
[mlir][AsmPrinter] Refactor printing to only print aliases for attributes/types that will exist in the output. This revision refactors the way that attributes/types are considered when generating aliases. Instead of considering all of the attributes/types of every operation, we perform a "fake" print step that prints the operations using a dummy printer to collect the attributes and types that would actually be printed during the real process. This removes a lot of attributes/types from consideration that generally won't end up in the final output, e.g. affine map attributes in an `affine.apply`/`affine.for`. This resolves a long standing TODO w.r.t aliases, and helps to have a much cleaner textual output format. As a datapoint to the latter, as part of this change several tests were identified as testing for the presence of attributes aliases that weren't actually referenced by the custom form of any operation. To ensure that this wouldn't cause a large degradation in compile time due to the second full print, I benchmarked this change on a very large module with a lot of operations(The file is ~673M/~4.7 million lines long). This file before this change take ~6.9 seconds to print in the custom form, and ~7 seconds after this change. In the custom assembly case, this added an average of a little over ~100 miliseconds to the compile time. This increase was due to the way that argument attributes on functions are structured and how they get printed; i.e. with a better representation the negative impact here can be greatly decreased. When printing in the generic form, this revision had no observable impact on the compile time. This benchmarking leads me to believe that the impact of this change on compile time w.r.t printing is closely related to `print` methods that perform a lot of additional/complex processing outside of the OpAsmPrinter. Differential Revision: https://reviews.llvm.org/D90512
2020-11-10 13:50:31 +08:00
// CHECK: [[$MAP0:#map[0-9]*]] = affine_map<(d0, d1) -> (d0 * 10 + d1)>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-LABEL: func @should_fuse_src_depth1_at_dst_depth2
func @should_fuse_src_depth1_at_dst_depth2() {
%a = alloc() : memref<100xf32>
%c0 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 100 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %c0, %a[%i0] : memref<100xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
affine.for %i2 = 0 to 10 {
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
%a0 = affine.apply affine_map<(d0, d1) -> (d0 * 10 + d1)> (%i1, %i2)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%a0] : memref<100xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
}
}
// The source loop nest slice loop bound is a function of both destination
// loop IVs, so we should slice at depth 1 and insert the slice at depth 2.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.apply [[$MAP0]](%{{.*}}, %{{.*}})
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.apply [[$MAP0]](%{{.*}}, %{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @fusion_at_depth0_not_currently_supported
func @fusion_at_depth0_not_currently_supported() {
%0 = alloc() : memref<10xf32>
%c0 = constant 0 : index
%cst = constant 0.000000e+00 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cst, %0[%i0] : memref<10xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%1 = affine.load %0[%c0] : memref<10xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
}
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// NOTE: Should shrink memref size to 1 element access by load in dst loop
// nest, and make the store in the slice store to the same element.
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
Extend map canonicalization to propagate constant operands - extend canonicalizeMapAndOperands to propagate constant operands into the map's expressions (and thus drop those operands). - canonicalizeMapAndOperands previously only dropped duplicate and unused operands; however, operands that were constants were retained. This change makes IR maps/expressions generated by various utilities/passes even simpler; also makes some of the test checks more accurate and simpler -- for eg., 0' instead of symbol(%{{.*}}). Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#107 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/107 from bondhugula:canonicalize-maps c889a51486d14fbf7db489f224f881e7e1ff7d72 PiperOrigin-RevId: 266085289
2019-08-29 16:13:01 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Allocate private/local buffers for slices accurately during fusion - the size of the private memref created for the slice should be based on the memref region accessed at the depth at which the slice is being materialized, i.e., symbolic in the outer IVs up until that depth, as opposed to the region accessed based on the entire domain. - leads to a significant contraction of the temporary / intermediate memref whenever the memref isn't reduced to a single scalar (through store fwd'ing). Other changes - update to promoteIfSingleIteration - avoid introducing unnecessary identity map affine_apply from IV; makes it much easier to write and read test cases and pass output for all passes that use promoteIfSingleIteration; loop-fusion test cases become much simpler - fix replaceAllMemrefUsesWith bug that was exposed by the above update - 'domInstFilter' could be one of the ops erased due to a memref replacement in it. - fix getConstantBoundOnDimSize bug: a division by the coefficient of the identifier was missing (the latter need not always be 1); add lbFloorDivisors output argument - rename getBoundingConstantSizeAndShape -> getConstantBoundingSizeAndShape PiperOrigin-RevId: 230405218
2019-01-23 05:58:52 +08:00
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-LABEL: func @should_fuse_deep_loop_nests
func @should_fuse_deep_loop_nests() {
%0 = alloc() : memref<2x2x3x3x16x10xf32, 2>
%1 = alloc() : memref<2x2x3x3x16x10xf32, 2>
%2 = alloc() : memref<3x3x3x3x16x10xf32, 2>
%c0 = constant 0 : index
%c1 = constant 1 : index
%c1_0 = constant 1 : index
%cst = constant 0.000000e+00 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 2 {
affine.for %i1 = 0 to 2 {
affine.for %i2 = 0 to 3 {
affine.for %i3 = 0 to 3 {
affine.for %i4 = 0 to 16 {
affine.for %i5 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%3 = affine.load %0[%i0, %i1, %i2, %i3, %i4, %i5]
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
: memref<2x2x3x3x16x10xf32, 2>
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i6 = 0 to 16 {
affine.for %i7 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cst, %1[%i0, %i1, %i2, %i3, %i6, %i7]
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
: memref<2x2x3x3x16x10xf32, 2>
}
}
}
}
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i8 = 0 to 3 {
affine.for %i9 = 0 to 3 {
affine.for %i10 = 0 to 2 {
affine.for %i11 = 0 to 2 {
affine.for %i12 = 0 to 3 {
affine.for %i13 = 0 to 3 {
affine.for %i14 = 0 to 2 {
affine.for %i15 = 0 to 2 {
affine.for %i16 = 0 to 16 {
affine.for %i17 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%5 = affine.load %0[%i14, %i15, %i12, %i13, %i16, %i17]
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
: memref<2x2x3x3x16x10xf32, 2>
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i18 = 0 to 16 {
affine.for %i19 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%6 = affine.load %1[%i10, %i11, %i8, %i9, %i18, %i19]
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
: memref<2x2x3x3x16x10xf32, 2>
}
}
}
}
}
}
}
}
}
}
// The first four loops of the source loop nest can be sliced with iteration
// bounds which are a function of the first four loops of destination loop nest,
// where the destination loops nests have been interchanged.
Update replaceAllMemRefUsesWith to generate single result affine_apply's for index remapping - generate a sequence of single result affine_apply's for the index remapping (instead of one multi result affine_apply) - update dma-generate and loop-fusion test cases; while on this, change test cases to use single result affine apply ops - some fusion comment fix/cleanup PiperOrigin-RevId: 230985830
2019-01-26 08:00:50 +08:00
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<1x1x1x1x16x10xf32, 2>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 3 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 3 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 2 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 2 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 3 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 3 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}] : memref<2x2x3x3x16x10xf32, 2>
Update fusion cost model + some additional infrastructure and debug information for -loop-fusion - update fusion cost model to fuse while tolerating a certain amount of redundant computation; add cl option -fusion-compute-tolerance evaluate memory footprint and intermediate memory reduction - emit debug info from -loop-fusion showing what was fused and why - introduce function to compute memory footprint for a loop nest - getMemRefRegion readability update - NFC PiperOrigin-RevId: 230541857
2019-01-24 01:16:24 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0, 0, 0, %{{.*}}, %{{.*}}] : memref<1x1x1x1x16x10xf32, 2>
Update fusion cost model + some additional infrastructure and debug information for -loop-fusion - update fusion cost model to fuse while tolerating a certain amount of redundant computation; add cl option -fusion-compute-tolerance evaluate memory footprint and intermediate memory reduction - emit debug info from -loop-fusion showing what was fused and why - introduce function to compute memory footprint for a loop nest - getMemRefRegion readability update - NFC PiperOrigin-RevId: 230541857
2019-01-24 01:16:24 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 2 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 2 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}] : memref<2x2x3x3x16x10xf32, 2>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0, 0, 0, %{{.*}}, %{{.*}}] : memref<1x1x1x1x16x10xf32, 2>
LoopFusion: automate selection of source loop nest slice depth and destination loop nest insertion depth based on a simple cost model (cost model can be extended/replaced at a later time). *) LoopFusion: Adds fusion cost function which compares the cost of the fused loop nest, with the cost of the two unfused loop nests to determine if it is profitable to fuse the candidate loop nests. The fusion cost function is run for various combinations for src/dst loop depths attempting find the minimum cost setting for src/dst loop depths which does not increase the computational cost when the loop nests are fused. Combinations of src/dst loop depth are evaluated attempting to maximize loop depth (i.e. take a bigger computation slice from the source loop nest, and insert it deeper in the destination loop nest for better locality). *) LoopFusion: Adds utility to compute op instance count for loop nests, sliced loop nests, and to compute the cost of a loop nest fused with another sliced loop nest. *) LoopFusion: canonicalizes slice bound AffineMaps (and updates related tests). *) Analysis::Utils: Splits getBackwardComputationSlice into two functions: one which calculates and returns the slice loop bounds for analysis by LoopFusion, and the other for insertion of the computation slice (ones fusion has calculated the min-cost src/dst loop depths). *) Test: Adds multiple unit tests to test the new functionality. PiperOrigin-RevId: 229219757
2019-01-15 03:26:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// -----
// CHECK-LABEL: func @should_fuse_at_depth1_and_reduce_slice_trip_count
func @should_fuse_at_depth1_and_reduce_slice_trip_count() {
%a = alloc() : memref<4x256xf32>
%b = alloc() : memref<4x256xf32>
%c0 = constant 0 : index
%cf0 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 4 {
affine.for %i1 = 0 to 256 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %b[%i0, %i1] : memref<4x256xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 256 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf0, %a[%i0, %i2] : memref<4x256xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %d0 = 0 to 4 {
affine.for %d1 = 0 to 16 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %a[%d0, %d1] : memref<4x256xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
}
// The cost of fusing at depth 2 is greater than the cost of fusing at depth 1
// for two reasons:
// 1) Inserting the unsliceable src loop %i1 to a higher depth removes
// redundant computation and reduces costs.
// 2) Inserting the sliceable src loop %i2 at depth 1, we can still reduce
// its trip count to 16 (from 256) reducing costs.
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// NOTE: the size of the private memref created for the fused loop nest
// is reduced from the original shape from 4x256 to 4x16 because of the
// data accessed by the load.
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<1x16xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 4 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 256 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<4x256xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, %{{.*}}] : memref<1x16xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, %{{.*}}] : memref<1x16xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @should_fuse_at_depth1_with_trip_count_20
func @should_fuse_at_depth1_with_trip_count_20() {
%a = alloc() : memref<100xf32>
%c0 = constant 0 : index
%cf0 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 100 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf0, %a[%i0]: memref<100xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 5 {
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i2]: memref<100xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i3 = 0 to 10 {
affine.for %i4 = 0 to 20 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %a[%i4]: memref<100xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
}
}
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// NOTE: The size of the private memref created for fusion is shrunk to 20xf32
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<20xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 5 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 20 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<20xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<20xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 20 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<20xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @should_fuse_at_depth1_with_trip_count_19
func @should_fuse_at_depth1_with_trip_count_19() {
%a = alloc() : memref<100xf32>
%c0 = constant 0 : index
%cf0 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 100 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf0, %a[%i0]: memref<100xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 5 {
affine.for %i2 = 0 to 19 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i2]: memref<100xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i3 = 0 to 10 {
affine.for %i4 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %a[%i4]: memref<100xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
}
}
}
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// NOTE: The size of the private memref created for fusion is shrunk to 19xf32
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<19xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 5 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 19 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<19xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 19 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<19xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<19xf32>
LoopFusion improvements: *) Adds support for fusing into consumer loop nests with multiple loads from the same memref. *) Adds support for reducing slice loop trip count by projecting out destination loop IVs greater than destination loop depth. *) Removes dependence on src loop depth and simplifies cost model computation. PiperOrigin-RevId: 229575126
2019-01-17 01:55:02 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// -----
// CHECK-LABEL: func @should_fuse_with_private_memrefs_with_diff_shapes() {
func @should_fuse_with_private_memrefs_with_diff_shapes() {
%m = alloc() : memref<100xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 100 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0] : memref<100xf32>
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 17 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %m[%i1] : memref<100xf32>
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 82 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %m[%i2] : memref<100xf32>
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
}
// Should create two new private memrefs customized to the shapes accessed
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// by loops %{{.*}} and %{{.*}}.
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<1xf32>
// CHECK-DAG: alloc() : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 17 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 82 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
LoopFusion: Creates private MemRefs which are used only by operations in the fused loop. *) Enables reduction of private memref size based on MemRef region accessed by fused slice. *) Enables maximal fusion by creating a private memref to break a fusion-preventing dependence. *) Adds maximal fusion flag to enable fusing as much as possible (though it still fuses the minimum cost computation slice). PiperOrigin-RevId: 229936698
2019-01-19 00:56:27 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
Fix test cases that were accessing out of bounds to start with (b/123072438) - detected with memref-bound-check - fixes b/123072438; while on this, fix another test case which was reported out of bounds PiperOrigin-RevId: 229978187
2019-01-19 04:55:46 +08:00
}
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
// -----
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-LABEL: func @should_fuse_live_out_arg_but_preserve_src_loop(%{{.*}}: memref<10xf32>) {
func @should_fuse_live_out_arg_but_preserve_src_loop(%arg0: memref<10xf32>) {
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %arg0[%i0] : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
}
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
affine.for %i1 = 0 to 9 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %arg0[%i1] : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
}
// This tests that the loop nest '%i0' should not be removed after fusion
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
// because it writes to memref argument '%arg0', and its read region
// does not cover its write region (so fusion would shrink the write region
// in the fused loop nest, so complete live out data region would not
// be written).
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 9 {
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-LABEL: func @should_fuse_live_out_arg(%{{.*}}: memref<10xf32>) {
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
func @should_fuse_live_out_arg(%arg0: memref<10xf32>) {
%cf7 = constant 7.0 : f32
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %arg0[%i0] : memref<10xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
}
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %arg0[%i1] : memref<10xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// The read/write regions for memref '%{{.*}}' are the same for both
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
// loops, so they should fuse.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-LABEL: func @should_fuse_escaping_memref_but_preserve_src_loop() -> memref<10xf32>
func @should_fuse_escaping_memref_but_preserve_src_loop() -> memref<10xf32> {
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
%cf7 = constant 7.0 : f32
%m = alloc() : memref<10xf32>
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %m[%i0] : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
}
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
affine.for %i1 = 0 to 9 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %m[%i1] : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
}
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// This tests that the loop nest '%i0' should not be removed after fusion
// because it writes to memref '%m', which is returned by the function, and
// the '%i1' memory region does not cover '%i0' memory region.
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 9 {
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: return %{{.*}} : memref<10xf32>
Handle escaping memrefs in loop fusion pass: *) Do not remove loop nests which write to memrefs which escape the function. *) Do not remove memrefs which escape the function (e.g. are used in the return instruction). PiperOrigin-RevId: 230398630
2019-01-23 05:23:37 +08:00
return %m : memref<10xf32>
}
Allocate private/local buffers for slices accurately during fusion - the size of the private memref created for the slice should be based on the memref region accessed at the depth at which the slice is being materialized, i.e., symbolic in the outer IVs up until that depth, as opposed to the region accessed based on the entire domain. - leads to a significant contraction of the temporary / intermediate memref whenever the memref isn't reduced to a single scalar (through store fwd'ing). Other changes - update to promoteIfSingleIteration - avoid introducing unnecessary identity map affine_apply from IV; makes it much easier to write and read test cases and pass output for all passes that use promoteIfSingleIteration; loop-fusion test cases become much simpler - fix replaceAllMemrefUsesWith bug that was exposed by the above update - 'domInstFilter' could be one of the ops erased due to a memref replacement in it. - fix getConstantBoundOnDimSize bug: a division by the coefficient of the identifier was missing (the latter need not always be 1); add lbFloorDivisors output argument - rename getBoundingConstantSizeAndShape -> getConstantBoundingSizeAndShape PiperOrigin-RevId: 230405218
2019-01-23 05:58:52 +08:00
// -----
// This should fuse with the %in becoming a 1x1x1.
func @R3_to_R2_reshape() {
%in = alloc() : memref<2x3x16xi32>
%c0 = constant 0 : index
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 2 {
affine.for %i1 = 0 to 3 {
affine.for %i2 = 0 to 16 {
Allocate private/local buffers for slices accurately during fusion - the size of the private memref created for the slice should be based on the memref region accessed at the depth at which the slice is being materialized, i.e., symbolic in the outer IVs up until that depth, as opposed to the region accessed based on the entire domain. - leads to a significant contraction of the temporary / intermediate memref whenever the memref isn't reduced to a single scalar (through store fwd'ing). Other changes - update to promoteIfSingleIteration - avoid introducing unnecessary identity map affine_apply from IV; makes it much easier to write and read test cases and pass output for all passes that use promoteIfSingleIteration; loop-fusion test cases become much simpler - fix replaceAllMemrefUsesWith bug that was exposed by the above update - 'domInstFilter' could be one of the ops erased due to a memref replacement in it. - fix getConstantBoundOnDimSize bug: a division by the coefficient of the identifier was missing (the latter need not always be 1); add lbFloorDivisors output argument - rename getBoundingConstantSizeAndShape -> getConstantBoundingSizeAndShape PiperOrigin-RevId: 230405218
2019-01-23 05:58:52 +08:00
%val = "foo"(%i0, %i1, %i2) : (index, index, index) -> i32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %val, %in[%i0, %i1, %i2] : memref<2x3x16xi32>
Allocate private/local buffers for slices accurately during fusion - the size of the private memref created for the slice should be based on the memref region accessed at the depth at which the slice is being materialized, i.e., symbolic in the outer IVs up until that depth, as opposed to the region accessed based on the entire domain. - leads to a significant contraction of the temporary / intermediate memref whenever the memref isn't reduced to a single scalar (through store fwd'ing). Other changes - update to promoteIfSingleIteration - avoid introducing unnecessary identity map affine_apply from IV; makes it much easier to write and read test cases and pass output for all passes that use promoteIfSingleIteration; loop-fusion test cases become much simpler - fix replaceAllMemrefUsesWith bug that was exposed by the above update - 'domInstFilter' could be one of the ops erased due to a memref replacement in it. - fix getConstantBoundOnDimSize bug: a division by the coefficient of the identifier was missing (the latter need not always be 1); add lbFloorDivisors output argument - rename getBoundingConstantSizeAndShape -> getConstantBoundingSizeAndShape PiperOrigin-RevId: 230405218
2019-01-23 05:58:52 +08:00
}
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %ii = 0 to 32 {
affine.for %jj = 0 to 3 {
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
%a0 = affine.apply affine_map<(d0, d1) -> (d0 * 3 + d1)> (%ii, %jj)
%idx = affine.apply affine_map<(d0) -> (d0 floordiv (3 * 16))> (%a0)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v = affine.load %in[%idx, %jj, %c0]
Allocate private/local buffers for slices accurately during fusion - the size of the private memref created for the slice should be based on the memref region accessed at the depth at which the slice is being materialized, i.e., symbolic in the outer IVs up until that depth, as opposed to the region accessed based on the entire domain. - leads to a significant contraction of the temporary / intermediate memref whenever the memref isn't reduced to a single scalar (through store fwd'ing). Other changes - update to promoteIfSingleIteration - avoid introducing unnecessary identity map affine_apply from IV; makes it much easier to write and read test cases and pass output for all passes that use promoteIfSingleIteration; loop-fusion test cases become much simpler - fix replaceAllMemrefUsesWith bug that was exposed by the above update - 'domInstFilter' could be one of the ops erased due to a memref replacement in it. - fix getConstantBoundOnDimSize bug: a division by the coefficient of the identifier was missing (the latter need not always be 1); add lbFloorDivisors output argument - rename getBoundingConstantSizeAndShape -> getConstantBoundingSizeAndShape PiperOrigin-RevId: 230405218
2019-01-23 05:58:52 +08:00
: memref<2x3x16xi32>
}
}
return
}
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-DAG: [[$MAP0:#map[0-9]+]] = affine_map<(d0, d1) -> ((d0 * 3 + d1) floordiv 48)>
// CHECK-DAG: [[$MAP1:#map[0-9]+]] = affine_map<(d0, d1) -> (d0 * 3 + d1)>
// CHECK-DAG: [[$MAP2:#map[0-9]+]] = affine_map<(d0) -> (d0 floordiv 48)>
Update replaceAllMemRefUsesWith to generate single result affine_apply's for index remapping - generate a sequence of single result affine_apply's for the index remapping (instead of one multi result affine_apply) - update dma-generate and loop-fusion test cases; while on this, change test cases to use single result affine apply ops - some fusion comment fix/cleanup PiperOrigin-RevId: 230985830
2019-01-26 08:00:50 +08:00
Allocate private/local buffers for slices accurately during fusion - the size of the private memref created for the slice should be based on the memref region accessed at the depth at which the slice is being materialized, i.e., symbolic in the outer IVs up until that depth, as opposed to the region accessed based on the entire domain. - leads to a significant contraction of the temporary / intermediate memref whenever the memref isn't reduced to a single scalar (through store fwd'ing). Other changes - update to promoteIfSingleIteration - avoid introducing unnecessary identity map affine_apply from IV; makes it much easier to write and read test cases and pass output for all passes that use promoteIfSingleIteration; loop-fusion test cases become much simpler - fix replaceAllMemrefUsesWith bug that was exposed by the above update - 'domInstFilter' could be one of the ops erased due to a memref replacement in it. - fix getConstantBoundOnDimSize bug: a division by the coefficient of the identifier was missing (the latter need not always be 1); add lbFloorDivisors output argument - rename getBoundingConstantSizeAndShape -> getConstantBoundingSizeAndShape PiperOrigin-RevId: 230405218
2019-01-23 05:58:52 +08:00
// CHECK-LABEL: func @R3_to_R2_reshape()
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<1x1x1xi32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 32 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 3 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.apply [[$MAP0]](%{{.*}}, %{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: "foo"(%{{.*}}, %{{.*}}, %{{.*}}) : (index, index, index) -> i32
Extend map canonicalization to propagate constant operands - extend canonicalizeMapAndOperands to propagate constant operands into the map's expressions (and thus drop those operands). - canonicalizeMapAndOperands previously only dropped duplicate and unused operands; however, operands that were constants were retained. This change makes IR maps/expressions generated by various utilities/passes even simpler; also makes some of the test checks more accurate and simpler -- for eg., 0' instead of symbol(%{{.*}}). Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Closes tensorflow/mlir#107 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/107 from bondhugula:canonicalize-maps c889a51486d14fbf7db489f224f881e7e1ff7d72 PiperOrigin-RevId: 266085289
2019-08-29 16:13:01 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0, 0] : memref<1x1x1xi32>
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.apply [[$MAP1]](%{{.*}}, %{{.*}})
// CHECK-NEXT: affine.apply [[$MAP2]](%{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0, 0] : memref<1x1x1xi32>
Update replaceAllMemRefUsesWith to generate single result affine_apply's for index remapping - generate a sequence of single result affine_apply's for the index remapping (instead of one multi result affine_apply) - update dma-generate and loop-fusion test cases; while on this, change test cases to use single result affine apply ops - some fusion comment fix/cleanup PiperOrigin-RevId: 230985830
2019-01-26 08:00:50 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
Fix single producer check in loop fusion pass. PiperOrigin-RevId: 230565482
2019-01-24 03:11:43 +08:00
// -----
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
func @should_fuse_multi_output_producer() {
Fix single producer check in loop fusion pass. PiperOrigin-RevId: 230565482
2019-01-24 03:11:43 +08:00
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%i0] : memref<10xf32>
affine.store %cf7, %b[%i0] : memref<10xf32>
Fix single producer check in loop fusion pass. PiperOrigin-RevId: 230565482
2019-01-24 03:11:43 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i1] : memref<10xf32>
Add support for some multi-store cases in affine fusion This PR is a stepping stone towards supporting generic multi-store source loop nests in affine loop fusion. It extends the algorithm to support fusion of multi-store loop nests that: 1. have only one store that writes to a function-local live out, and 2. the remaining stores are involved in loop nest self dependences or no dependences within the function. Closes tensorflow/mlir#162 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/162 from dcaballe:dcaballe/multi-output-fusion 7fb7dec6fe8b45f5ce176f018bfe37b256420c45 PiperOrigin-RevId: 273773907
2019-10-10 01:36:54 +08:00
%v1 = affine.load %b[%i1] : memref<10xf32>
Fix single producer check in loop fusion pass. PiperOrigin-RevId: 230565482
2019-01-24 03:11:43 +08:00
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Fix single producer check in loop fusion pass. PiperOrigin-RevId: 230565482
2019-01-24 03:11:43 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// -----
// CHECK-LABEL: func @fusion_preventing_deps_on_middle_loop() {
func @fusion_preventing_deps_on_middle_loop() {
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i0] : memref<10xf32>
affine.store %v0, %b[%i0] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%i1] : memref<10xf32>
%v1 = affine.load %c[%i1] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v2 = affine.load %b[%i2] : memref<10xf32>
affine.store %v2, %c[%i2] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
// Loops '%i0' and '%i2' cannot fuse along producer/consumer edge on memref
// '%b', because of the WAR dep from '%i0' to '%i1' on memref '%a' and
// because of the WAR dep from '%i1' to '%i2' on memref '%c'.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @should_fuse_and_move_to_preserve_war_dep() {
func @should_fuse_and_move_to_preserve_war_dep() {
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %b[%i0] : memref<10xf32>
affine.store %v0, %a[%i0] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 3 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v2 = affine.load %c[%i1] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 5 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %b[%i2] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i3 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %a[%i3] : memref<10xf32>
affine.store %cf7, %c[%i3] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
// Dependence graph:
//
// %i0 ---------
// | |
// --- %i1 | %b | %a
// | | |
// %c | %i2 <-- |
// | |
// --> %i3 <--------
//
// It is possible to fuse loop '%i0' into '%i3' and preserve dependences
// if the fused loop nest is inserted between loops '%i1' and '%i2'.
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-DAG: alloc() : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 3 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 5 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @fusion_preventing_dep_on_constant() {
func @fusion_preventing_dep_on_constant() {
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %b[%i0] : memref<10xf32>
affine.store %cf7, %a[%i0] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %b[%i1] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
%cf11 = constant 11.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v2 = affine.load %a[%i2] : memref<10xf32>
affine.store %cf11, %c[%i2] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
// Loops '%i0' and '%i2' cannot fuse along producer/consumer edge on memref
// '%a', because of the WAR dep from '%i0' to '%i1' on memref '%b' and
// because of the SSA value dep from '%cf11' def to use in '%i2'.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: %{{.*}} = constant 1.100000e+01 : f32
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @should_fuse_and_preserve_dep_on_constant() {
func @should_fuse_and_preserve_dep_on_constant() {
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
%cf11 = constant 11.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %b[%i0] : memref<10xf32>
affine.store %cf7, %a[%i0] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %b[%i1] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v2 = affine.load %a[%i2] : memref<10xf32>
affine.store %cf11, %c[%i2] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
}
// Loops '%i0' and '%i2' can fuse along producer/consumer edge on memref
// '%a', and preserve the WAR dep from '%i0' to '%i1' on memref '%b', and
// the SSA value dep from '%cf11' def to use in '%i2'.
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK: constant 1.100000e+01 : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing loop nests which require insertion into a new instruction Block position while preserving dependences, opening up additional fusion opportunities. - Adds SSA Value edges to the data dependence graph used in the loop fusion pass. PiperOrigin-RevId: 231417649
2019-01-30 01:36:41 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
// -----
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-LABEL: func @should_fuse_at_depth_above_loop_carried_dependence(%{{.*}}: memref<64x4xf32>, %{{.*}}: memref<64x4xf32>) {
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
func @should_fuse_at_depth_above_loop_carried_dependence(%arg0: memref<64x4xf32>, %arg1: memref<64x4xf32>) {
%out = alloc() : memref<64x4xf32>
%0 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 64 {
affine.for %i1 = 0 to 4 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %0, %out[%i0, %i1] : memref<64x4xf32>
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 4 {
affine.for %i3 = 0 to 4 {
affine.for %i4 = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
%v = affine.load %arg1[16 * %i3 - %i4 + 15, %i2] : memref<64x4xf32>
"op0"(%v) : (f32) -> ()
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i5 = 0 to 4 {
affine.for %i6 = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
%v = affine.load %arg0[16 * %i5 - %i6 + 15, %i3] : memref<64x4xf32>
"op1"(%v) : (f32) -> ()
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i7 = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
%r = "op2"() : () -> (f32)
%v = affine.load %out[16 * %i5 + %i7, %i2] : memref<64x4xf32>
%s = addf %v, %r : f32
affine.store %s, %out[16 * %i5 + %i7, %i2] : memref<64x4xf32>
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
}
}
}
}
// We can fuse source loop nest '%i0' into dst loop nest '%i2', but the
// depth at which we can insert the src loop nest slice into the dst loop
// lest must be decreased because of a loop carried dependence on loop '%i3'.
// As a result, the source loop nest is inserted at dst loop nest depth 1,
Fix minor spelling tweaks (NFC) Closes tensorflow/mlir#175 PiperOrigin-RevId: 275726876
2019-10-21 00:44:06 +08:00
// just above the loop with the carried dependence. In addition, the source
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
// loop nest iteration bounds on its loop '%i1' are reduced to 1, so the
// memref size can be reduced to 128x1xf32.
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK: alloc() : memref<64x1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 4 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 64 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, 0] : memref<64x1xf32>
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 4 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}} * 16 - %{{.*}} + 15, %{{.*}}] : memref<64x4xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: "op0"(%{{.*}}) : (f32) -> ()
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 4 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}} * 16 - %{{.*}} + 15, %{{.*}}] : memref<64x4xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: "op1"(%{{.*}}) : (f32) -> ()
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
// CHECK-NEXT: %{{.*}} = "op2"() : () -> f32
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK: affine.load %{{.*}}[%{{.*}} * 16 + %{{.*}}, 0] : memref<64x1xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
// CHECK: affine.store %{{.*}}, %{{.*}}[%{{.*}} * 16 + %{{.*}}, 0] : memref<64x1xf32>
LoopFusion: insert the source loop nest slice at a depth in the destination loop nest which preserves dependences (above any loop carried or other dependences). This is accomplished by updating the maximum destination loop depth based on dependence checks between source loop nest loads and stores which access the memref on which the source loop nest has a store op. In addition, prevent fusing in source loop nests which write to memrefs which escape or are live out. PiperOrigin-RevId: 231684492
2019-01-31 07:53:41 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
Loop fusion improvements: *) After a private memref buffer is created for a fused loop nest, dependences on the old memref are reduced, which can open up fusion opportunities. In these cases, users of the old memref are added back to the worklist to be reconsidered for fusion. *) Fixed a bug in fusion insertion point dependence check where the memref being privatized was being skipped from the check. PiperOrigin-RevId: 232477853
2019-02-05 22:57:08 +08:00
// -----
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-LABEL: func @should_fuse_only_two_loops_and_remove_producer() {
func @should_fuse_only_two_loops_and_remove_producer() {
Loop fusion improvements: *) After a private memref buffer is created for a fused loop nest, dependences on the old memref are reduced, which can open up fusion opportunities. In these cases, users of the old memref are added back to the worklist to be reconsidered for fusion. *) Fixed a bug in fusion insertion point dependence check where the memref being privatized was being skipped from the check. PiperOrigin-RevId: 232477853
2019-02-05 22:57:08 +08:00
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%i0] : memref<10xf32>
Loop fusion improvements: *) After a private memref buffer is created for a fused loop nest, dependences on the old memref are reduced, which can open up fusion opportunities. In these cases, users of the old memref are added back to the worklist to be reconsidered for fusion. *) Fixed a bug in fusion insertion point dependence check where the memref being privatized was being skipped from the check. PiperOrigin-RevId: 232477853
2019-02-05 22:57:08 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i1] : memref<10xf32>
affine.store %v0, %b[%i1] : memref<10xf32>
Loop fusion improvements: *) After a private memref buffer is created for a fused loop nest, dependences on the old memref are reduced, which can open up fusion opportunities. In these cases, users of the old memref are added back to the worklist to be reconsidered for fusion. *) Fixed a bug in fusion insertion point dependence check where the memref being privatized was being skipped from the check. PiperOrigin-RevId: 232477853
2019-02-05 22:57:08 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %a[%i2] : memref<10xf32>
affine.store %v1, %b[%i2] : memref<10xf32>
Loop fusion improvements: *) After a private memref buffer is created for a fused loop nest, dependences on the old memref are reduced, which can open up fusion opportunities. In these cases, users of the old memref are added back to the worklist to be reconsidered for fusion. *) Fixed a bug in fusion insertion point dependence check where the memref being privatized was being skipped from the check. PiperOrigin-RevId: 232477853
2019-02-05 22:57:08 +08:00
}
// On the first visit to '%i2', the fusion algorithm can not fuse loop nest
// '%i0' into '%i2' because of the dependences '%i0' and '%i2' each have on
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// '%i1'. Then, '%i0' is fused into '%i1' and no private memref is created for
// memref '%a' to be able to remove '%i0' and still preserve the depencence on
// '%a' with '%i2'.
// TODO: Alternatively, we could fuse '%i0' into '%i1' with a private memref,
// the dependence between '%i0' and '%i1' on memref '%a' would no longer exist,
// and '%i0' could be fused into '%i2' as well. Note that this approach would
// duplicate the computation in loop nest '%i0' to loop nests '%i1' and '%i2',
// which would limit its profitability.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop fusion improvements: *) After a private memref buffer is created for a fused loop nest, dependences on the old memref are reduced, which can open up fusion opportunities. In these cases, users of the old memref are added back to the worklist to be reconsidered for fusion. *) Fixed a bug in fusion insertion point dependence check where the memref being privatized was being skipped from the check. PiperOrigin-RevId: 232477853
2019-02-05 22:57:08 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop fusion improvements: *) After a private memref buffer is created for a fused loop nest, dependences on the old memref are reduced, which can open up fusion opportunities. In these cases, users of the old memref are added back to the worklist to be reconsidered for fusion. *) Fixed a bug in fusion insertion point dependence check where the memref being privatized was being skipped from the check. PiperOrigin-RevId: 232477853
2019-02-05 22:57:08 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
// -----
// CHECK-LABEL: func @should_fuse_after_one_loop_interchange() {
func @should_fuse_after_one_loop_interchange() {
%a = alloc() : memref<10xf32>
%cf0 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf0, %a[%i0] : memref<10xf32>
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 5 {
affine.for %i2 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i2] : memref<10xf32>
affine.store %v0, %a[%i2] : memref<10xf32>
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
}
}
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
// The dependence between the load and affine.store is carried on loop '%i1', and
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
// cannot be fused with loop '%i0' without violating this dependence.
// Once loops '%i1' and %i2' are interchanged, loop '%i0' can be fused
// at loop depth 1, because the loop carrying the dependence has been
// interchanged and is now at depth 2.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.for %{{.*}} = 0 to 5 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @should_fuse_after_two_loop_interchanges() {
func @should_fuse_after_two_loop_interchanges() {
%a = alloc() : memref<6x8xf32>
%cf0 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 6 {
affine.for %i1 = 0 to 8 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf0, %a[%i0, %i1] : memref<6x8xf32>
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 4 {
affine.for %i3 = 0 to 6 {
affine.for %i4 = 0 to 2 {
affine.for %i5 = 0 to 8 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i3, %i5] : memref<6x8xf32>
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
%v1 = addf %v0, %v0 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %v1, %a[%i3, %i5] : memref<6x8xf32>
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
}
}
}
}
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
// The dependence between the load and affine.store is carried on loops '%i2' and
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
// '%i4', and cannot be fused with loop '%i0' without violating this
// dependence.
// Once loop '%i2' is interchanged with loop '%i3', and again with loop
// '%i5', then loop '%i0' can be fused at loop depth 2, because the loop
[mlir] NFC: Fix trivial typo in comment Summary: Also, an exercise to merge this into the master myself after a reviewer gives LGTM. Reviewers: nicolasvasilache, mehdi_amini Reviewed By: mehdi_amini Subscribers: Joonsoo, merge_guards_bot, mehdi_amini, rriddle, jpienaar, burmako, shauheen, antiagainst, arpith-jacob, mgester, lucyrfox, liufengdb, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D73432
2020-01-26 23:13:26 +08:00
// carrying the dependences have been interchanged with loops at depth > 2.
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 6 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 8 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0] : memref<1x1xf32>
// CHECK-NEXT: affine.for %{{.*}} = 0 to 4 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 2 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0] : memref<1x1xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0] : memref<1x1xf32>
LoopFusion: perform a series of loop interchanges to increase the loop depth at which slices of producer loop nests can be fused into constumer loop nests. *) Adds utility to LoopUtils to perform loop interchange of two AffineForOps. *) Adds utility to LoopUtils to sink a loop to a specified depth within a loop nest, using a series of loop interchanges. *) Computes dependences between all loads and stores in the loop nest, and classifies each loop as parallel or sequential. *) Computes loop interchange permutation required to sink sequential loops (and raise parallel loop nests) while preserving relative order among them. *) Checks each dependence against the permutation to make sure that dependences would not be violated by the loop interchange transformation. *) Calls loop interchange in LoopFusion pass on consumer loop nests before fusing in producers, sinking loops with loop carried dependences deeper into the consumer loop nest. *) Adds and updates related unit tests. PiperOrigin-RevId: 234158370
2019-02-16 01:32:18 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
Support fusing producer loop nests which write to a memref which is live out, provided that the write region of the consumer loop nest to the same memref is a super set of the producer's write region. PiperOrigin-RevId: 234240958
2019-02-16 09:12:19 +08:00
// -----
func @should_fuse_live_out_writer(%arg0 : memref<10xf32>) -> memref<10xf32> {
%cst = constant 0.000000e+00 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cst, %arg0[%i0] : memref<10xf32>
Support fusing producer loop nests which write to a memref which is live out, provided that the write region of the consumer loop nest to the same memref is a super set of the producer's write region. PiperOrigin-RevId: 234240958
2019-02-16 09:12:19 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%1 = affine.load %arg0[%i1] : memref<10xf32>
affine.store %1, %arg0[%i1] : memref<10xf32>
Support fusing producer loop nests which write to a memref which is live out, provided that the write region of the consumer loop nest to the same memref is a super set of the producer's write region. PiperOrigin-RevId: 234240958
2019-02-16 09:12:19 +08:00
}
return %arg0 : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: %{{.*}} = constant 0.000000e+00 : f32
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Support fusing producer loop nests which write to a memref which is live out, provided that the write region of the consumer loop nest to the same memref is a super set of the producer's write region. PiperOrigin-RevId: 234240958
2019-02-16 09:12:19 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: return %{{.*}} : memref<10xf32>
Support fusing producer loop nests which write to a memref which is live out, provided that the write region of the consumer loop nest to the same memref is a super set of the producer's write region. PiperOrigin-RevId: 234240958
2019-02-16 09:12:19 +08:00
}
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
// -----
// The fused slice has 16 iterations from along %i0.
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-DAG: [[$MAP_LB:#map[0-9]+]] = affine_map<(d0) -> (d0 * 16)>
// CHECK-DAG: [[$MAP_UB:#map[0-9]+]] = affine_map<(d0) -> (d0 * 16 + 16)>
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
// CHECK-LABEL: slice_tile
Make sure that fusion test cases don't have out of bounds accesses - fix out of bounds test case - -memref-bound-check on the test/Transforms/loop-fusion.mlir no longer reports any errors, before or after -loop-fusion is run PiperOrigin-RevId: 236757658
2019-03-05 09:26:46 +08:00
func @slice_tile(%arg0: memref<128x8xf32>, %arg1: memref<32x8xf32>, %0 : f32) -> memref<32x8xf32> {
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 32 {
affine.for %i1 = 0 to 8 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %0, %arg1[%i0, %i1] : memref<32x8xf32>
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i = 0 to 2 {
affine.for %j = 0 to 8 {
affine.for %k = 0 to 8 {
affine.for %kk = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
%v = affine.load %arg0[16 * %k + %kk, %j] : memref<128x8xf32>
%r = "foo"(%v) : (f32) -> f32
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %ii = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
%v = affine.load %arg1[16 * %i + %ii, %j] : memref<32x8xf32>
%s = addf %v, %v : f32
affine.store %s, %arg1[16 * %i + %ii, %j] : memref<32x8xf32>
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
}
}
}
}
Make sure that fusion test cases don't have out of bounds accesses - fix out of bounds test case - -memref-bound-check on the test/Transforms/loop-fusion.mlir no longer reports any errors, before or after -loop-fusion is run PiperOrigin-RevId: 236757658
2019-03-05 09:26:46 +08:00
return %arg1 : memref<32x8xf32>
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 2 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 8 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// CHECK-NEXT: affine.for %{{.*}} = [[$MAP_LB]](%{{.*}}) to [[$MAP_UB]](%{{.*}}) {
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<32x8xf32>
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 8 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}} * 16 + %{{.*}}, %{{.*}}] : memref<128x8xf32>
// CHECK-NEXT: "foo"(%{{.*}}) : (f32) -> f32
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}} * 16 + %{{.*}}, %{{.*}}] : memref<32x8xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}} * 16 + %{{.*}}, %{{.*}}] : memref<32x8xf32>
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: return %{{.*}} : memref<32x8xf32>
Extend/improve getSliceBounds() / complete TODO + update unionBoundingBox - compute slices precisely where the destination iteration depends on multiple source iterations (instead of over-approximating to the whole source loop extent) - update unionBoundingBox to deal with input with non-matching symbols - reenable disabled backend test case PiperOrigin-RevId: 234714069
2019-02-20 10:17:19 +08:00
// CHECK-NEXT:}
Fix bug in memref region computation with slice loop bounds. Adds loop IV values to ComputationSliceState which are used in FlatAffineConstraints::addSliceBounds, to ensure that constraints are only added for loop IV values which are present in the constraint system. PiperOrigin-RevId: 235952912
2019-02-28 03:01:49 +08:00
// -----
// Test case which illustrates fix for b/126454413
func @test_add_slice_bounds() {
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
%c0 = constant 0 : index
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
affine.for %i1 = 0 to 10 {
affine.for %i2 = 0 to 10 {
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
%a0 = affine.apply affine_map<(d0) -> (d0)> (%i0)
%a1 = affine.apply affine_map<(d0) -> (d0)> (%i0)
%a2 = affine.apply affine_map<(d0, d1) -> (d0 - d1)> (%a0, %a1)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %a[%a2] : memref<10xf32>
Fix bug in memref region computation with slice loop bounds. Adds loop IV values to ComputationSliceState which are used in FlatAffineConstraints::addSliceBounds, to ensure that constraints are only added for loop IV values which are present in the constraint system. PiperOrigin-RevId: 235952912
2019-02-28 03:01:49 +08:00
}
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i3 = 0 to 10 {
affine.for %i4 = 0 to 10 {
affine.for %i5 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%c0] : memref<10xf32>
Fix bug in memref region computation with slice loop bounds. Adds loop IV values to ComputationSliceState which are used in FlatAffineConstraints::addSliceBounds, to ensure that constraints are only added for loop IV values which are present in the constraint system. PiperOrigin-RevId: 235952912
2019-02-28 03:01:49 +08:00
}
}
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.apply #map0(%{{.*}})
// CHECK-NEXT: affine.apply #map0(%{{.*}})
// CHECK-NEXT: affine.apply #map1(%{{.*}}, %{{.*}})
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Fix bug in memref region computation with slice loop bounds. Adds loop IV values to ComputationSliceState which are used in FlatAffineConstraints::addSliceBounds, to ensure that constraints are only added for loop IV values which are present in the constraint system. PiperOrigin-RevId: 235952912
2019-02-28 03:01:49 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
Fix bug in memref region computation with slice loop bounds. Adds loop IV values to ComputationSliceState which are used in FlatAffineConstraints::addSliceBounds, to ensure that constraints are only added for loop IV values which are present in the constraint system. PiperOrigin-RevId: 235952912
2019-02-28 03:01:49 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
return
}
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
// -----
func @should_fuse_init_loops_siblings_then_shared_producer(%arg0: memref<10x10xf32>, %arg1: memref<10x10xf32>) {
%0 = alloc() : memref<10x10xf32>
%cst = constant 0.000000e+00 : f32
%cst_0 = constant 1.000000e+00 : f32
%cst_1 = constant 7.000000e+00 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cst_1, %0[%i0, %i1] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 3 {
affine.for %i3 = 0 to 3 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cst, %arg0[%i2, %i3] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i4 = 0 to 3 {
affine.for %i5 = 0 to 3 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%1 = affine.load %0[%i4, %i5] : memref<10x10xf32>
%2 = affine.load %arg0[%i4, %i5] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
%3 = mulf %1, %2 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %3, %arg0[%i4, %i5] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i6 = 0 to 3 {
affine.for %i7 = 0 to 3 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cst_0, %arg1[%i6, %i7] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i8 = 0 to 3 {
affine.for %i9 = 0 to 3 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%4 = affine.load %0[%i8, %i9] : memref<10x10xf32>
%5 = affine.load %arg1[%i8, %i9] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
%6 = addf %4, %5 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %6, %arg1[%i8, %i9] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
}
}
// Pass 1: should fuse single-use producer loop nests into their unique user,
// so '%i2' will fuse into '%i4' and '%i6' will fuse into '%i8'.
// Pass 2: should fuse sibling loop nests which share no dependence edges,
// so should fuse '%i4' into '%i8'.
// Pass 3: should fuse single-use producer loop nest '%i0' into '%i8'. Note
// that loop nest '%i0' now has a single user after Pass 2 fused its
// two users together).
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 3 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 3 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0] : memref<1x1xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x10xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0] : memref<1x1xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x10xf32>
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x10xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x10xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0] : memref<1x1xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x10xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
func @two_matrix_vector_products() {
%in_matrix = alloc() : memref<10x10xf32>
%in_vec0 = alloc() : memref<10xf32>
%in_vec1 = alloc() : memref<10xf32>
%out_vec0 = alloc() : memref<10xf32>
%out_vec1 = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
// Populate input matrix.
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 10 {
affine.for %i1 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf7, %in_matrix[%i0, %i1] : memref<10x10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
}
}
// out_vec0 = in_matrix x in_vec0
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 10 {
affine.for %i3 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %in_matrix[%i2, %i3] : memref<10x10xf32>
%v1 = affine.load %in_vec0[%i3] : memref<10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
%v2 = mulf %v0, %v1 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v3 = affine.load %out_vec0[%i3] : memref<10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
%v4 = addf %v2, %v3 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %v4, %out_vec0[%i3] : memref<10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
}
}
// out_vec1 = in_matrix x in_vec1
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i4 = 0 to 10 {
affine.for %i5 = 0 to 10 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v5 = affine.load %in_matrix[%i4, %i5] : memref<10x10xf32>
%v6 = affine.load %in_vec1[%i5] : memref<10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
%v7 = mulf %v5, %v6 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v8 = affine.load %out_vec1[%i5] : memref<10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
%v9 = addf %v7, %v8 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %v9, %out_vec1[%i5] : memref<10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
}
}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, 0] : memref<10x1xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, 0] : memref<10x1xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 10 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, 0] : memref<10x1xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
Loop fusion for input reuse. *) Breaks fusion pass into multiple sub passes over nodes in data dependence graph: - first pass fuses single-use producers into their unique consumer. - second pass enables fusing for input-reuse by fusing sibling nodes which read from the same memref, but which do not share dependence edges. - third pass fuses remaining producers into their consumers (Note that the sibling fusion pass may have transformed a producer with multiple uses into a single-use producer). *) Fusion for input reuse is enabled by computing a sibling node slice using the load/load accesses to the same memref, and fusion safety is guaranteed by checking that the sibling node memref write region (to a different memref) is preserved. *) Enables output vector and output matrix computations from KFAC patches-second-moment operation to fuse into a single loop nest and reuse input from the image patches operation. *) Adds a generic loop utilitiy for finding all sequential loops in a loop nest. *) Adds and updates unit tests. PiperOrigin-RevId: 236350987
2019-03-02 03:50:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
Bug fix for getConstantBoundOnDimSize - this was detected when memref-bound-check was run on the output of the loop-fusion pass - the addition (to represent ceildiv as a floordiv) had to be performed only for the constant term of the constraint - update test cases - memref-bound-check no longer returns an error on the output of this test case PiperOrigin-RevId: 236731137
2019-03-05 06:58:59 +08:00
}
Adds loop attribute as a temporary work around to prevent slice fusion of loop nests containing instructions with side effects (the proper solution will be do use memref read/write regions in the future). PiperOrigin-RevId: 236733739
2019-03-05 07:14:12 +08:00
// -----
func @should_not_slice_past_slice_barrier() {
%0 = alloc() : memref<100x16xf32>
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 100 {
affine.for %i1 = 0 to 16 {
Adds loop attribute as a temporary work around to prevent slice fusion of loop nests containing instructions with side effects (the proper solution will be do use memref read/write regions in the future). PiperOrigin-RevId: 236733739
2019-03-05 07:14:12 +08:00
%1 = "op1"() : () -> f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %1, %0[%i0, %i1] : memref<100x16xf32>
Change the attribute dictionary syntax to separate name and value with '='. The current syntax separates the name and value with ':', but ':' is already overloaded by several other things(e.g. trailing types). This makes the syntax difficult to parse in some situtations: Old: "foo: 10 : i32" New: "foo = 10 : i32" PiperOrigin-RevId: 255097928
2019-06-26 10:06:06 +08:00
} {slice_fusion_barrier = true}
Adds loop attribute as a temporary work around to prevent slice fusion of loop nests containing instructions with side effects (the proper solution will be do use memref read/write regions in the future). PiperOrigin-RevId: 236733739
2019-03-05 07:14:12 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 100 {
affine.for %i3 = 0 to 16 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%2 = affine.load %0[%i2, %i3] : memref<100x16xf32>
Adds loop attribute as a temporary work around to prevent slice fusion of loop nests containing instructions with side effects (the proper solution will be do use memref read/write regions in the future). PiperOrigin-RevId: 236733739
2019-03-05 07:14:12 +08:00
"op2"(%2) : (f32) -> ()
}
}
// The 'slice_fusion_barrier' attribute on '%i1' prevents slicing the
// iteration space of '%i1' and any enclosing loop nests.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 100 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
// CHECK-NEXT: %{{.*}} = "op1"() : () -> f32
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, %{{.*}}] : memref<1x16xf32>
Change the attribute dictionary syntax to separate name and value with '='. The current syntax separates the name and value with ':', but ':' is already overloaded by several other things(e.g. trailing types). This makes the syntax difficult to parse in some situtations: Old: "foo: 10 : i32" New: "foo = 10 : i32" PiperOrigin-RevId: 255097928
2019-06-26 10:06:06 +08:00
// CHECK-NEXT: } {slice_fusion_barrier = true}
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 16 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, %{{.*}}] : memref<1x16xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: "op2"(%{{.*}}) : (f32) -> ()
Adds loop attribute as a temporary work around to prevent slice fusion of loop nests containing instructions with side effects (the proper solution will be do use memref read/write regions in the future). PiperOrigin-RevId: 236733739
2019-03-05 07:14:12 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
return
}
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
// -----
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
#map0 = affine_map<(d0, d1) -> (d0 * 16 + d1)>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
func @fuse_across_dim_mismatch(%arg0: memref<4x4x16x1xf32>, %arg1: memref<144x9xf32>, %arg2: memref<9xf32>) {
%1 = alloc() : memref<144x4xf32>
%2 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 9 {
affine.for %i3 = 0 to 4 {
affine.for %i5 = 0 to 16 {
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
%7 = affine.apply #map0(%i2, %i5)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %2, %1[%7, %i3] : memref<144x4xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
}
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i6 = 0 to 9 {
affine.for %i7 = 0 to 9 {
affine.for %i8 = 0 to 4 {
affine.for %i10 = 0 to 16 {
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
%10 = affine.apply #map0(%i6, %i10)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%11 = affine.load %1[%10, %i8] : memref<144x4xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
}
}
}
}
return
}
[mlir][AsmPrinter] Refactor printing to only print aliases for attributes/types that will exist in the output. This revision refactors the way that attributes/types are considered when generating aliases. Instead of considering all of the attributes/types of every operation, we perform a "fake" print step that prints the operations using a dummy printer to collect the attributes and types that would actually be printed during the real process. This removes a lot of attributes/types from consideration that generally won't end up in the final output, e.g. affine map attributes in an `affine.apply`/`affine.for`. This resolves a long standing TODO w.r.t aliases, and helps to have a much cleaner textual output format. As a datapoint to the latter, as part of this change several tests were identified as testing for the presence of attributes aliases that weren't actually referenced by the custom form of any operation. To ensure that this wouldn't cause a large degradation in compile time due to the second full print, I benchmarked this change on a very large module with a lot of operations(The file is ~673M/~4.7 million lines long). This file before this change take ~6.9 seconds to print in the custom form, and ~7 seconds after this change. In the custom assembly case, this added an average of a little over ~100 miliseconds to the compile time. This increase was due to the way that argument attributes on functions are structured and how they get printed; i.e. with a better representation the negative impact here can be greatly decreased. When printing in the generic form, this revision had no observable impact on the compile time. This benchmarking leads me to believe that the impact of this change on compile time w.r.t printing is closely related to `print` methods that perform a lot of additional/complex processing outside of the OpAsmPrinter. Differential Revision: https://reviews.llvm.org/D90512
2020-11-10 13:50:31 +08:00
// MAXIMAL: #map = affine_map<(d0, d1) -> (d0 * 16 + d1)>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
// MAXIMAL-LABEL: func @fuse_across_dim_mismatch
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// MAXIMAL: alloc() : memref<1x1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// MAXIMAL: affine.for %{{.*}} = 0 to 9 {
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 9 {
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 4 {
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 16 {
[mlir][AsmPrinter] Refactor printing to only print aliases for attributes/types that will exist in the output. This revision refactors the way that attributes/types are considered when generating aliases. Instead of considering all of the attributes/types of every operation, we perform a "fake" print step that prints the operations using a dummy printer to collect the attributes and types that would actually be printed during the real process. This removes a lot of attributes/types from consideration that generally won't end up in the final output, e.g. affine map attributes in an `affine.apply`/`affine.for`. This resolves a long standing TODO w.r.t aliases, and helps to have a much cleaner textual output format. As a datapoint to the latter, as part of this change several tests were identified as testing for the presence of attributes aliases that weren't actually referenced by the custom form of any operation. To ensure that this wouldn't cause a large degradation in compile time due to the second full print, I benchmarked this change on a very large module with a lot of operations(The file is ~673M/~4.7 million lines long). This file before this change take ~6.9 seconds to print in the custom form, and ~7 seconds after this change. In the custom assembly case, this added an average of a little over ~100 miliseconds to the compile time. This increase was due to the way that argument attributes on functions are structured and how they get printed; i.e. with a better representation the negative impact here can be greatly decreased. When printing in the generic form, this revision had no observable impact on the compile time. This benchmarking leads me to believe that the impact of this change on compile time w.r.t printing is closely related to `print` methods that perform a lot of additional/complex processing outside of the OpAsmPrinter. Differential Revision: https://reviews.llvm.org/D90512
2020-11-10 13:50:31 +08:00
// MAXIMAL-NEXT: affine.apply #map(%{{.*}}, %{{.*}})
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// MAXIMAL-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0] : memref<1x1xf32>
[mlir][AsmPrinter] Refactor printing to only print aliases for attributes/types that will exist in the output. This revision refactors the way that attributes/types are considered when generating aliases. Instead of considering all of the attributes/types of every operation, we perform a "fake" print step that prints the operations using a dummy printer to collect the attributes and types that would actually be printed during the real process. This removes a lot of attributes/types from consideration that generally won't end up in the final output, e.g. affine map attributes in an `affine.apply`/`affine.for`. This resolves a long standing TODO w.r.t aliases, and helps to have a much cleaner textual output format. As a datapoint to the latter, as part of this change several tests were identified as testing for the presence of attributes aliases that weren't actually referenced by the custom form of any operation. To ensure that this wouldn't cause a large degradation in compile time due to the second full print, I benchmarked this change on a very large module with a lot of operations(The file is ~673M/~4.7 million lines long). This file before this change take ~6.9 seconds to print in the custom form, and ~7 seconds after this change. In the custom assembly case, this added an average of a little over ~100 miliseconds to the compile time. This increase was due to the way that argument attributes on functions are structured and how they get printed; i.e. with a better representation the negative impact here can be greatly decreased. When printing in the generic form, this revision had no observable impact on the compile time. This benchmarking leads me to believe that the impact of this change on compile time w.r.t printing is closely related to `print` methods that perform a lot of additional/complex processing outside of the OpAsmPrinter. Differential Revision: https://reviews.llvm.org/D90512
2020-11-10 13:50:31 +08:00
// MAXIMAL-NEXT: affine.apply #map(%{{.*}}, %{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// MAXIMAL-NEXT: affine.load %{{.*}}[0, 0] : memref<1x1xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
// MAXIMAL-NEXT: }
// MAXIMAL-NEXT: }
// MAXIMAL-NEXT: }
// MAXIMAL-NEXT: }
// -----
[mlir] Change the syntax of AffineMapAttr and IntegerSetAttr to avoid conflicts with function types. Summary: The current syntax for AffineMapAttr and IntegerSetAttr conflict with function types, making it currently impossible to round-trip function types(and e.g. FuncOp) in the IR. This revision changes the syntax for the attributes by wrapping them in a keyword. AffineMapAttr is wrapped with `affine_map<>` and IntegerSetAttr is wrapped with `affine_set<>`. Reviewed By: nicolasvasilache, ftynse Differential Revision: https://reviews.llvm.org/D72429
2020-01-14 05:12:37 +08:00
#map3 = affine_map<(d0, d1) -> ((d0 * 72 + d1) floordiv 2304)>
#map4 = affine_map<(d0, d1) -> (((d0 * 72 + d1) mod 2304) floordiv 1152)>
#map5 = affine_map<(d0, d1) -> (((((d0 * 72 + d1) mod 2304) mod 1152) floordiv 9) floordiv 8)>
#map6 = affine_map<(d0, d1) -> (((((d0 * 72 + d1) mod 2304) mod 1152) mod 9) floordiv 3)>
#map7 = affine_map<(d0, d1) -> (((((d0 * 72 + d1) mod 2304) mod 1152) mod 9) mod 3)>
#map10 = affine_map<(d0, d1) -> (d0 * 16 + d1)>
#map11 = affine_map<(d0, d1) -> (d0 * 16 + d1)>
#map12 = affine_map<(d0, d1) -> (d0 * 16 - d1 + 15)>
Affine loop parallelism detection: conservatively handle unknown ops The loop parallelism detection utility only collects the affine.load and affine.store operations appearing inside the loop to analyze the access patterns for the absence of dependences. However, any operation, including unregistered operations, can appear in a body of an affine loop. If such operation has side effects, the result of parallelism analysis is incorrect. Conservatively assume affine loops are not parallel in presence of operations other than affine.load, affine.store, affine.for, affine.terminator that may have side effects. This required to update the loop-fusion unit test that relies on parallelism analysis and was exercising loop fusion in presence of an unregistered operation. PiperOrigin-RevId: 259560935
2019-07-24 01:18:18 +08:00
func @fuse_across_varying_dims_complex(%arg0: f32) {
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
%c0 = constant 0 : index
%0 = alloc() : memref<2x2x3x3x16x1xf32>
%1 = alloc() : memref<64x9xf32>
%2 = alloc() : memref<144x4xf32>
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 64 {
affine.for %i1 = 0 to 9 {
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
%4 = affine.apply #map3(%i0, %i1)
%5 = affine.apply #map4(%i0, %i1)
%6 = affine.apply #map5(%i0, %i1)
%7 = affine.apply #map6(%i0, %i1)
%8 = affine.apply #map7(%i0, %i1)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%9 = affine.load %0[%4, %5, %7, %8, %6, %c0] : memref<2x2x3x3x16x1xf32>
affine.store %9, %1[%i0, %i1] : memref<64x9xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 0 to 9 {
affine.for %i3 = 0 to 4 {
affine.for %i4 = 0 to 16 {
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
%10 = affine.apply #map10(%i3, %i4)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%11 = affine.load %1[%10, %i2] : memref<64x9xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i5 = 0 to 16 {
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
%14 = affine.apply #map11(%i2, %i5)
Affine loop parallelism detection: conservatively handle unknown ops The loop parallelism detection utility only collects the affine.load and affine.store operations appearing inside the loop to analyze the access patterns for the absence of dependences. However, any operation, including unregistered operations, can appear in a body of an affine loop. If such operation has side effects, the result of parallelism analysis is incorrect. Conservatively assume affine loops are not parallel in presence of operations other than affine.load, affine.store, affine.for, affine.terminator that may have side effects. This required to update the loop-fusion unit test that relies on parallelism analysis and was exercising loop fusion in presence of an unregistered operation. PiperOrigin-RevId: 259560935
2019-07-24 01:18:18 +08:00
affine.store %arg0, %2[%14, %i3] : memref<144x4xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
}
}
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i6 = 0 to 9 {
affine.for %i7 = 0 to 9 {
affine.for %i8 = 0 to 4 {
affine.for %i9 = 0 to 16 {
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
%15 = affine.apply #map12(%i8, %i9)
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%16 = affine.load %1[%15, %i7] : memref<64x9xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
}
}
}
}
return
}
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// MAXIMAL-DAG: [[$MAP0:#map[0-9]+]] = affine_map<(d0, d1) -> ((d0 * 72 + d1) floordiv 2304)>
// MAXIMAL-DAG: [[$MAP1:#map[0-9]+]] = affine_map<(d0, d1) -> (((d0 * 72 + d1) mod 2304) floordiv 1152)>
// MAXIMAL-DAG: [[$MAP2:#map[0-9]+]] = affine_map<(d0, d1) -> (((((d0 * 72 + d1) mod 2304) mod 1152) floordiv 9) floordiv 8)>
// MAXIMAL-DAG: [[$MAP3:#map[0-9]+]] = affine_map<(d0, d1) -> (((((d0 * 72 + d1) mod 2304) mod 1152) mod 9) floordiv 3)>
// MAXIMAL-DAG: [[$MAP4:#map[0-9]+]] = affine_map<(d0, d1) -> (((((d0 * 72 + d1) mod 2304) mod 1152) mod 9) mod 3)>
// MAXIMAL-DAG: [[$MAP7:#map[0-9]+]] = affine_map<(d0, d1) -> (d0 * 16 + d1)>
// MAXIMAL-DAG: [[$MAP8:#map[0-9]+]] = affine_map<(d0, d1) -> (d0 * 16 - d1 + 15)>
Add missing run command to fusion test cases - follow up to cl/236882988 PiperOrigin-RevId: 236947383
2019-03-06 08:54:34 +08:00
// MAXIMAL-LABEL: func @fuse_across_varying_dims_complex
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// MAXIMAL-NEXT: alloc() : memref<64x1xf32>
// MAXIMAL-NEXT: constant 0 : index
// MAXIMAL-NEXT: alloc() : memref<2x2x3x3x16x1xf32>
// MAXIMAL-NEXT: alloc() : memref<144x4xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 9 {
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 9 {
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 4 {
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 16 {
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 64 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// MAXIMAL-NEXT: affine.apply [[$MAP0]](%{{.*}}, %{{.*}})
// MAXIMAL-NEXT: affine.apply [[$MAP1]](%{{.*}}, %{{.*}})
// MAXIMAL-NEXT: affine.apply [[$MAP2]](%{{.*}}, %{{.*}})
// MAXIMAL-NEXT: affine.apply [[$MAP3]](%{{.*}}, %{{.*}})
// MAXIMAL-NEXT: affine.apply [[$MAP4]](%{{.*}}, %{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// MAXIMAL-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}] : memref<2x2x3x3x16x1xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// MAXIMAL-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, 0] : memref<64x1xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
// MAXIMAL-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 4 {
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 16 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// MAXIMAL-NEXT: affine.apply [[$MAP7]](%{{.*}}, %{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// MAXIMAL-NEXT: affine.load %{{.*}}[%{{.*}} * 16 + %{{.*}}, 0] : memref<64x1xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
// MAXIMAL-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// MAXIMAL-NEXT: affine.for %{{.*}} = 0 to 16 {
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// MAXIMAL-NEXT: affine.apply [[$MAP7]](%{{.*}}, %{{.*}})
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// MAXIMAL-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<144x4xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
// MAXIMAL-NEXT: }
// MAXIMAL-NEXT: }
Enable FileCheck -enable-var-scope by default in MLIR test This option avoids to accidentally reuse variable across -LABEL match, it can be explicitly opted-in by prefixing the variable name with $ Differential Revision: https://reviews.llvm.org/D81531
2020-06-12 08:34:35 +08:00
// MAXIMAL-NEXT: affine.apply [[$MAP8]](%{{.*}}, %{{.*}})
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// MAXIMAL-NEXT: affine.load %{{.*}}[%{{.*}} * 16 - %{{.*}} + 15, 0] : memref<64x1xf32>
Fix and improve detectAsMod - fix for the mod detection - simplify/avoid the mod at construction (if the dividend is already known to be less than the divisor), since the information is available at hand there PiperOrigin-RevId: 236882988
2019-03-06 03:08:41 +08:00
// MAXIMAL-NEXT: }
// MAXIMAL-NEXT: }
// MAXIMAL-NEXT: }
// MAXIMAL-NEXT: }
Use FlatAffineConstraints::unionBoundingBox to perform slice bounds union for loop fusion pass (WIP). Adds utility to convert slice bounds to a FlatAffineConstraints representation. Adds utility to FlatAffineConstraints to promote loop IV symbol identifiers to dim identifiers. PiperOrigin-RevId: 236973261
2019-03-06 12:33:30 +08:00
// -----
func @should_fuse_with_slice_union() {
%a = alloc() : memref<100xf32>
%c0 = constant 0 : index
%cf0 = constant 0.0 : f32
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i0 = 0 to 100 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cf0, %a[%i0]: memref<100xf32>
Use FlatAffineConstraints::unionBoundingBox to perform slice bounds union for loop fusion pass (WIP). Adds utility to convert slice bounds to a FlatAffineConstraints representation. Adds utility to FlatAffineConstraints to promote loop IV symbol identifiers to dim identifiers. PiperOrigin-RevId: 236973261
2019-03-06 12:33:30 +08:00
}
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i1 = 10 to 20 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v0 = affine.load %a[%i1]: memref<100xf32>
NFC: Rename the 'for' operation in the AffineOps dialect to 'affine.for' and set the namespace of the AffineOps dialect to 'affine'. PiperOrigin-RevId: 240165792
2019-03-26 01:14:34 +08:00
affine.for %i2 = 15 to 25 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%v1 = affine.load %a[%i2]: memref<100xf32>
Use FlatAffineConstraints::unionBoundingBox to perform slice bounds union for loop fusion pass (WIP). Adds utility to convert slice bounds to a FlatAffineConstraints representation. Adds utility to FlatAffineConstraints to promote loop IV symbol identifiers to dim identifiers. PiperOrigin-RevId: 236973261
2019-03-06 12:33:30 +08:00
}
}
// The union of two slice bounds (calculated between the store and each of
// the loads) is computed and used in the fusion cost calculation, index
// remapping, and private memref size. The result is that the temporary
// memref is reduced from 100xf32 to 15xf32 and properly indexed by
// the fused loops based on the union calculation.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 10 to 20 {
// CHECK-NEXT: affine.for %{{.*}} = 10 to 25 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}} - 10] : memref<15xf32>
Use FlatAffineConstraints::unionBoundingBox to perform slice bounds union for loop fusion pass (WIP). Adds utility to convert slice bounds to a FlatAffineConstraints representation. Adds utility to FlatAffineConstraints to promote loop IV symbol identifiers to dim identifiers. PiperOrigin-RevId: 236973261
2019-03-06 12:33:30 +08:00
// CHECK-NEXT: }
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}} - 10] : memref<15xf32>
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 15 to 25 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}} - 10] : memref<15xf32>
Use FlatAffineConstraints::unionBoundingBox to perform slice bounds union for loop fusion pass (WIP). Adds utility to convert slice bounds to a FlatAffineConstraints representation. Adds utility to FlatAffineConstraints to promote loop IV symbol identifiers to dim identifiers. PiperOrigin-RevId: 236973261
2019-03-06 12:33:30 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
// -----
func @affine_add_mm_fused(%arg0: memref<1024x1024xf32>, %arg1: memref<1024x1024xf32>, %arg2: memref<1024x1024xf32>, %arg3: memref<1024x1024xf32>) {
affine.for %i2 = 0 to 1024 {
affine.for %i3 = 0 to 1024 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%0 = affine.load %arg3[%i2, %i3] : memref<1024x1024xf32>
%1 = affine.load %arg2[%i2, %i3] : memref<1024x1024xf32>
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
%2 = addf %1, %0 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %2, %arg2[%i2, %i3] : memref<1024x1024xf32>
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
}
}
affine.for %i4 = 0 to 1024 {
affine.for %i5 = 0 to 1024 {
affine.for %i6 = 0 to 1024 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%3 = affine.load %arg1[%i6, %i5] : memref<1024x1024xf32>
%4 = affine.load %arg0[%i4, %i6] : memref<1024x1024xf32>
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
%5 = mulf %4, %3 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%6 = affine.load %arg2[%i4, %i5] : memref<1024x1024xf32>
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
%7 = addf %6, %5 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %7, %arg2[%i4, %i5] : memref<1024x1024xf32>
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
}
}
}
// Should fuse elementwise add loop at loop depth 2, above loop-carried
// dependence between load/store on '%arg2', carried on reduction loop %i6.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 1024 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
Remove overly conservative check in LoopFusion pass (enables fusion in tutorial example). PiperOrigin-RevId: 240859227
2019-03-29 05:54:49 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
return
}
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
// -----
func @affine_2mm_fused(%arg0: memref<1024x1024xf32>, %arg1: memref<1024x1024xf32>, %arg2: memref<1024x1024xf32>, %arg3: memref<1024x1024xf32>, %arg4: memref<1024x1024xf32>) {
%cst = constant 0.000000e+00 : f32
affine.for %i0 = 0 to 1024 {
affine.for %i1 = 0 to 1024 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cst, %arg2[%i0, %i1] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
}
}
affine.for %i2 = 0 to 1024 {
affine.for %i3 = 0 to 1024 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %cst, %arg4[%i2, %i3] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
}
}
affine.for %i4 = 0 to 1024 {
affine.for %i5 = 0 to 1024 {
affine.for %i6 = 0 to 1024 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%0 = affine.load %arg1[%i6, %i5] : memref<1024x1024xf32>
%1 = affine.load %arg0[%i4, %i6] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
%2 = mulf %1, %0 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%3 = affine.load %arg2[%i4, %i5] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
%4 = addf %3, %2 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %4, %arg2[%i4, %i5] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
}
}
}
affine.for %i7 = 0 to 1024 {
affine.for %i8 = 0 to 1024 {
affine.for %i9 = 0 to 1024 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%5 = affine.load %arg1[%i9, %i8] : memref<1024x1024xf32>
%6 = affine.load %arg0[%i7, %i9] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
%7 = mulf %6, %5 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%8 = affine.load %arg4[%i7, %i8] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
%9 = addf %8, %7 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %9, %arg4[%i7, %i8] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
}
}
}
Fix minor spelling tweaks (NFC) Closes tensorflow/mlir#175 PiperOrigin-RevId: 275726876
2019-10-21 00:44:06 +08:00
// Should fuse MM initialization loops into their consumers, then fuse the
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
// two matmul loops together for input reuse on '%arg0/%arg1'.
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK: affine.for %{{.*}} = 0 to 1024 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
Enable input-reuse fusion to search function arguments for fusion candidates (takes care of a TODO, enables another tutorial test case). PiperOrigin-RevId: 240979894
2019-03-29 23:06:25 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
return
}
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
// -----
func @affine_2_dependent_mm_fused(%arg0: memref<1024x1024xf32>, %arg1: memref<1024x1024xf32>, %arg2: memref<1024x1024xf32>, %arg3: memref<1024x1024xf32>, %arg4: memref<1024x1024xf32>) {
affine.for %i0 = 0 to 1024 {
affine.for %i1 = 0 to 1024 {
affine.for %i2 = 0 to 1024 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%0 = affine.load %arg1[%i2, %i1] : memref<1024x1024xf32>
%1 = affine.load %arg0[%i0, %i2] : memref<1024x1024xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
%2 = mulf %1, %0 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%3 = affine.load %arg2[%i0, %i1] : memref<1024x1024xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
%4 = addf %3, %2 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %4, %arg2[%i0, %i1] : memref<1024x1024xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
}
}
}
affine.for %i3 = 0 to 1024 {
affine.for %i4 = 0 to 1024 {
affine.for %i5 = 0 to 1024 {
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%5 = affine.load %arg3[%i5, %i4] : memref<1024x1024xf32>
%6 = affine.load %arg2[%i3, %i5] : memref<1024x1024xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
%7 = mulf %6, %5 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
%8 = affine.load %arg4[%i3, %i4] : memref<1024x1024xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
%9 = addf %8, %7 : f32
Globally change load/store/dma_start/dma_wait operations over to affine.load/store/dma_start/dma_wait. In most places, this is just a name change (with the exception of affine.dma_start swapping the operand positions of its tag memref and num_elements operands). Significant code changes occur here: *) Vectorization: LoopAnalysis.cpp, Vectorize.cpp *) Affine Transforms: Transforms/Utils/Utils.cpp PiperOrigin-RevId: 256395088
2019-07-04 01:35:03 +08:00
affine.store %9, %arg4[%i3, %i4] : memref<1024x1024xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
}
}
}
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK: affine.for %{{.*}} = 0 to 1024 {
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
// CHECK-NEXT: affine.for %{{.*}} = 0 to 1024 {
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Standardize the value numbering in the AsmPrinter. Change the AsmPrinter to number values breadth-first so that values in adjacent regions can have the same name. This allows for ModuleOp to contain operations that produce results. This also standardizes the special name of region entry arguments to "arg[0-9+]" now that Functions are also operations. PiperOrigin-RevId: 257225069
2019-07-10 01:40:29 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}, %{{.*}}] : memref<1024x1024xf32>
Enable producer-consumer fusion for liveout memrefs if consumer read region matches producer write region. -- PiperOrigin-RevId: 241517207
2019-04-02 21:37:40 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: }
return
}
Add support for some multi-store cases in affine fusion This PR is a stepping stone towards supporting generic multi-store source loop nests in affine loop fusion. It extends the algorithm to support fusion of multi-store loop nests that: 1. have only one store that writes to a function-local live out, and 2. the remaining stores are involved in loop nest self dependences or no dependences within the function. Closes tensorflow/mlir#162 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/162 from dcaballe:dcaballe/multi-output-fusion 7fb7dec6fe8b45f5ce176f018bfe37b256420c45 PiperOrigin-RevId: 273773907
2019-10-10 01:36:54 +08:00
// -----
// CHECK-LABEL: func @should_fuse_self_dependence_multi_store_producer() {
func @should_fuse_self_dependence_multi_store_producer() {
%m = alloc() : memref<10xf32>
%local_m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
affine.for %i0 = 0 to 10 {
affine.store %cf7, %local_m[%i0] : memref<10xf32>
%v0 = affine.load %local_m[%i0] : memref<10xf32>
affine.store %v0, %m[%i0] : memref<10xf32>
}
affine.for %i1 = 0 to 10 {
%v1 = affine.load %m[%i1] : memref<10xf32>
}
// CHECK: affine.for %[[i0:.*]] = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, [[LOCAL_M:%.*]][%[[i0]]] : memref<10xf32>
// CHECK-NEXT: [[v0:%.*]] = affine.load [[LOCAL_M]][%[[i0]]] : memref<10xf32>
// CHECK-NEXT: affine.store [[v0]], %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @should_fuse_dead_multi_store_producer() {
func @should_fuse_dead_multi_store_producer() {
%m = alloc() : memref<10xf32>
%dead_m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
affine.for %i0 = 0 to 10 {
affine.store %cf7, %dead_m[%i0] : memref<10xf32>
affine.store %cf7, %m[%i0] : memref<10xf32>
}
affine.for %i1 = 0 to 10 {
%v0 = affine.load %m[%i1] : memref<10xf32>
}
// CHECK: affine.for %[[i0:.*]] = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%[[i0]]] : memref<10xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
// -----
// CHECK-LABEL: func @should_fuse_function_live_out_multi_store_producer
func @should_fuse_function_live_out_multi_store_producer(%live_in_out_m : memref<10xf32>) {
%m = alloc() : memref<10xf32>
%cf7 = constant 7.0 : f32
affine.for %i0 = 0 to 10 {
affine.store %cf7, %live_in_out_m[%i0] : memref<10xf32>
affine.store %cf7, %m[%i0] : memref<10xf32>
}
affine.for %i1 = 0 to 10 {
%v0 = affine.load %m[%i1] : memref<10xf32>
}
// CHECK: affine.for %[[i0:.*]] = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%[[i0]]] : memref<10xf32>
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
Add support for some multi-store cases in affine fusion This PR is a stepping stone towards supporting generic multi-store source loop nests in affine loop fusion. It extends the algorithm to support fusion of multi-store loop nests that: 1. have only one store that writes to a function-local live out, and 2. the remaining stores are involved in loop nest self dependences or no dependences within the function. Closes tensorflow/mlir#162 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/162 from dcaballe:dcaballe/multi-output-fusion 7fb7dec6fe8b45f5ce176f018bfe37b256420c45 PiperOrigin-RevId: 273773907
2019-10-10 01:36:54 +08:00
// CHECK-NEXT: }
// CHECK-NEXT: return
return
}
Fix Affine Loop Fusion test case reported on github. This CL utilizies the more robust fusion feasibility analysis being built out in LoopFusionUtils, which will eventually be used to replace the current affine loop fusion pass. PiperOrigin-RevId: 281112340
2019-11-19 03:20:03 +08:00
// -----
// Test case from github bug 777.
// CHECK-LABEL: func @mul_add_0
func @mul_add_0(%arg0: memref<3x4xf32>, %arg1: memref<4x3xf32>, %arg2: memref<3x3xf32>, %arg3: memref<3x3xf32>) {
%cst = constant 0.000000e+00 : f32
%0 = alloc() : memref<3x3xf32>
affine.for %arg4 = 0 to 3 {
affine.for %arg5 = 0 to 3 {
affine.store %cst, %0[%arg4, %arg5] : memref<3x3xf32>
}
}
affine.for %arg4 = 0 to 3 {
affine.for %arg5 = 0 to 3 {
affine.for %arg6 = 0 to 4 {
%1 = affine.load %arg1[%arg6, %arg5] : memref<4x3xf32>
%2 = affine.load %arg0[%arg4, %arg6] : memref<3x4xf32>
%3 = mulf %2, %1 : f32
%4 = affine.load %0[%arg4, %arg5] : memref<3x3xf32>
%5 = addf %4, %3 : f32
affine.store %5, %0[%arg4, %arg5] : memref<3x3xf32>
}
}
}
affine.for %arg4 = 0 to 3 {
affine.for %arg5 = 0 to 3 {
%6 = affine.load %arg2[%arg4, %arg5] : memref<3x3xf32>
%7 = affine.load %0[%arg4, %arg5] : memref<3x3xf32>
%8 = addf %7, %6 : f32
affine.store %8, %arg3[%arg4, %arg5] : memref<3x3xf32>
}
}
// CHECK: affine.for %[[i0:.*]] = 0 to 3 {
// CHECK-NEXT: affine.for %[[i1:.*]] = 0 to 3 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0] : memref<1x1xf32>
// CHECK-NEXT: affine.for %[[i2:.*]] = 0 to 4 {
// CHECK-NEXT: affine.load %{{.*}}[%[[i2]], %[[i1]]] : memref<4x3xf32>
// CHECK-NEXT: affine.load %{{.*}}[%[[i0]], %[[i2]]] : memref<3x4xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: mulf %{{.*}}, %{{.*}} : f32
Fix Affine Loop Fusion test case reported on github. This CL utilizies the more robust fusion feasibility analysis being built out in LoopFusionUtils, which will eventually be used to replace the current affine loop fusion pass. PiperOrigin-RevId: 281112340
2019-11-19 03:20:03 +08:00
// CHECK-NEXT: affine.load %{{.*}}[0, 0] : memref<1x1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Fix Affine Loop Fusion test case reported on github. This CL utilizies the more robust fusion feasibility analysis being built out in LoopFusionUtils, which will eventually be used to replace the current affine loop fusion pass. PiperOrigin-RevId: 281112340
2019-11-19 03:20:03 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0, 0] : memref<1x1xf32>
// CHECK-NEXT: }
// CHECK-NEXT: affine.load %{{.*}}[%[[i0]], %[[i1]]] : memref<3x3xf32>
// CHECK-NEXT: affine.load %{{.*}}[0, 0] : memref<1x1xf32>
[MLIR] NFC: modernize affine loop fusion test cases - update test case for readability, avoid unnecessary matches Signed-off-by: Uday Bondhugula <uday@polymagelabs.com> Differential Revision: https://reviews.llvm.org/D75823
2020-03-09 12:27:26 +08:00
// CHECK-NEXT: addf %{{.*}}, %{{.*}} : f32
Fix Affine Loop Fusion test case reported on github. This CL utilizies the more robust fusion feasibility analysis being built out in LoopFusionUtils, which will eventually be used to replace the current affine loop fusion pass. PiperOrigin-RevId: 281112340
2019-11-19 03:20:03 +08:00
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%[[i0]], %[[i1]]] : memref<3x3xf32>
// CHECK-NEXT: }
// CHECK-NEXT: }
// CHECK-NEXT: return
AffineLoopFusion: Prevent fusion of multi-out-edge producer loops tensorflow/mlir#162 introduced a bug that incorrectly allowed fusion of producer loops with multiple outgoing edges. This commit fixes that problem. It also introduces a new flag to disable sibling loop fusion so that we can test producer-consumer fusion in isolation. Closes tensorflow/mlir#259 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/259 from dcaballe:dcaballe/fix_multi_out_edge_producer_fusion 578d5661705fd5c56c555832d5e0528df88c5282 PiperOrigin-RevId: 283531105
2019-12-03 22:09:21 +08:00
return
}
// -----
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// Verify that 'fuseProducerConsumerNodes' fuse a producer loop with a store
// that has multiple outgoing edges.
Fix Affine Loop Fusion test case reported on github. This CL utilizies the more robust fusion feasibility analysis being built out in LoopFusionUtils, which will eventually be used to replace the current affine loop fusion pass. PiperOrigin-RevId: 281112340
2019-11-19 03:20:03 +08:00
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK-LABEL: func @should_fuse_multi_outgoing_edge_store_producer
func @should_fuse_multi_outgoing_edge_store_producer(%a : memref<1xf32>) {
AffineLoopFusion: Prevent fusion of multi-out-edge producer loops tensorflow/mlir#162 introduced a bug that incorrectly allowed fusion of producer loops with multiple outgoing edges. This commit fixes that problem. It also introduces a new flag to disable sibling loop fusion so that we can test producer-consumer fusion in isolation. Closes tensorflow/mlir#259 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/259 from dcaballe:dcaballe/fix_multi_out_edge_producer_fusion 578d5661705fd5c56c555832d5e0528df88c5282 PiperOrigin-RevId: 283531105
2019-12-03 22:09:21 +08:00
%cst = constant 0.000000e+00 : f32
affine.for %arg0 = 0 to 1 {
affine.store %cst, %a[%arg0] : memref<1xf32>
}
affine.for %arg0 = 0 to 1 {
%0 = affine.load %a[%arg0] : memref<1xf32>
}
affine.for %arg0 = 0 to 1 {
%0 = affine.load %a[%arg0] : memref<1xf32>
}
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// CHECK: affine.for %{{.*}} = 0 to 1 {
// CHECK-NEXT: affine.store
// CHECK-NEXT: affine.load
// CHECK-NEXT: affine.load
// CHECK-NEXT: }
AffineLoopFusion: Prevent fusion of multi-out-edge producer loops tensorflow/mlir#162 introduced a bug that incorrectly allowed fusion of producer loops with multiple outgoing edges. This commit fixes that problem. It also introduces a new flag to disable sibling loop fusion so that we can test producer-consumer fusion in isolation. Closes tensorflow/mlir#259 COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/mlir/pull/259 from dcaballe:dcaballe/fix_multi_out_edge_producer_fusion 578d5661705fd5c56c555832d5e0528df88c5282 PiperOrigin-RevId: 283531105
2019-12-03 22:09:21 +08:00
return
}
// -----
// Verify that 'fuseProducerConsumerNodes' fuses a producer loop that: 1) has
// multiple outgoing edges, 2) producer store has a single outgoing edge.
// Sibling loop fusion should not fuse any of these loops due to
// dependencies on external memrefs '%a' and '%b'.
// CHECK-LABEL: func @should_fuse_producer_with_multi_outgoing_edges
func @should_fuse_producer_with_multi_outgoing_edges(%a : memref<1xf32>, %b : memref<1xf32>) {
%cst = constant 0.000000e+00 : f32
affine.for %arg0 = 0 to 1 {
%0 = affine.load %a[%arg0] : memref<1xf32>
affine.store %cst, %b[%arg0] : memref<1xf32>
}
affine.for %arg0 = 0 to 1 {
affine.store %cst, %a[%arg0] : memref<1xf32>
%1 = affine.load %b[%arg0] : memref<1xf32>
}
// CHECK: affine.for %{{.*}} = 0 to 1
// CHECK-NEXT: affine.load %[[A:.*]][{{.*}}]
// CHECK-NEXT: affine.store %{{.*}}, %[[B:.*]][{{.*}}]
// CHECK-NEXT: affine.store %{{.*}}, %[[A]]
// CHECK-NEXT: affine.load %[[B]]
// CHECK-NOT: affine.for %{{.*}}
[MLIR] Fix affine fusion bug/efficiency issue / enable more fusion The list of destination load ops while evaluating producer-consumer fusion wasn't being maintained as a set, and as such, duplicate load ops were being added to it. Although this is harmless correctness-wise, it's a killer efficiency-wise and it prevents interesting/useful fusions (including for eg. reshapes into a matmul). The reason the latter fusions would be missed is that a slice union would be unnecessarily needed due to the duplicate load ops on a memref added to the 'dst loads' list. Since slice union is unimplemented for the local var case, a single destination load op that leads to local vars (like a floordiv / mod producing fusion), a common case, would not get fused due to an unnecessary union being tried with itself. (The union would actually be the same thing but we would bail out.) Besides the above, this would also significantly speed up fusion as all the unnecessary slice computations / unions, checks, etc. due to the duplicates go away. Differential Revision: https://reviews.llvm.org/D79547
2020-05-07 11:51:11 +08:00
// CHECK: return
Fix Affine Loop Fusion test case reported on github. This CL utilizies the more robust fusion feasibility analysis being built out in LoopFusionUtils, which will eventually be used to replace the current affine loop fusion pass. PiperOrigin-RevId: 281112340
2019-11-19 03:20:03 +08:00
return
}
[MLIR] Fix affine fusion bug/efficiency issue / enable more fusion The list of destination load ops while evaluating producer-consumer fusion wasn't being maintained as a set, and as such, duplicate load ops were being added to it. Although this is harmless correctness-wise, it's a killer efficiency-wise and it prevents interesting/useful fusions (including for eg. reshapes into a matmul). The reason the latter fusions would be missed is that a slice union would be unnecessarily needed due to the duplicate load ops on a memref added to the 'dst loads' list. Since slice union is unimplemented for the local var case, a single destination load op that leads to local vars (like a floordiv / mod producing fusion), a common case, would not get fused due to an unnecessary union being tried with itself. (The union would actually be the same thing but we would bail out.) Besides the above, this would also significantly speed up fusion as all the unnecessary slice computations / unions, checks, etc. due to the duplicates go away. Differential Revision: https://reviews.llvm.org/D79547
2020-05-07 11:51:11 +08:00
// -----
// MAXIMAL-LABEL: func @reshape_into_matmul
func @reshape_into_matmul(%lhs : memref<1024x1024xf32>,
%R: memref<16x64x1024xf32>, %out: memref<1024x1024xf32>) {
%rhs = alloc() : memref<1024x1024xf32>
// Reshape from 3-d to 2-d.
affine.for %i0 = 0 to 16 {
affine.for %i1 = 0 to 64 {
affine.for %k = 0 to 1024 {
%v = affine.load %R[%i0, %i1, %k] : memref<16x64x1024xf32>
affine.store %v, %rhs[64*%i0 + %i1, %k] : memref<1024x1024xf32>
}
}
}
// Matmul.
affine.for %i = 0 to 1024 {
affine.for %j = 0 to 1024 {
affine.for %k = 0 to 1024 {
%0 = affine.load %rhs[%k, %j] : memref<1024x1024xf32>
%1 = affine.load %lhs[%i, %k] : memref<1024x1024xf32>
%2 = mulf %1, %0 : f32
%3 = affine.load %out[%i, %j] : memref<1024x1024xf32>
%4 = addf %3, %2 : f32
affine.store %4, %out[%i, %j] : memref<1024x1024xf32>
}
}
}
return
}
// MAXIMAL-NEXT: alloc
// MAXIMAL-NEXT: affine.for
// MAXIMAL-NEXT: affine.for
// MAXIMAL-NEXT: affine.for
// MAXIMAL-NOT: affine.for
// MAXIMAL: return
[mlir][Affine] Enable fusion of loops with vector loads/stores This patch enables affine loop fusion for loops with affine vector loads and stores. For that, we only had to use affine memory op interfaces in LoopFusionUtils.cpp and Utils.cpp so that vector loads and stores are also taken into account. Reviewed By: andydavis1, ftynse Differential Revision: https://reviews.llvm.org/D80971
2020-06-03 06:10:47 +08:00
// -----
// CHECK-LABEL: func @vector_loop
func @vector_loop(%a : memref<10x20xf32>, %b : memref<10x20xf32>,
%c : memref<10x20xf32>) {
affine.for %j = 0 to 10 {
affine.for %i = 0 to 5 {
%ld0 = affine.vector_load %a[%j, %i*4] : memref<10x20xf32>, vector<4xf32>
affine.vector_store %ld0, %b[%j, %i*4] : memref<10x20xf32>, vector<4xf32>
}
}
affine.for %j = 0 to 10 {
affine.for %i = 0 to 5 {
%ld0 = affine.vector_load %b[%j, %i*4] : memref<10x20xf32>, vector<4xf32>
affine.vector_store %ld0, %c[%j, %i*4] : memref<10x20xf32>, vector<4xf32>
}
}
return
}
// CHECK: affine.for
// CHECK-NEXT: affine.for
// CHECK-NEXT: affine.vector_load
// CHECK-NEXT: affine.vector_store
// CHECK-NEXT: affine.vector_load
// CHECK-NEXT: affine.vector_store
// CHECK-NOT: affine.for
[mlir][Affine] Revisit fusion candidates after successful fusion This patch changes the fusion algorithm so that after fusing two loop nests we revisit previously visited nodes so that they are considered again for fusion in the context of the new fused loop nest. Reviewed By: bondhugula Differential Revision: https://reviews.llvm.org/D81609
2020-06-12 05:39:44 +08:00
// -----
// CHECK-LABEL: func @multi_outgoing_edges
func @multi_outgoing_edges(%in0 : memref<32xf32>,
%in1 : memref<32xf32>) {
affine.for %d = 0 to 32 {
%lhs = affine.load %in0[%d] : memref<32xf32>
%rhs = affine.load %in1[%d] : memref<32xf32>
%add = addf %lhs, %rhs : f32
affine.store %add, %in0[%d] : memref<32xf32>
}
affine.for %d = 0 to 32 {
%lhs = affine.load %in0[%d] : memref<32xf32>
%rhs = affine.load %in1[%d] : memref<32xf32>
%add = subf %lhs, %rhs : f32
affine.store %add, %in0[%d] : memref<32xf32>
}
affine.for %d = 0 to 32 {
%lhs = affine.load %in0[%d] : memref<32xf32>
%rhs = affine.load %in1[%d] : memref<32xf32>
%add = mulf %lhs, %rhs : f32
affine.store %add, %in0[%d] : memref<32xf32>
}
affine.for %d = 0 to 32 {
%lhs = affine.load %in0[%d] : memref<32xf32>
%rhs = affine.load %in1[%d] : memref<32xf32>
%add = divf %lhs, %rhs : f32
affine.store %add, %in0[%d] : memref<32xf32>
}
return
}
// CHECK: affine.for
// CHECK-NOT: affine.for
// CHECK: addf
// CHECK-NOT: affine.for
// CHECK: subf
// CHECK-NOT: affine.for
// CHECK: mulf
// CHECK-NOT: affine.for
// CHECK: divf
[MLIR] Fix affine loop fusion private memref alloc Drop stale code that provided the wrong operands to alloc. Reported-by: rjnw on discourse Differential Revision: https://reviews.llvm.org/D82409
2020-06-24 04:52:03 +08:00
// -----
// Test fusion when dynamically shaped memrefs are used with constant trip count loops.
// CHECK-LABEL: func @calc
func @calc(%arg0: memref<?xf32>, %arg1: memref<?xf32>, %arg2: memref<?xf32>, %len: index) {
%c1 = constant 1 : index
%1 = alloc(%len) : memref<?xf32>
affine.for %arg4 = 1 to 10 {
%7 = affine.load %arg0[%arg4] : memref<?xf32>
%8 = affine.load %arg1[%arg4] : memref<?xf32>
%9 = addf %7, %8 : f32
affine.store %9, %1[%arg4] : memref<?xf32>
}
affine.for %arg4 = 1 to 10 {
%7 = affine.load %1[%arg4] : memref<?xf32>
%8 = affine.load %arg1[%arg4] : memref<?xf32>
%9 = mulf %7, %8 : f32
affine.store %9, %arg2[%arg4] : memref<?xf32>
}
return
}
// CHECK: alloc() : memref<1xf32>
// CHECK: affine.for %arg{{.*}} = 1 to 10 {
// CHECK-NEXT: affine.load %arg{{.*}}
// CHECK-NEXT: affine.load %arg{{.*}}
// CHECK-NEXT: addf
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %arg{{.*}}[%arg{{.*}}] : memref<?xf32>
// CHECK-NEXT: mulf
// CHECK-NEXT: affine.store %{{.*}}, %arg{{.*}}[%arg{{.*}}] : memref<?xf32>
// CHECK-NEXT: }
// CHECK-NEXT: return
[MLIR][Affine-loop-fusion] Fix a bug in affine-loop-fusion pass when there are non-affine operations When there is a mix of affine load/store and non-affine operations (e.g. std.load, std.store), affine-loop-fusion ignores the present of non-affine ops, thus changing the program semantics. E.g. we have a program of three affine loops operating on the same memref in which one of them uses std.load and std.store, as follows. ``` affine.for affine.store %1 affine.for std.load %1 std.store %1 affine.for affine.load %1 affine.store %1 ``` affine-loop-fusion will produce the following result which changed the program semantics: ``` affine.for std.load %1 std.store %1 affine.for affine.store %1 affine.load %1 affine.store %1 ``` This patch is to fix the above problem by checking non-affine users of the memref that are between the source and destination nodes of interest. Differential Revision: https://reviews.llvm.org/D82158
2020-06-25 00:56:05 +08:00
// -----
// CHECK-LABEL: func @should_not_fuse_since_non_affine_users
func @should_not_fuse_since_non_affine_users(%in0 : memref<32xf32>,
%in1 : memref<32xf32>) {
affine.for %d = 0 to 32 {
%lhs = affine.load %in0[%d] : memref<32xf32>
%rhs = affine.load %in1[%d] : memref<32xf32>
%add = addf %lhs, %rhs : f32
affine.store %add, %in0[%d] : memref<32xf32>
}
affine.for %d = 0 to 32 {
%lhs = load %in0[%d] : memref<32xf32>
%rhs = load %in1[%d] : memref<32xf32>
%add = subf %lhs, %rhs : f32
store %add, %in0[%d] : memref<32xf32>
}
affine.for %d = 0 to 32 {
%lhs = affine.load %in0[%d] : memref<32xf32>
%rhs = affine.load %in1[%d] : memref<32xf32>
%add = mulf %lhs, %rhs : f32
affine.store %add, %in0[%d] : memref<32xf32>
}
return
}
// CHECK: affine.for
// CHECK: addf
// CHECK: affine.for
// CHECK: subf
// CHECK: affine.for
// CHECK: mulf
// -----
// CHECK-LABEL: func @should_not_fuse_since_top_level_non_affine_users
func @should_not_fuse_since_top_level_non_affine_users(%in0 : memref<32xf32>,
%in1 : memref<32xf32>) {
%sum = alloc() : memref<f32>
affine.for %d = 0 to 32 {
%lhs = affine.load %in0[%d] : memref<32xf32>
%rhs = affine.load %in1[%d] : memref<32xf32>
%add = addf %lhs, %rhs : f32
store %add, %sum[] : memref<f32>
affine.store %add, %in0[%d] : memref<32xf32>
}
%load_sum = load %sum[] : memref<f32>
affine.for %d = 0 to 32 {
%lhs = affine.load %in0[%d] : memref<32xf32>
%rhs = affine.load %in1[%d] : memref<32xf32>
%add = mulf %lhs, %rhs : f32
%sub = subf %add, %load_sum: f32
affine.store %sub, %in0[%d] : memref<32xf32>
}
dealloc %sum : memref<f32>
return
}
// CHECK: affine.for
// CHECK: addf
// CHECK: affine.for
// CHECK: mulf
// CHECK: subf
[MLIR][Affine] Fix createPrivateMemRef in affine fusion Always define a remapping for the memref replacement (`indexRemap`) with the proper number of inputs, including all the `outerIVs`, so that the number of inputs and the operands provided for the map don't mismatch. Reviewed By: bondhugula, andydavis1 Differential Revision: https://reviews.llvm.org/D85177
2020-08-05 02:22:19 +08:00
// -----
// MAXIMAL-LABEL: func @fuse_minor_affine_map
func @fuse_minor_affine_map(%in: memref<128xf32>, %out: memref<20x512xf32>) {
%tmp = alloc() : memref<128xf32>
affine.for %arg4 = 0 to 128 {
%ld = affine.load %in[%arg4] : memref<128xf32>
affine.store %ld, %tmp[%arg4] : memref<128xf32>
}
affine.for %arg3 = 0 to 20 {
affine.for %arg4 = 0 to 512 {
%ld = affine.load %tmp[%arg4 mod 128] : memref<128xf32>
affine.store %ld, %out[%arg3, %arg4] : memref<20x512xf32>
}
}
return
}
// TODO: The size of the private memref is not properly computed in the presence
// of the 'mod' operation. It should be memref<1xf32> instead of
// memref<128xf32>: https://bugs.llvm.org/show_bug.cgi?id=46973
// MAXIMAL: alloc() : memref<128xf32>
// MAXIMAL: affine.for
// MAXIMAL-NEXT: affine.for
// MAXIMAL-NOT: affine.for
[mlir][Affine] Add support for multi-store producer fusion This patch adds support for producer-consumer fusion scenarios with multiple producer stores to the AffineLoopFusion pass. The patch introduces some changes to the producer-consumer algorithm, including: * For a given consumer loop, producer-consumer fusion iterates over its producer candidates until a fixed point is reached. * Producer candidates are gathered beforehand for each iteration of the consumer loop and visited in reverse program order (not strictly guaranteed) to maximize the number of loops fused per iteration. In general, these changes were needed to simplify the multi-store producer support and remove some of the workarounds that were introduced in the past to support more fusion cases under the single-store producer limitation. This patch also preserves the existing functionality of AffineLoopFusion with one minor change in behavior. Producer-consumer fusion didn't fuse scenarios with escaping memrefs and multiple outgoing edges (from a single store). Multi-store producer scenarios will usually (always?) have multiple outgoing edges so we couldn't fuse any with escaping memrefs, which would greatly limit the applicability of this new feature. Therefore, the patch enables fusion for these scenarios. Please, see modified tests for specific details. Reviewed By: andydavis1, bondhugula Differential Revision: https://reviews.llvm.org/D92876
2021-01-21 06:41:51 +08:00
// MAXIMAL: return
// -----
// CHECK-LABEL: func @should_fuse_multi_store_producer_and_privatize_memfefs
func @should_fuse_multi_store_producer_and_privatize_memfefs() {
%a = alloc() : memref<10xf32>
%b = alloc() : memref<10xf32>
%c = alloc() : memref<10xf32>
%cst = constant 0.000000e+00 : f32
affine.for %arg0 = 0 to 10 {
affine.store %cst, %a[%arg0] : memref<10xf32>
affine.store %cst, %b[%arg0] : memref<10xf32>
affine.store %cst, %c[%arg0] : memref<10xf32>
%0 = affine.load %c[%arg0] : memref<10xf32>
}
affine.for %arg0 = 0 to 10 {
%0 = affine.load %a[%arg0] : memref<10xf32>
}
affine.for %arg0 = 0 to 10 {
%0 = affine.load %b[%arg0] : memref<10xf32>
}
// All the memrefs should be privatized except '%c', which is not involved in
// the producer-consumer fusion.
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: affine.load %{{.*}}[0] : memref<1xf32>
// CHECK-NEXT: }
return
}
// -----
func @should_fuse_multi_store_producer_with_scaping_memrefs_and_remove_src(
%a : memref<10xf32>, %b : memref<10xf32>) {
%cst = constant 0.000000e+00 : f32
affine.for %i0 = 0 to 10 {
affine.store %cst, %a[%i0] : memref<10xf32>
affine.store %cst, %b[%i0] : memref<10xf32>
}
affine.for %i1 = 0 to 10 {
%0 = affine.load %a[%i1] : memref<10xf32>
}
affine.for %i2 = 0 to 10 {
%0 = affine.load %b[%i2] : memref<10xf32>
}
// Producer loop '%i0' should be removed after fusion since fusion is maximal.
// No memref should be privatized since they escape the function.
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: }
// CHECK-NOT: affine.for
return
}
// -----
func @should_fuse_multi_store_producer_with_scaping_memrefs_and_preserve_src(
%a : memref<10xf32>, %b : memref<10xf32>) {
%cst = constant 0.000000e+00 : f32
affine.for %i0 = 0 to 10 {
affine.store %cst, %a[%i0] : memref<10xf32>
affine.store %cst, %b[%i0] : memref<10xf32>
}
affine.for %i1 = 0 to 5 {
%0 = affine.load %a[%i1] : memref<10xf32>
}
affine.for %i2 = 0 to 10 {
%0 = affine.load %b[%i2] : memref<10xf32>
}
// Loops '%i0' and '%i2' should be fused first and '%i0' should be removed
// since fusion is maximal. Then the fused loop and '%i1' should be fused
// and the fused loop shouldn't be removed since fusion is not maximal.
// CHECK: affine.for %{{.*}} = 0 to 10 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: }
// CHECK: affine.for %{{.*}} = 0 to 5 {
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.store %{{.*}}, %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: affine.load %{{.*}}[%{{.*}}] : memref<10xf32>
// CHECK-NEXT: }
// CHECK-NOT: affine.for
return
}