Commit Graph

58 Commits

Author SHA1 Message Date
Mehdi Amini 1881d6fc80 Apply clang-tidy fixes for performance-unnecessary-copy-initialization in MathOps.cpp (NFC) 2022-04-19 00:47:58 +00:00
jacquesguan 83bd4fe2e8 [mlir][Math] Replace some constant folder functions with common folder functions.
Differential Revision: https://reviews.llvm.org/D123485
2022-04-12 11:34:47 +00:00
jacquesguan e609417cdc [mlir][Math] Add more constant folder for Math ops.
This revision add constant folder for abs, copysign, ctlz, cttz and
ctpop.

Differential Revision: https://reviews.llvm.org/D122115
2022-03-22 10:23:15 +08:00
jacquesguan 26c95ae389 [mlir][Math] Add constant folder for sqrt.
Differential Revision: https://reviews.llvm.org/D121980
2022-03-18 16:01:44 +08:00
William S. Moses 164c7afaf5 [MLIR][Math] Add constant folder for powf
Constant fold powf, given two constant operands and a compatible type

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D121845
2022-03-17 14:19:47 -04:00
River Riddle 1f971e23f0 [mlir] Trim a huge number of unnecessary dependencies on the Func dialect
The Func has a large number of legacy dependencies carried over from the old
Standard dialect, which was pervasive and contained a large number of varied
operations. With the split of the standard dialect and its demise, a lot of lingering
dead dependencies have survived to the Func dialect. This commit removes a
large majority of then, greatly reducing the dependence surface area of the
Func dialect.
2022-03-01 12:10:04 -08:00
River Riddle 23aa5a7446 [mlir] Rename the Standard dialect to the Func dialect
The last remaining operations in the standard dialect all revolve around
FuncOp/function related constructs. This patch simply handles the initial
renaming (which by itself is already huge), but there are a large number
of cleanups unlocked/necessary afterwards:

* Removing a bunch of unnecessary dependencies on Func
* Cleaning up the From/ToStandard conversion passes
* Preparing for the move of FuncOp to the Func dialect

See the discussion at https://discourse.llvm.org/t/standard-dialect-the-final-chapter/6061

Differential Revision: https://reviews.llvm.org/D120624
2022-03-01 12:10:04 -08:00
Tres Popp f5efe28070 [mlir] Propagate NaNs in PolynomialApproximation
Previously, NaNs would be dropped in favor of bounded values which was
strictly incorrect. Now the min/max operation propagate this
information. Not all uses of min/max need this, but the given change
will help protect future additions, and this prevents the need for an
additional cmpf and select operation to handle NaNs.

Differential Revision: https://reviews.llvm.org/D120020
2022-02-18 09:25:36 +01:00
Adrian Kuegel b122cbebec [mlir][Math] Fix NaN handling in Exp approximation
Differential Revision: https://reviews.llvm.org/D119832
2022-02-15 15:17:56 +01:00
Adrian Kuegel 87de451bc5 [mlir][Math] Fix NaN handling in ExpM1 approximation.
Differential Revision: https://reviews.llvm.org/D119822
2022-02-15 12:10:12 +01:00
Jacques Pienaar bbddd19ec7 [mlir][math] Expand coverage of atan2 expansion
Reuse the higher precision F32 approximation for the F16 one (by expanding and
truncating). This is partly RFC as I'm not sure what the expectations are here
(e.g., these are only for F32 and should not be expanded, that reusing
higher-precision ones for lower precision is undesirable due to increased
compute cost and only approximations per exact type is preferred, or this is
appropriate [at least as fallback] but we need to see how to make it more
generic across all the patterns here).

Differential Revision: https://reviews.llvm.org/D118968
2022-02-08 15:00:39 -08:00
River Riddle 3c69bc4d6e [mlir][NFC] Remove a few op builders that simply swap parameter order
Differential Revision: https://reviews.llvm.org/D119093
2022-02-07 19:03:57 -08:00
River Riddle 8e123ca65f [mlir:Standard] Remove support for creating a `unit` ConstantOp
This is completely unused upstream, and does not really have well defined semantics
on what this is supposed to do/how this fits into the ecosystem. Given that, as part of
splitting up the standard dialect it's best to just remove this behavior, instead of try
to awkwardly fit it somewhere upstream. Downstream users are encouraged to
define their own operations that clearly can define the semantics of this.

This also uncovered several lingering uses of ConstantOp that weren't
updated to use arith::ConstantOp, and worked during conversions because
the constant was removed/converted into something else before
verification.

See https://llvm.discourse.group/t/standard-dialect-the-final-chapter/ for more discussion.

Differential Revision: https://reviews.llvm.org/D118654
2022-02-02 14:45:12 -08:00
River Riddle dec8af701f [mlir] Move SelectOp from Standard to Arithmetic
This is part of splitting up the standard dialect. See https://llvm.discourse.group/t/standard-dialect-the-final-chapter/ for discussion.

Differential Revision: https://reviews.llvm.org/D118648
2022-02-02 14:45:12 -08:00
Matthias Springer 99ef9eebad [mlir][vector][NFC] Split into IR, Transforms and Utils
This reduces the dependencies of the MLIRVector target and makes the dialect consistent with other dialects.

Differential Revision: https://reviews.llvm.org/D118533
2022-01-31 19:17:09 +09:00
Alexandre Ganea dc3b9365b6 [mlir] Silence warnings when building with MSVC
Differential Revision: https://reviews.llvm.org/D118536
2022-01-30 17:31:35 -05:00
Mehdi Amini 70ed93ec4c Apply clang-tidy fixes for readability-identifier-naming in PolynomialApproximation.cpp (NFC) 2022-01-30 19:49:22 +00:00
River Riddle 633f5badbf [mlir] Add more missing dependencies after D118062
These used to be covered transitively, but now need to be explicit.
2022-01-24 22:12:03 -08:00
Rob Suderman 2f9f9afa4e [mlir] Add polynomial approximation for atan and atan2
Implement a taylor series approximation for atan and add an atan2 lowering
that uses atan's appromation. This includes tests for edge cases and tests
for each quadrant.

Reviewed By: NatashaKnk

Differential Revision: https://reviews.llvm.org/D115682
2022-01-21 12:22:58 -08:00
Mehdi Amini 1c173c9442 Apply clang-tidy fixes for readability-identifier-naming in MathOps.cpp (NFC) 2022-01-14 18:08:42 +00:00
William S. Moses 1773dddadf [MLIR][Math] Enable constant folding of ops
Enable constant folding of ops within the math dialect, and introduce constant folders for ceil and log2

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D117085
2022-01-12 12:19:29 -05:00
William S. Moses d2c547342c Revert "[MLIR][Math] Enable constant folding of ops"
This reverts commit 2f8b956ab6.

There is a linker error for mlir-nvidia as seen on
https://lab.llvm.org/buildbot/#/builders/61/builds/19939.

As it's late for me here, I'm oing to rever this for now to be
investigated later.
2022-01-12 02:00:53 -05:00
William S. Moses 2f8b956ab6 [MLIR][Math] Enable constant folding of ops
Enable constant folding of ops within the math dialect, and introduce constant folders for ceil and log2

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D117085
2022-01-12 01:55:48 -05:00
Mehdi Amini 1fc096af1e Apply clang-tidy fixes for performance-unnecessary-value-param to MLIR (NFC)
Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D116250
2022-01-02 01:45:18 +00:00
Mehdi Amini 89de9cc8a7 Apply clang-tidy fixes for performance-for-range-copy to MLIR (NFC)
Differential Revision: https://reviews.llvm.org/D116248
2022-01-02 01:13:42 +00:00
Mehdi Amini ced8690d84 Apply clang-tidy fixes for bugprone-argument-comment to MLIR (NFC)
Differential Revision: https://reviews.llvm.org/D116244
2022-01-02 01:05:06 +00:00
Mehdi Amini 02b6fb218e Fix clang-tidy issues in mlir/ (NFC)
Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115956
2021-12-20 20:25:01 +00:00
Mehdi Amini be0a7e9f27 Adjust "end namespace" comment in MLIR to match new agree'd coding style
See D115115 and this mailing list discussion:
https://lists.llvm.org/pipermail/llvm-dev/2021-December/154199.html

Differential Revision: https://reviews.llvm.org/D115309
2021-12-08 06:05:26 +00:00
Jacques Pienaar 62fea88bc5 [mlir] Update accessors prefixed form (NFC) 2021-11-30 19:42:37 -08:00
Alexander Belyaev f89bb3c012 [mlir] Move bufferization-related passes to `bufferization` dialect.
[RFC](https://llvm.discourse.group/t/rfc-dialect-for-bufferization-related-ops/4712)

Differential Revision: https://reviews.llvm.org/D114698
2021-11-30 09:58:47 +01:00
Eugene Zhulenev ec32d540f8 [mlir] MathApproximations: scalars shape must be 0-rank
Using [1] for representing shape of a scalar is incorrect, and will break with vectors of size 1.

- remove redundant helper functions
- fix couple of style warnings

Reviewed By: cota

Differential Revision: https://reviews.llvm.org/D112764
2021-10-29 04:02:38 -07:00
Eugene Zhulenev 627fa0b9a8 [mlir] MathApproximations: unroll virtual vectors into hardware vectors for ISA specific operation
Reviewed By: cota

Differential Revision: https://reviews.llvm.org/D112736
2021-10-28 12:52:04 -07:00
Alexander Belyaev 7d79a25769 [mlir] Use float literals to make Windows build happy. 2021-10-26 21:36:04 +02:00
Alexander Belyaev a9a0ea92d1 [mlir] Update Erf approximation. 2021-10-26 21:27:20 +02:00
Alexander Belyaev 96cee29762 [mlir] Allow polynomial approximations for N-d vectors.
Polynomial approximation can be extented to support N-d vectors.
N-dimensional vectors are useful when vectorizing operations on N-dimensional
tiles. Before lowering to LLVM these vectors are usually unrolled or flattened
to 1-dimensional vectors.

Differential Revision: https://reviews.llvm.org/D112566
2021-10-26 20:50:00 +02:00
Boian Petkantchin f1b922188e [MLIR][Math] Add erf to math dialect
Add math.erf lowering to libm call.
Add math.erf polynomial approximation.

Reviewed By: silvas, ezhulenev

Differential Revision: https://reviews.llvm.org/D112200
2021-10-25 18:30:17 +00:00
Emilio Cota 35553d452b [mlir] Add polynomial approximation for vectorized math::Rsqrt
This patch adds a polynomial approximation that matches the
approximation in Eigen.

Note that the approximation only applies to vectorized inputs;
the scalar rsqrt is left unmodified.

The approximation is protected with a flag since it emits an AVX2
intrinsic (generated via the X86Vector). This is the only reasonably
clean way that I could find to generate the exact approximation that
I wanted (i.e. an identical one to Eigen's).

I considered two alternatives:

1. Introduce a Rsqrt intrinsic in LLVM, which doesn't exist yet.
   I believe this is because there is no definition of Rsqrt that
   all backends could agree on, since hardware instructions that
   implement it have widely varying degrees of precision.
   This is something that the standard could mandate, but Rsqrt is
   not part of IEEE754, so I don't think this option is feasible.

2. Emit fdiv(1.0, sqrt) with fast math flags to allow reciprocal
   transformations. Although portable, this doesn't allow us
   to generate exactly the code we want; it is the LLVM backend,
   and not MLIR, who controls what code is generated based on the
   target CPU.

Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D112192
2021-10-23 04:56:12 -07:00
Ahmed Taei 21f9e4a1ed Avoid infinity arithmetics when computing exp approximations
Otherwise this can result a poison value on some platforms see https://bugs.llvm.org/show_bug.cgi?id=51204

Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D112115
2021-10-21 10:09:18 -07:00
Eugene Zhulenev bf32bb7e05 [mlir] Update approximation range for Tanh operation
Use wider range for approximating Tanh to match results computed in Eigen with AVX.

Reviewed By: cota

Differential Revision: https://reviews.llvm.org/D112011
2021-10-18 10:57:31 -07:00
Mogball a54f4eae0e [MLIR] Replace std ops with arith dialect ops
Precursor: https://reviews.llvm.org/D110200

Removed redundant ops from the standard dialect that were moved to the
`arith` or `math` dialects.

Renamed all instances of operations in the codebase and in tests.

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D110797
2021-10-13 03:07:03 +00:00
Benjamin Kramer c1ebefdf77 [mlir] Make polynomial approximation emit std instead of LLVM ops
This is a bit cleaner and removes issues with 2d vectors. It also has a
big impact on constant folding, hence the test changes.

Differential Revision: https://reviews.llvm.org/D107896
2021-08-11 16:37:21 +02:00
bakhtiyar 391456f33c Fix a bug in algebraic simplification, and enable the tests.
Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D107788
2021-08-10 04:15:56 -07:00
Eugene Zhulenev d94426d22a [mlir] Math: add algebraic simplification patterns to math transforms
Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/D106822
2021-07-27 09:22:33 -07:00
Stella Laurenzo 485cc55edf [mlir] Generare .cpp.inc files for dialects.
* Previously, we were only generating .h.inc files. We foresee the need to also generate implementations and this is a step towards that.
* Discussed in https://llvm.discourse.group/t/generating-cpp-inc-files-for-dialects/3732/2
* Deviates from the discussion above by generating a default constructor in the .cpp.inc file (and adding a tablegen bit that disables this in case if this is user provided).
* Generating the destructor started as a way to flush out the missing includes (produces a link error), but it is a strict improvement on its own that is worth doing (i.e. by emitting key methods in the .cpp file, we root vtables in one translation unit, which is a non-controversial improvement).

Differential Revision: https://reviews.llvm.org/D105070
2021-06-29 20:10:30 +00:00
Ahmed S. Taei 7e2d672a67 Add polynomial approximation for trigonometric sine and cosine functions
The approximation relays on range reduced version y \in [0, pi/2]. An input x will have
the property that sin(x) = sin(y), -sin(y), cos(y), -cos(y) depends on which quadrable x
is in, where sin(y) and cos(y) are approximated with 5th degree polynomial (of x^2).
As a result a single pattern can be used to compute approximation for both sine and cosine.

Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D104582
2021-06-21 13:00:33 -07:00
Emilio Cota 0edc4bc84a [mlir] Add polynomial approximation for math::ExpM1
This approximation matches the one in Eigen.

```
name                      old cpu/op  new cpu/op  delta
BM_mlir_Expm1_f32/10      90.9ns ± 4%  52.2ns ± 4%  -42.60%    (p=0.000 n=74+87)
BM_mlir_Expm1_f32/100      837ns ± 3%   231ns ± 4%  -72.43%    (p=0.000 n=79+69)
BM_mlir_Expm1_f32/1k      8.43µs ± 3%  1.58µs ± 5%  -81.30%    (p=0.000 n=77+83)
BM_mlir_Expm1_f32/10k     83.8µs ± 3%  15.4µs ± 5%  -81.65%    (p=0.000 n=83+69)
BM_eigen_s_Expm1_f32/10   68.8ns ±17%  72.5ns ±14%   +5.40%  (p=0.000 n=118+115)
BM_eigen_s_Expm1_f32/100   694ns ±11%   717ns ± 2%   +3.34%   (p=0.000 n=120+75)
BM_eigen_s_Expm1_f32/1k   7.69µs ± 2%  7.97µs ±11%   +3.56%   (p=0.000 n=95+117)
BM_eigen_s_Expm1_f32/10k  88.0µs ± 1%  89.3µs ± 6%   +1.45%   (p=0.000 n=74+106)
BM_eigen_v_Expm1_f32/10   44.3ns ± 6%  45.0ns ± 8%   +1.45%   (p=0.018 n=81+111)
BM_eigen_v_Expm1_f32/100   351ns ± 1%   360ns ± 9%   +2.58%    (p=0.000 n=73+99)
BM_eigen_v_Expm1_f32/1k   3.31µs ± 1%  3.42µs ± 9%   +3.37%   (p=0.000 n=71+100)
BM_eigen_v_Expm1_f32/10k  33.7µs ± 8%  34.1µs ± 9%   +1.04%    (p=0.007 n=99+98)
```

Reviewed By: ezhulenev

Differential Revision: https://reviews.llvm.org/D101852
2021-05-05 14:31:34 -07:00
Emilio Cota 1c0374e770 [mlir] Add polynomial approximation for math::Log1p
This approximation matches the one in Eigen.

```
name                      old cpu/op  new cpu/op  delta
BM_mlir_Log1p_f32/10      83.2ns ± 7%  34.8ns ± 5%  -58.19%    (p=0.000 n=84+71)
BM_mlir_Log1p_f32/100      664ns ± 4%   129ns ± 4%  -80.57%    (p=0.000 n=82+82)
BM_mlir_Log1p_f32/1k      6.75µs ± 4%  0.81µs ± 3%  -88.07%    (p=0.000 n=88+79)
BM_mlir_Log1p_f32/10k     76.5µs ± 3%   7.8µs ± 4%  -89.84%    (p=0.000 n=80+80)
BM_eigen_s_Log1p_f32/10   70.1ns ±14%  72.6ns ±14%   +3.49%  (p=0.000 n=116+112)
BM_eigen_s_Log1p_f32/100   706ns ± 9%   717ns ± 3%   +1.60%   (p=0.018 n=117+80)
BM_eigen_s_Log1p_f32/1k   8.26µs ± 1%  8.26µs ± 1%     ~       (p=0.567 n=84+86)
BM_eigen_s_Log1p_f32/10k  92.1µs ± 5%  92.6µs ± 6%   +0.60%  (p=0.047 n=115+115)
BM_eigen_v_Log1p_f32/10   31.8ns ±24%  34.9ns ±17%   +9.72%    (p=0.000 n=98+96)
BM_eigen_v_Log1p_f32/100   169ns ±10%   177ns ± 5%   +4.66%   (p=0.000 n=119+81)
BM_eigen_v_Log1p_f32/1k   1.42µs ± 4%  1.46µs ± 8%   +2.70%   (p=0.000 n=93+113)
BM_eigen_v_Log1p_f32/10k  14.4µs ± 5%  14.9µs ± 8%   +3.61%  (p=0.000 n=115+110)
```

Reviewed By: ezhulenev, ftynse

Differential Revision: https://reviews.llvm.org/D101765
2021-05-03 15:11:37 -07:00
Chris Lattner dc4e913be9 [PatternMatch] Big mechanical rename OwningRewritePatternList -> RewritePatternSet and insert -> add. NFC
This doesn't change APIs, this just cleans up the many in-tree uses of these
names to use the new preferred names.  We'll keep the old names around for a
couple weeks to help transitions.

Differential Revision: https://reviews.llvm.org/D99127
2021-03-22 17:20:50 -07:00
Chris Lattner 549e190236 [PatternRewriter] Rename OwningRewritePatternList -> RewritePatternSet and insert -> add
This maintains the old name to have minimal source impact on downstream codes, and
does not do the huge mechanical patch.  I expect the huge mechanical patch to land
sometime this week, but we can keep around the old names for a couple weeks to reduce
impact on downstream projects.

Differential Revision: https://reviews.llvm.org/D99119
2021-03-22 16:33:18 -07:00
Chris Lattner 6874726610 [PatternMatching] Add convenience insert method to OwningRewritePatternList. NFC.
This allows adding a C function pointer as a matchAndRewrite style pattern, which
is a very common case.  This adopts it in ExpandTanh to show how it reduces a level
of nesting.

We could allow C++ lambdas here, but that doesn't work as well with type inference
in the common case.  Instead of:

  patterns.insert(convertTanhOp);

you need to specify:

  patterns.insert<math::TanhOp>(convertTanhOp);

which is boilerplate'y.  Capturing state like this is very uncommon, so we choose
to require clients to define their own structs and use the non-convenience method
when they need to do so.

Differential Revision: https://reviews.llvm.org/D99039
2021-03-22 11:18:21 -07:00