Commit Graph

372 Commits

Author SHA1 Message Date
Javier Setoain cd0d21b47b [mlir][LLVM] Allow scalable vectors in ShuffleVectorOp
The current implementation of ShuffleVectorOp assumes all vectors are
scalable. LLVM IR allows shufflevector operations on scalable vectors,
and the current translation between LLVM Dialect and LLVM IR does the
rigth thing when the shuffle mask is all zeroes. This is required to
do a splat operation on a scalable vector, but it doesn't make sense
for scalable vectors outside of that operation, i.e.: with non-all zero
masks.

Differential Revision: https://reviews.llvm.org/D118371
2022-02-24 11:24:34 +00:00
Shraiysh Vaishay 5ee500acbb [mlir][OpenMP] Remove clauses that are not being handled
This patch removes the following clauses from OpenMP Dialect:

 - private
 - firstprivate
 - lastprivate
 - shared
 - default
 - copyin
 - copyprivate

The privatization clauses are being handled in the flang frontend. The
data copying clauses are not being handled anywhere for now. Once
we have a better picture of how to handle these clauses in OpenMP
Dialect, we can add these. For the time being, removing unneeded
clauses.

For detailed discussion about this refer to [[ https://discourse.llvm.org/t/rfc-privatisation-in-openmp-dialect/3526 | Privatisation in OpenMP dialect ]]

Reviewed By: kiranchandramohan, clementval

Differential Revision: https://reviews.llvm.org/D120029
2022-02-19 01:13:05 +05:30
Krzysztof Drewniak 5f6ce97408 [MLIR] Add support for alloca() of non-zero address spaces
This change is needed when lowering alloca()-using code on targets
such as ROCDL that represent private scratch space as a separate
address space.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D119775
2022-02-15 18:41:30 +00:00
Ivan Butygin 32389d0c2e [mlir][spirv] Add OpenCL fma op and lowering
Also, it seems Khronos has changed html spec format so small adjustment to script was needed.
Base op parsing is also probably broken.

Differential Revision: https://reviews.llvm.org/D119678
2022-02-15 11:28:20 +03:00
Marius Brehler 88b9d1a49a [mlir][emitc] Add a pointer type
Adds a pointer type to EmitC. The emission of pointers is so far only
possible by using the `emitc.opaque` type

Co-authored-by: Simon Camphausen <simon.camphausen@iml.fraunhofer.de>

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D119337
2022-02-14 16:42:21 +00:00
Markus Böck 1bf7921374 [mlir][LLVM] Add support for adding a garbage collector to a LLVM function
This patch simply adds an optional garbage collector attribute to LLVMFuncOp which maps 1:1 to the "gc" property of functions in LLVM.

Differential Revision: https://reviews.llvm.org/D119492
2022-02-11 10:23:51 +01:00
River Riddle ace01605e0 [mlir] Split out a new ControlFlow dialect from Standard
This dialect is intended to model lower level/branch based control-flow constructs. The initial set
of operations are: AssertOp, BranchOp, CondBranchOp, SwitchOp; all split out from the current
standard dialect.

See https://discourse.llvm.org/t/standard-dialect-the-final-chapter/6061

Differential Revision: https://reviews.llvm.org/D118966
2022-02-06 14:51:16 -08:00
Marius Brehler 578122c18a [mlir] Don't emit unused labels
Stop the Cpp target from emitting unused labels. The previosly generated
code generated warning if `-Wunused-label` is passed to a compiler.

Co-authored-by: Simon Camphausen <simon.camphausen@iml.fraunhofer.de>

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D118154
2022-01-27 07:56:03 +00:00
Chuanqi Xu dbbe010908 [MLIR] [AsyncToLLVM] Use llvm.coro.align intrinsic
Use llvm.coro.align to align coroutine frame properly.

Reviewed By: bkramer

Differential Revision: https://reviews.llvm.org/D117978
2022-01-25 19:28:25 +08:00
Peixin-Qiao a56a7d99e8 [MLIR][OpenMP] Support schedule chunk size with various bit width
The chunk size in schedule clause is one integer expression, which can
be either constant integer or integer variable. Fix schedule clause in
MLIR Op Def to support integer expression with different bit width.

Reviewed By: shraiysh

Differential Revision: https://reviews.llvm.org/D116073
2022-01-19 12:36:53 +08:00
Mogball aae5125550 [mlir] Replace StrEnumAttr -> EnumAttr in core dialects
Removes uses of `StrEnumAttr` in core dialects

Reviewed By: mehdi_amini, rriddle

Differential Revision: https://reviews.llvm.org/D117514
2022-01-18 17:15:00 +00:00
Mogball 5c36ee8d57 [mlir] Drop the leading space when printing regions
The leading space that is always printed at the beginning of regions is not consistent with other parts of the printing API. Moreover, this leading space can lead to undesirable assembly formats:

```
attr-dict-with-keyword $region
```

Prints as:

```
// Two spaces between `}` and `{`
attributes {foo}  { ... }
```

Moreover, the leading space results in the odd generic op format:

```
"test.op"() ( {...}) : () -> ()
```

Reviewed By: rriddle, mehdi_amini

Differential Revision: https://reviews.llvm.org/D117411
2022-01-18 16:52:34 +00:00
Alex Zinenko 1ad48d6de2 [mlir] handle nested regions in llvm-legalize-for-export
The translation from the MLIR LLVM dialect to LLVM IR includes a mechanism that
ensures the successors of a block to be different blocks in case block
arguments are passed to them since the opposite cannot be expressed in LLVM IR.
This mechanism previously only worked for functions because it was written
prior to the introduction of other region-carrying operations such as the
OpenMP dialect, which also translates directly to LLVM IR. Modify this
mechanism to handle all regions in the module and not only functions.

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D117548
2022-01-18 17:09:14 +01:00
Alex Zinenko bea16e72a7 [mlir] Fix invalid assertion in ModuleTranslation.cpp
LLVM dialect supports terminators with repeated successor blocks that take
different operands. This cannot be directly expressed in LLVM IR though since
it uses the number of the predecessor block to differentiate values in its PHI
nodes. Therefore, the translation to LLVM IR inserts dummy blocks to forward
arguments in case of repeated succesors with arguments. The insertion works
correctly. However, when connecting PHI nodes to their source values, the
assertion of the insertion having worked correctly was incorrect: it would only
trigger if repeated blocks were adjacent in the successor list (not guaranteed
by anything) and would not check if the successors have operands (no need for
dummy blocks in absence of operands since no PHIs are being created). Change
the assertion to only trigger in case of duplicate successors with operands,
and don't expect them to be adjacent.

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D117214
2022-01-14 12:56:33 +01:00
Peixin-Qiao f52805539b [OMPIRBuilder] Fix store inst alignment for ordered depend directive
OpenMP runtime requires depend vec with i64 type and the alignment of
store instruction should be set as 8.

Reviewed By: kiranchandramohan, shraiysh

Differential Revision: https://reviews.llvm.org/D116300
2022-01-14 09:47:11 +08:00
Javier Setoain 7c56458616 [mlir] Fix scalable type translation in splat element attr
LLVM Dialect Constant Op translations assume that if the attribute is a
vector, it's a fixed length one, generating an invalid translation for
constant scalable vector initializations.

Differential Revision: https://reviews.llvm.org/D117125
2022-01-13 11:14:41 +00:00
Shraiysh Vaishay a8586b573e [mlir][OpenMP] Change the syntax of omp.atomic.read op
This patch changes the syntax of omp.atomic.read to take the address of
destination, instead of having the value in a result. This will allow
using omp.atomic.read operation within an omp.atomic.capture operation
thus making its implementation less complex.

Reviewed By: peixin

Differential Revision: https://reviews.llvm.org/D116396
2022-01-10 16:19:45 +05:30
Alex Zinenko f50cfc44d6 [mlir] Require struct indices in LLVM::GEPOp to be constant
Recent commits added a possibility for indices in LLVM dialect GEP operations
to be supplied directly as constant attributes to ensure they remain such until
translation to LLVM IR happens. Make this required for indexing into LLVM
struct types to match LLVM IR requirements, otherwise the translation would
assert on constructing such IR.

For better compatibility with MLIR-style operation construction interface,
allow GEP operations to be constructed programmatically using Values pointing
to known constant operations as struct indices.

Depends On D116758

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D116759
2022-01-07 09:56:05 +01:00
Alex Zinenko cafaa35036 [mlir] Make it possible to directly supply constant values to LLVM GEPOp
In LLVM IR, the GEP indices that correspond to structures are required to be
i32 constants. MLIR models constants as just values defined by special
operations, and there is no verification that it is the case for structure
indices in GEP. Furthermore, some common transformations such as control flow
simplification may lead to the operands becoming non-constant. Make it possible
to directly supply constant values to LLVM GEPOp to guarantee they remain
constant until the translation to LLVM IR. This is not yet a requirement and
the verifier is not modified, this will be introduced separately.

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D116757
2022-01-07 09:56:01 +01:00
Shraiysh Vaishay 6bcb4c44de [mlir][OpenMP] Added omp.atomic.write lowering to LLVM IR
This patch adds omp.atomic.write lowering to LLVM IR.
Also, changed the syntax to have equal symbol instead of the comma to
make it more intuitive.

Reviewed By: kiranchandramohan, peixin

Differential Revision: https://reviews.llvm.org/D116416
2022-01-07 10:01:57 +05:30
Markus Böck 560972052a [mlir][LLVM] Implement mapping of phi source values of `llvm.invoke`
This patch allows the usage of the normalDestOperands and unwindDestOperands operands of llvm.invoke and have them be correctly mapped to phis in the successor when exported to LLVM IR.

Differential Revision: https://reviews.llvm.org/D116706
2022-01-06 11:27:14 +01:00
Markus Böck 2a0e05100c [mlir][LLVM] Set cleanup flag on `llvm.landingpad` when exporting to LLVM IR
Exporting a llvm.landingpad operation with the cleanup flag set is currently ignored by the export code.

Differential Revision: https://reviews.llvm.org/D116565
2022-01-04 08:19:26 +01:00
Markus Böck c343c200ea [mlir][LLVM] Fix mapping of result values of `llvm.invoke` during export
The result value of a llvm.invoke operation is currently not mapped to the corresponding llvm::Value* when exporting to LLVM IR. This leads to any later operations using the result to crash as it receives a nullptr.

Differential Revision: https://reviews.llvm.org/D116564
2022-01-03 23:53:01 +01:00
Markus Böck 3536d24a1a [mlir][LLVMIR] Add `llvm.eh.typeid.for` intrinsic
MLIR already exposes landingpads, the invokeop and the personality function on LLVM functions. With this intrinsic it should be possible to implement exception handling via the exception handling mechanisms provided by the Itanium ABI.

Differential Revision: https://reviews.llvm.org/D116436
2022-01-01 02:03:00 +01:00
Johannes Doerfert 7e14e881c4 [OpenMP][OpenACC] Update test after encoding change in D113126 2021-12-29 01:29:07 -06:00
William S. Moses 2709fd1520 [MLIR][LLVM] Add MemmoveOp to LLVM Dialect
LLVM Dialect in MLIR doesn't have a memmove op. This adds one.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D116274
2021-12-24 19:48:04 -05:00
Markus Böck 1d10bddfa3 [mlir][LLVMIR] Add `llvm.umin` and `llvm.umax` intrinsics
Ops for the signed counterparts "llvm.smin" and "llvm.smax" already exist. This patch adds the unsigned versions as well.

Differential Revision: https://reviews.llvm.org/D115796
2021-12-15 13:54:31 +01:00
Shraiysh Vaishay 3425b1bcb4 [mlir][OpenMP] omp.sections and omp.section lowering to LLVM IR
This patch adds lowering from omp.sections and omp.section (simple lowering along with the nowait clause) to LLVM IR.
Tests for the same are also added.

Reviewed By: ftynse, kiranchandramohan

Differential Revision: https://reviews.llvm.org/D115030
2021-12-15 15:41:12 +05:30
Javier Setoain a4830d14ed [mlir][RFC] Add scalable dimensions to VectorType
With VectorType supporting scalable dimensions, we don't need many of
the operations currently present in ArmSVE, like mask generation and
basic arithmetic instructions. Therefore, this patch also gets
rid of those.

Having built-in scalable vector support also simplifies the lowering of
scalable vector dialects down to LLVMIR.

Scalable dimensions are indicated with the scalable dimensions
between square brackets:

        vector<[4]xf32>

Is a scalable vector of 4 single precission floating point elements.

More generally, a VectorType can have a set of fixed-length dimensions
followed by a set of scalable dimensions:

        vector<2x[4x4]xf32>

Is a vector with 2 scalable 4x4 vectors of single precission floating
point elements.

The scale of the scalable dimensions can be obtained with the Vector
operation:

        %vs = vector.vscale

This change is being discussed in the discourse RFC:

https://llvm.discourse.group/t/rfc-add-built-in-support-for-scalable-vector-types/4484

Differential Revision: https://reviews.llvm.org/D111819
2021-12-15 09:31:37 +00:00
Krzysztof Drewniak c57b2a0635 [MLIR][GPU] Make max flat work group size for ROCDL kernels configurable
While the default value for the amdgpu-flat-work-group-size attribute,
"1, 256", matches the defaults from Clang, some users of the ROCDL dialect,
namely Tensorflow, use larger workgroups, such as 1024. Therefore,
instead of hardcoding this value, we add a rocdl.max_flat_work_group_size
attribute that can be set on GPU kernels to override the default value.

Reviewed By: whchung

Differential Revision: https://reviews.llvm.org/D115741
2021-12-14 20:12:23 +00:00
Lei Zhang 5e55a20119 [mlir][spirv] Serialize selection with separate header block
The previous "optimization" that tries to reuse existing block for
selection header block can be problematic for deserialization
because it effectively pulls in previous ops in the selection op's
enclosing block into the selection op's header. When deserializing,
those ops will be placed in the selection op's region. If any of
the previous ops has usage after the section op, it will break. That
is, the following IR cannot round trip:

```mlir
^bb:
  %def = ...
  spv.mlir.selection { ... }
  %use = spv.SomeOp %def
```

This commit removes the "optimization" to always create new blocks
for the selection header.

Along the way, also made error reporting better in deserialization
by turning asserts into proper errors and add check of uses outside
of sinked structured control flow region blocks.

Reviewed By: Hardcode84

Differential Revision: https://reviews.llvm.org/D115582
2021-12-13 10:42:26 -05:00
Lei Zhang 731676b10d [mlir][spirv] Fix nested control flow serialization
If we have a `spv.mlir.selection` op nested in a `spv.mlir.loop`
op, when serializing the loop's block, we might need to jump
from the selection op's merge block, which might be different
than the immediate MLIR IR predecessor block. But we still need
to get the block argument from the MLIR IR predecessor block.

Also, if the `spv.mlir.selection` is in the `spv.mlir.loop`'s
header block, we need to make sure `OpLoopMerge` is emitted
in the current block before start processing the nested selection
op. Otherwise we'll see the LoopMerge in the wrong SPIR-V
basic block.

Reviewed By: Hardcode84

Differential Revision: https://reviews.llvm.org/D115560
2021-12-11 14:47:19 -05:00
Rob Suderman 23149d522b [mlir] Added ctlz and cttz to math dialect and LLVM dialect
Count leading/trailing zeros are an existing LLVM intrinsic. Added LLVM
support for the intrinsics with lowerings from the math dialect to LLVM
dialect.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115206
2021-12-08 14:32:15 -08:00
Butygin d8fce785de [mlir][spirv] math.erf OpenCL lowering
Differential Revision: https://reviews.llvm.org/D115335
2021-12-08 21:59:46 +03:00
Thomas Raoux 579c1ff67d [mlir][nvvm] Add async copy ops to nvvm dialect
Differential Revision: https://reviews.llvm.org/D115314
2021-12-08 09:42:20 -08:00
Shraiysh Vaishay 31cf42bd9a [mlir][OpenMP] Added omp.atomic.read lowering
This patch adds lowering from omp.atomic.read to LLVM IR along with the
memory ordering clause. Tests for the same are also added.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D115134
2021-12-07 11:17:30 +05:30
Mats Petersson 30238c3676 [mlir][OpenMP] Add support for SIMD modifier
Add support for SIMD modifier in OpenMP worksharing loops.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D111051
2021-11-26 14:04:46 +00:00
Butygin 75a1bee05d [mlir][spirv] Add math to OpenCL conversion
Differential Revision: https://reviews.llvm.org/D113780
2021-11-24 02:31:21 +03:00
Thomas Raoux 47555d73f6 [mlir][gpu] Extend shuffle op modes and add nvvm lowering
Add up, down and idx modes to gpu shuffle ops, also change the mode from
string to enum

Differential Revision: https://reviews.llvm.org/D114188
2021-11-19 11:14:31 -08:00
William S. Moses 30d87d4a5d [MLIR][LLVM] Permit integer types in switch other than i32
LLVM switchop currently only permits i32. Both LLVM IR and MLIR Standard switch permit other integer types leading to an illegal state when lowering an i8 switch from MLIR standard

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D113955
2021-11-16 12:00:37 -05:00
Butygin 6c48f6aafe [mlir][spirv] add AtomicFAddEXTOp
Differential Revision: https://reviews.llvm.org/D113764
2021-11-16 14:24:22 +03:00
Mehdi Amini 1585b13024 Revert "[MLIR][LLVM] Permit integer types in switch other than i32"
This reverts commit 94992670fc.
Build is broken with:

tools/mlir/include/mlir/Dialect/LLVMIR/LLVMOps.cpp.inc:23996:3: error: no matching function for call to 'printSwitchOpCases'
  printSwitchOpCases(_odsPrinter, *this, getValue().getType(), getCaseValuesAttr(), getCaseDestinations(), getCaseOperands(), getCaseOperands().getTypes());
  ^~~~~~~~~~~~~~~~~~
2021-11-16 05:59:12 +00:00
William S. Moses 94992670fc [MLIR][LLVM] Permit integer types in switch other than i32
LLVM switchop currently only permits i32. Both LLVM IR and MLIR Standard switch permit other integer types leading to an illegal state when lowering an i8 switch from MLIR standard

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D113955
2021-11-16 00:46:25 -05:00
Butygin 1cb13fddb9 [mlir] spirv: Add some atomic ops
Differential Revision: https://reviews.llvm.org/D112812
2021-11-03 14:47:12 +03:00
thomasraoux 77eafb8430 [mlir][nvvm] Generalize wmma ops to handle more types and shapes
wmma intrinsics have a large number of combinations, ideally we want to be able
to target all the different variants. To avoid a combinatorial explosion in the
number of mlir op we use attributes to represent the different variation of
load/store/mma ops. We also can generate with tablegen helpers to know which
combinations are available. Using this we can avoid having too hardcode a path
for specific shapes and can support more types.
This patch also adds boiler plates for tf32 op support.

Differential Revision: https://reviews.llvm.org/D112689
2021-11-01 10:27:26 -07:00
Uday Bondhugula 57b9b29649 [MLIR][LLVM] Add llvm.mlir.global_ctors/dtors and translation support
Add llvm.mlir.global_ctors and global_dtors ops and their translation
support to LLVM global_ctors/global_dtors global variables.

Differential Revision: https://reviews.llvm.org/D112524
2021-10-28 18:09:34 +05:30
Roman Lebedev 42712698fd
Revert "[IR] `IRBuilderBase::CreateAdd()`: short-circuit `x + 0` --> `x`"
Clang OpenMP codegen tests are failing.

This reverts commit 288f1f8abe.
This reverts commit cb90e5356a.
2021-10-27 22:21:37 +03:00
Roman Lebedev 288f1f8abe
Fix MLIR LLVMIR test after 4723c9b3c6 2021-10-27 21:52:56 +03:00
Shraiysh Vaishay a81672b31a [NFC][MLIR][OpenMP] Splitting the WsLoop tests.
Splitting the WsLoop tests they were getting harder to debug with the offsets over 100 for some of them.

Reviewed By: clementval

Differential Revision: https://reviews.llvm.org/D112407
2021-10-25 14:00:36 +05:30
Jacques Pienaar 42e9af9e8f [mlir] Rename to avoid overlap in accessor prefixing
Split out renaming from D112383 into standalone change.
2021-10-24 18:17:09 -07:00
Mats Petersson 3f00e10bdd [mlir][OpenMP]Support for modifiers in workshare loops
Pass the modifiers from the Flang parser to FIR/MLIR workshare
loop operation.

Not yet supporting the SIMD modifier, which is a bit more work
than just adding it to the list of modifiers, so will go in a
separate patch.

This adds a new field to the WsLoopOp.

Also add test for dynamic WSLoop, checking that dynamic schedule calls
the init and next functions as expected.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D111053
2021-10-22 14:19:33 +01:00
Benjamin Kramer 898e80964c [mlir] Fix a crash when creating a 1d zero element LLVM constant
Fixes a regression introduced in f9be7a7afd

Differential Revision: https://reviews.llvm.org/D112208
2021-10-21 12:55:08 +02:00
Peixin-Qiao b37e5187f2 [MLIR][OpenMP] Add support for ordered construct
This patch supports the ordered construct in OpenMP dialect following
Section 2.19.9 of the OpenMP 5.1 standard. Also lowering to LLVM IR
using OpenMP IRBduiler. Lowering to LLVM IR for ordered simd directive
is not supported yet since LLVM optimization passes do not support it
for now.

Reviewed By: kiranchandramohan, clementval, ftynse, shraiysh

Differential Revision: https://reviews.llvm.org/D110015
2021-10-21 16:30:46 +08:00
Mehdi Amini cb11ddb96c Revert "[MLIR][OpenMP] Add support for ordered construct"
This reverts commit dc2be87ecf.

Seems like this broke all the CI bots.
2021-10-21 04:53:45 +00:00
Peixin-Qiao dc2be87ecf [MLIR][OpenMP] Add support for ordered construct
This patch supports the ordered construct in OpenMP dialect following
Section 2.19.9 of the OpenMP 5.1 standard. Also lowering to LLVM IR
using OpenMP IRBduiler. Lowering to LLVM IR for ordered simd directive
is not supported yet since LLVM optimization passes do not support it
for now.

Reviewed By: kiranchandramohan, clementval, ftynse, shraiysh

Differential Revision: https://reviews.llvm.org/D110015
2021-10-21 09:16:04 +08:00
Shraiysh Vaishay c4c7e06bd7 [MLIR][OpenMP] Shifted hint from CriticalOp to CriticalDeclareOp
According to the OpenMP 5.0 standard, names and hints of critical operation are
closely related. The following are the restrictions on them:
 - Unless the effect is as if `hint(omp_sync_hint_none)` was specified, the
   critical construct must specify a name.
 - If the hint clause is specified, each of the critical constructs with the
   same name must have a hint clause for which the hint-expression evaluates to
   the same value.

These restrictions will be enforced by design if the hint expression is a part
of the `omp.critical.declare` operation.
 - Any operation with no "name" will be considered to have
   `hint(omp_sync_hint_none)`.
 - All the operations with the same "name" will have the same hint value.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D112134
2021-10-20 21:36:09 +05:30
Caitlyn Cano 2ea5e7ba57 [mlir] SPIR-V: add sin, cos, log, sqrt OCL ops
Differential Revision: https://reviews.llvm.org/D111884
2021-10-18 20:48:59 +03:00
William S. Moses 40b9c39db1 [MLIR][LLVM] Add memset intrinsic
Add memset intrinsic into LLVM dialect

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D111906
2021-10-16 18:20:48 -04:00
Mogball a54f4eae0e [MLIR] Replace std ops with arith dialect ops
Precursor: https://reviews.llvm.org/D110200

Removed redundant ops from the standard dialect that were moved to the
`arith` or `math` dialects.

Renamed all instances of operations in the codebase and in tests.

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D110797
2021-10-13 03:07:03 +00:00
Lei Zhang 7a89444cd9 [mlir][spirv] Add ops and patterns for lowering standard max/min ops
Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D111143
2021-10-05 14:27:32 -04:00
Weiwei Li 1e4cfe5e4f [mlir][SPIRVToLLVM] Propagate location attribute from spv.GlobalVariable to llvm.mlir.global
This patch is mainly to propogate location attribute from spv.GlobalVariable to llvm.mlir.global.

It also contains three small changes.

1. Remove the restriction on UniformConstant In SPIRVToLLVM.cpp;
2. Remove the errorCheck on relaxedPrecision when deserializering SPIR-V in Deserializer.cpp
3. In SPIRVOps.cpp, let ConstantOp take signedInteger too.

Co-authered: Alan Liu <alanliu.yf@gmail.com> and Xinyi Liu <xyliuhelen@gmail.com>

Reviewed by:antiagainst

Differential revision: https://reviews.llvm.org/D110207
2021-10-05 00:09:09 +08:00
Alex Zinenko bdaf038266 [mlir] Always create a list of alias scopes when emitting LLVM IR
Previously, the translation to LLVM IR would emit IR that directly uses
a scope metadata node in case only one scope was in use in alias.scopes
or noalias metadata. It should always be a list of scopes. The verifier
change in 8700f2bd36 enforced this and
broke the test. Fix the translation to always create a list of scopes
using a new metadata node, update and reenable the respective test.

Fixes PR51919.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D110140
2021-09-22 00:00:46 +02:00
River Riddle 4f21152af1 [mlir] Tighten verification of SparseElementsAttr
SparseElementsAttr currently does not perform any verfication on construction, with the only verification existing within the parser. This revision moves the parser verification to SparseElementsAttr, and also adds additional verification for when a sparse index is not valid.

Differential Revision: https://reviews.llvm.org/D109189
2021-09-21 01:57:42 +00:00
Mehdi Amini f18f1ab4fd Temporarily XFAIL MLIR test that fails the LLVM verifier after 8700f2bd3 2021-09-20 17:20:11 +00:00
Valentin Clement d6929aaa67
[mlir][openacc] Make use of the second counter extension in DataOp translation
Make use of runtime extension for the second reference counter used in
structured data region. This extension is implemented in D106510 and D106509.

Differential Revision: https://reviews.llvm.org/D106517
2021-09-20 13:43:50 +02:00
Simon Camphausen 1b79efdc72 [mlir] Fix printing of EmitC attrs/types with escape characters
Attributes and types were not escaped when printing.

Reviewed By: jpienaar, marbre

Differential Revision: https://reviews.llvm.org/D109143
2021-09-15 18:15:38 +00:00
Simon Camphausen ec92f788f3 [mlir][emitc] Print signed integers properly
Previously negative integers were printed as large unsigned values.

Reviewed By: marbre

Differential Revision: https://reviews.llvm.org/D109690
2021-09-13 15:29:30 +00:00
Alex Zinenko 8b58ab8ccd [mlir] Factor type reconciliation out of Standard-to-LLVM conversion
Conversion to the LLVM dialect is being refactored to be more progressive and
is now performed as a series of independent passes converting different
dialects. These passes may produce `unrealized_conversion_cast` operations that
represent pending conversions between built-in and LLVM dialect types.
Historically, a more monolithic Standard-to-LLVM conversion pass did not need
these casts as all operations were converted in one shot. Previous refactorings
have led to the requirement of running the Standard-to-LLVM conversion pass to
clean up `unrealized_conversion_cast`s even though the IR had no standard
operations in it. The pass must have been also run the last among all to-LLVM
passes, in contradiction with the partial conversion logic. Additionally, the
way it was set up could produce invalid operations by removing casts between
LLVM and built-in types even when the consumer did not accept the uncasted
type, or could lead to cryptic conversion errors (recursive application of the
rewrite pattern on `unrealized_conversion_cast` as a means to indicate failure
to eliminate casts).

In fact, the need to eliminate A->B->A `unrealized_conversion_cast`s is not
specific to to-LLVM conversions and can be factored out into a separate type
reconciliation pass, which is achieved in this commit. While the cast operation
itself has a folder pattern, it is insufficient in most conversion passes as
the folder only applies to the second cast. Without complex legality setup in
the conversion target, the conversion infra will either consider the cast
operations valid and not fold them (a separate canonicalization would be
necessary to trigger the folding), or consider the first cast invalid upon
generation and stop with error. The pattern provided by the reconciliation pass
applies to the first cast operation instead. Furthermore, having a separate
pass makes it clear when `unrealized_conversion_cast`s could not have been
eliminated since it is the only reason why this pass can fail.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D109507
2021-09-09 16:51:24 +02:00
Mehdi Amini 78accf9f35 Make LLVM Linkage a first class attribute instead of using an integer attribute
This makes the IR more readable, in particular when this will be used on
the builtin func outside of the LLVM dialect.

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D109209
2021-09-03 21:21:46 +00:00
Alex Zinenko f9be7a7afd [mlir] speed up construction of LLVM IR constants when possible
The translation to LLVM IR used to construct sequential constants by recurring
down to individual elements, creating constant values for them, and wrapping
them into aggregate constants in post-order. This is highly inefficient for
large constants with known data such as DenseElementsAttr. Use LLVM's
ConstantData for the innermost dimension instead. LLVM does seem to support
data constants for nested sequential constants so the outer dimensions are
still handled recursively. Nevertheless, this speeds up the translation of
large constants with equal dimensions by up to 30x.

Users are advised to rewrite large constants to use flat types before
translating to LLVM IR if more efficiency in translation is necessary. This is
not done automatically as the translation is not aware of the expectations of
the overall compilation flow about type changes and indexing, in particular for
global constants with external linkage.

Reviewed By: silvas

Differential Revision: https://reviews.llvm.org/D109152
2021-09-02 23:07:30 +02:00
Kiran Chandramohan 711aa35759 [MLIR][OpenMP] Add support for declaring critical construct names
Add an operation omp.critical.declare to declare names/symbols of
critical sections. Named omp.critical operations should use symbols
declared by omp.critical.declare. Having a declare operation ensures
that the names of critical sections are global and unique. In the
lowering flow to LLVM IR, the OpenMP IRBuilder creates unique names
for critical sections.

Reviewed By: ftynse, jeanPerier

Differential Revision: https://reviews.llvm.org/D108713
2021-09-02 14:31:19 +00:00
Marius Brehler 2f0750dd2e [mlir] Add Cpp emitter
This upstreams the Cpp emitter, initially presented with [1], from [2]
to MLIR core. Together with the previously upstreamed EmitC dialect [3],
the target allows to translate MLIR to C/C++.

[1] https://reviews.llvm.org/D76571
[2] https://github.com/iml130/mlir-emitc
[3] https://reviews.llvm.org/D103969

Co-authored-by: Jacques Pienaar <jpienaar@google.com>
Co-authored-by: Simon Camphausen <simon.camphausen@iml.fraunhofer.de>
Co-authored-by: Oliver Scherf <oliver.scherf@iml.fraunhofer.de>

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D104632
2021-09-02 13:51:05 +00:00
Alex Zinenko 8647e4c3a0 [mlir] support translating OpenMP loops with reductions
Use the recently introduced OpenMPIRBuilder facility to transate OpenMP
workshare loops with reductions to LLVM IR calling OpenMP runtime. Most of the
heavy lifting is done at the OpenMPIRBuilder. When other OpenMP dialect
constructs grow support for reductions, the translation can be updated to
operate on, e.g., an operation interface for all reduction containers instead
of workshare loops specifically. Designing such a generic translation for the
single operation that currently supports reductions is premature since we don't
know how the reduction modeling itself will be generalized.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D107343
2021-09-02 15:38:20 +02:00
Tyler Augustine d25e91d7f6 Support alias.scope and noalias metadata
Introduces new Ops to represent 1. alias.scope metadata in LLVM, and 2. domains for these scopes. These correspond to the metadata described in https://llvm.org/docs/LangRef.html#noalias-and-alias-scope-metadata. Lists of scopes are modeled the same way as access groups - as an ArrayAttr on the Op (added in https://reviews.llvm.org/D97944).

Lowering 'noalias' attributes on function parameters is already supported. However, lowering `noalias` metadata on individual Ops is not, which is added in this change. LLVM uses the same keyword for these, but this change introduces a separate attribute name 'noalias_scopes' to represent this distinct concept.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D107870
2021-08-24 20:42:59 +02:00
Florian Hahn f999312872
Recommit "[Matrix] Overload stride arg in matrix.columnwise.load/store."
This reverts the revert 28c04794df.

The failing MLIR test that caused the revert should be fixed  in this
version.

Also includes a PPC test fix previously in 1f87c7c478.
2021-08-12 18:31:57 +01:00
Alex Zinenko c4c1030976 [mlir] support collapsed loops in OpenMP-to-LLVM translation
Reviewed By: Meinersbur

Differential Revision: https://reviews.llvm.org/D105706
2021-08-06 17:13:12 +02:00
Kiran Chandramohan 59989d68ba [MLIR][OpenMP] Add support for critical construct
This patch adds the critical construct to the OpenMP dialect. The
implementation models the definition in 2.17.1 of the OpenMP 5 standard.
A name and hint can be specified. The name is a global entity or has
external linkage, it is modelled as a FlatSymbolRefAttr. Hint is
modelled as an integer enum attribute.
Also lowering to LLVM IR using the OpenMP IRBuilder.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D107135
2021-08-03 10:50:21 +01:00
Eli Friedman 6eb2ffbaeb Fix a couple regression tests I missed updating in 2a284782 2021-07-31 13:41:15 -07:00
Lei Zhang 233a3a75ac [mlir][spirv] Add support for i8 serialization
Reviewed By: hanchung

Differential Revision: https://reviews.llvm.org/D107122
2021-07-30 18:06:11 -04:00
Valentin Clement fe7ca1a9fc [mlir][openacc] Initial translation for DataOp to LLVM IR
Add basic translation of acc.data to LLVM IR with runtime calls.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D104301
2021-07-27 22:04:04 -04:00
Weiwei Li 108a320a58 [mlir][spirv] Add support for GLSL FMix
Add spv.GLSL.FMix opertaion.

co-authered-by: Alan Liu <alanliu.yf@gmail.com>

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D104153
2021-07-16 08:29:46 +08:00
Alex Zinenko c282d55a38 [mlir] add support for reductions in OpenMP WsLoopOp
Use a modeling similar to SCF ParallelOp to support arbitrary parallel
reductions. The two main differences are: (1) reductions are named and declared
beforehand similarly to functions using a special op that provides the neutral
element, the reduction code and optionally the atomic reduction code; (2)
reductions go through memory instead because this is closer to the OpenMP
semantics.

See https://llvm.discourse.group/t/rfc-openmp-reduction-support/3367.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D105358
2021-07-09 17:54:20 +02:00
Alex Zinenko d4df3825bd [mlir] don't drop undef initializers in translation to LLVM IR
LLVM IR allows globals with external linkage to have initializers, including
undef. The translation was incorrectly using undef as a indicator that the
initializer should be ignored in translation, leading to the impossibility to
create an external global with an explicit undef initializer. Fix this and use
nullptr as a marker instead.

Reviewed By: wsmoses

Differential Revision: https://reviews.llvm.org/D105631
2021-07-09 17:52:43 +02:00
Sean Silva da289a174f [mlir] Allow conversion to LLVM for ElementsAttr's with size 0
The code seems to correctly handle the case that this assertion was
guarding.

Differential Revision: https://reviews.llvm.org/D105379
2021-07-07 11:15:20 -07:00
Felipe de Azevedo Piovezan 8ca04b0513 [mlir] Add support for LLVM's dso_local attr
This patch brings support for setting runtime preemption specifiers of
LLVM's GlobalValues. In LLVM semantics, if the `dso_local` attribute
is not explicitly requested, then it is inferred based on linkage and
visibility. We model this same behavior with a UnitAttribute: if it is
present, then we explicitly request the GlobalValue to marked as
`dso_local`, otherwise we rely on the GlobalValue itself to make this
decision.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D104983
2021-06-29 15:00:48 +02:00
Benoit Jacob 20daedacca 2d Arm Neon sdot op, and lowering to the intrinsic.
This adds Sdot2d op, which is similar to the usual Neon
intrinsic except that it takes 2d vector operands, reflecting the
structure of the arithmetic that it's performing: 4 separate
4-dimensional dot products, whence the vector<4x4xi8> shape.

This also adds a new pass, arm-neon-2d-to-intr, lowering
this new 2d op to the 1d intrinsic.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D102504
2021-06-10 14:36:39 -07:00
Javier Setoain 96ca2d92b5 [mlir][ArmSVE] Add basic load/store operations
ArmSVE-specific memory operations are needed to generate end-to-end
code for as long as MLIR core doesn't support scalable vectors. This
instructions will be eventually unnecessary, for now they're required
for more complex testing.

Differential Revision: https://reviews.llvm.org/D103535
2021-06-09 15:53:40 +01:00
Javier Setoain f880bd261f [mlir][ArmSVE] Add basic mask generation operations
These `arm_sve.cmp` functions are needed to generate scalable vector
masks as long as scalable vectors are not part of the standard types.
Once in standard, these can be removed and `std.cmp` can be used
instead.

Differential Revision: https://reviews.llvm.org/D103473
2021-06-09 09:56:53 +01:00
Javier Setoain 57546f5b22 Revert "[mlir][ArmSVE] Add basic mask generation operations"
This reverts commit 392af6a78b
2021-06-08 10:02:19 +01:00
Javier Setoain 392af6a78b [mlir][ArmSVE] Add basic mask generation operations
These `arm_sve.cmp` functions are needed to generate scalable vector
masks as long as scalable vectors are not part of the standard types.
Once in standard, these can be removed and `std.cmp` can be used
instead.

Differential Revision: https://reviews.llvm.org/D103473
2021-06-08 08:56:31 +01:00
Valentin Clement 1005ef445d [mlir][openacc] Translate UpdateOp to LLVM IR
Add translation to LLVM IR for the UpdateOp with host and device operands.
Translation is done with call using the runtime. This is done in a similar way as
D101504 and D102381.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D102382
2021-05-26 11:42:15 -04:00
Valentin Clement ab5ff154ab [mlir][openacc] Translate ExitDataop to LLVM IR
Translate ExitDataOp with delete and copyout operands to runtime call.
This is done in a similar way as D101504.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D102381
2021-05-17 11:11:59 -04:00
Adrian Kuegel 5ef21506b9 Add support for complex constants to MLIR core.
BEGIN_PUBLIC
Add support for complex constants to MLIR core.
END_PUBLIC

Differential Revision: https://reviews.llvm.org/D101908
2021-05-17 09:12:39 +02:00
Benoit Jacob e0a88db545 Fix some typos.
Fix some typos

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D102503
2021-05-14 21:34:09 +05:30
Weiwei Li cd0eeb52ad [mlir][spirv] Define spv.ImageQuerySize operation
Support OpImageQuerySize in spirv dialect

co-authored-by: Alan Liu <alanliu.yf@gmail.com>

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D102029
2021-05-13 13:17:08 -04:00
Valentin Clement 113b807017 [mlir][openacc] Add OpenACC translation to LLVM IR (enter_data op create/copyin)
This patch begins to translate acc.enter_data operation to call to tgt runtime call.
It currently only translate create/copyin operands of memref type. This acts as a basis to add support
for FIR types in the Flang/OpenACC support. It follows more or less a similar path than clang
with `omp target enter data map` directives.
This patch is taking a different approach than D100678 and perform a translation to LLVM IR
and make use of the OpenMPIRBuilder instead of doing a conversion to the LLVMIR dialect.

OpenACC support in Flang will rely on the current OpenMP runtime where 1:1 lowering can be
applied. Some extension will be added where features are not available yet.

Big part of this code will be shared for other standalone data operations in the OpenACC
dialect such as acc.exit_data and acc.update.

It is likely that parts of the lowering can also be shared later with the ops for
standalone data directives in the OpenMP dialect when they are introduced.

This is an initial translation and it probably needs more work.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D101504
2021-05-12 13:41:14 -04:00
Dumitru Potop 9a0ea5994b [mlir] Support alignment in LLVM dialect GlobalOp
First step in adding alignment as an attribute to MLIR global definitions. Alignment can be specified for global objects in LLVM IR. It can also be specified as a named attribute in the LLVMIR dialect of MLIR. However, this attribute has no standing and is discarded during translation from MLIR to LLVM IR. This patch does two things: First, it adds the attribute to the syntax of the llvm.mlir.global operation, and by doing this it also adds accessors and verifications. The syntax is "align=XX" (with XX being an integer), placed right after the value of the operation. Second, it allows transforming this operation to and from LLVM IR. It is checked whether the value is an integer power of 2.

Reviewed By: ftynse, mehdi_amini

Differential Revision: https://reviews.llvm.org/D101492
2021-05-12 09:07:20 +02:00
Uday Bondhugula 1c777ab459 [MLIR] Switch llvm.noalias to a unit attribute
Switch llvm.noalias attribute from a boolean attribute to a unit
attribute.

Differential Revision: https://reviews.llvm.org/D102225
2021-05-11 15:41:09 +05:30
Mats Petersson 7280f4b279 [OpenMP][MLIR]Add support for guided, auto and runtime scheduling
When using parallel loop construct, the OpenMP specification allows for
guided, auto and runtime as scheduling variants (as well as static and
dynamic which are already supported).

This adds the translation from MLIR to LLVM-IR for these scheduling
variants.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D101435
2021-05-10 09:18:52 +00:00
Alex Zinenko 72d013dd73 [mlir] OpenMP-to-LLVM: properly set outer alloca insertion point
Previously, the OpenMP to LLVM IR conversion was setting the alloca insertion
point to the same position as the main compuation when converting OpenMP
`parallel` operations. This is problematic if, for example, the `parallel`
operation is placed inside a loop and would keep allocating on stack on each
iteration leading to stack overflow.

Reviewed By: kiranchandramohan

Differential Revision: https://reviews.llvm.org/D101307
2021-05-10 10:04:52 +02:00