Commit Graph

485 Commits

Author SHA1 Message Date
Jon Chesterfield 626a31de15 [libomptarget] Add register usage info to kernel metadata
Add register usage information to the runtime metadata so that it can be used during kernel launch (that change will be in a different commit). Add this information to the kernel trace.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D98829
2021-03-18 17:00:42 +00:00
Jon Chesterfield dbf8f2b089 Revert "[libomptarget] Build amdgcn devicertl by default"
This reverts commit e23f3502d9.
It broke the build of openmp for clang built without amdgcn
support. D98746, under review, would allow this to reland.
2021-03-17 11:34:44 +00:00
Johannes Doerfert 0a954a528b [OpenMP][FIX] Repair accidental replacement of _shfl_sync with _shfl
This was broken accidentally in D95752.

Reviewed By: ye-luo

Differential Revision: https://reviews.llvm.org/D98677
2021-03-15 22:46:00 -05:00
Jon Chesterfield e23f3502d9 [libomptarget] Build amdgcn devicertl by default
[libomptarget] Build amdgcn devicertl by default

The cmake for this looks for an llvm install and does the right thing when
building as part of enable_runtimes. It will probably do the right thing
in other settings - at least, it won't try to build this with gcc.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D98658
2021-03-15 23:17:50 +00:00
Jon Chesterfield bb38d7ff05 [libomptarget][nfc][amdgcn] Use precise triple for devicertl build 2021-03-15 20:24:13 +00:00
Jon Chesterfield d0bc85f04a [libomptarget][nfc] Drop unused DEVICE macro
[libomptarget][nfc] Drop unused DEVICE macro

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D98655
2021-03-15 20:12:50 +00:00
Jon Chesterfield 7da76aaaf4 [libomptarget] Build amdgpu plugin by default
[libomptarget] Build amdgpu plugin by default

This will build the amdgpu plugin if cmake is able to find the hsa
runtime library, which will be the case if rocm is installed or if
the hsa library has been installed somewhere cmake looks.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D98654
2021-03-15 20:12:01 +00:00
Jon Chesterfield bcb3f0f867 [libomptarget] Fix devicertl build
[libomptarget] Fix devicertl build

The target specific functions in target_interface are extern C, but the
implementations for nvptx were mostly C++ mangling. That worked out as
a quirk of DEVICE macro expanding to nothing, except for shuffle.h which
only forward declared the functions with C++ linkage.

Also implements GetWarpSize, as used by shuffle, and includes target_interface
in nvptx target_impl.cu to help catch future divergence between interface and
implementation.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D98651
2021-03-15 19:50:22 +00:00
Jon Chesterfield f675b3df48 [libomptarget] Drop assert.h, use freestanding for amdgcn devicertl
[libomptarget] Drop assert.h, use freestanding for amdgcn devicertl

Promotes the runtime assert to a link time error for the unimplemented
fallback functions. Enables amdgcn to build with only clang provided
headers, which makes it less likely to break other builds when enabled.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D98649
2021-03-15 18:50:09 +00:00
Jon Chesterfield 156842937f [libomptarget][amdgcn] Drop use of inttypes.h, moving closer to freestanding
[libomptarget][amdgcn] Drop use of inttypes.h, moving closer to freestanding

The glibc headers are a periodic source of problems compiling the devicertl.
This patch resolves the following error run into while building llvm on a slightly
different linux system.
```
In file included from .../lib/clang/13.0.0/include/inttypes.h:21:
In file included from /usr/include/inttypes.h:25:
/usr/include/features.h:461:12: fatal error: 'sys/cdefs.h' file not found
#  include <sys/cdefs.h>
           ^~~~~~~~~~~~~
```
As a second patch, removing assert.h from shuffle will let amdgcn build as
-ffreestanding, at which point only the headers that clang itself provides are
used and interactions with the host glibc are eliminated. Doing the same for
nvptx is complicated by printf handling but also seems worthwhile.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D98565
2021-03-15 16:54:58 +00:00
George Rokos 2468fdd9af [libomptarget] Add allocator support for target memory
This patch adds the infrastructure for allocator support for target memory.
Three allocators are introduced for device, host and shared memory.
The corresponding API functions have the llvm_ prefix temporarily, until they become part of the OpenMP standard.

Differential Revision: https://reviews.llvm.org/D97883
2021-03-13 03:47:07 -08:00
Johannes Doerfert 5449fbb5d4 [OpenMP][NFC] Use `AsyncInfo` as the variable name for a `__tgt_async_info`
Reviewed By: grokos, tianshilei1992

Differential Revision: https://reviews.llvm.org/D96444
2021-03-11 23:31:34 -06:00
Johannes Doerfert 66ba494b49 [OpenMP][DeviceRTL] Extract shuffle idiom and port it to declare variant
The shuffle idiom is differently implemented in our supported targets.
To reduce the "target_impl" file we now move the shuffle idiom in it's
own self-contained header that provides the implementation for AMDGPU
and NVPTX. A fallback can be added later on.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D95752
2021-03-11 23:31:30 -06:00
Joseph Huber 807466ef28 [OpenMP] Restore backwards compatibility for libomptarget
Summary:
The changes introduced in D87946 changed the API for libomptarget
functions. `__kmpc_push_target_tripcount` was a function in Clang 11.x
but was not given a backward-compatible interface. This change will
require people using Clang 13.x or 12.x to recompile their offloading
programs.

Reviewed By: jdoerfert cchen

Differential Revision: https://reviews.llvm.org/D98358
2021-03-11 09:52:11 -05:00
Shilei Tian c41ae246ac [OpenMP][Clang][NVPTX] Only build one bitcode library for each SM
In D97003, CUDA 9.2 is the minimum requirement for OpenMP offloading on
NVPTX target. We don't need to have macros in source code to select right functions
based on CUDA version. we don't need to compile multiple bitcode libraries of
different CUDA versions for each SM. We don't need to worry about future
compatibility with newer CUDA version.

`-target-feature +ptx61` is used in this patch, which corresponds to the highest
PTX version that CUDA 9.2 can support.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D97198
2021-03-08 12:03:04 -05:00
Joel E. Denny d0eb25a643 [OpenMP] Encapsulate more in checkDeviceAndCtors
This patch just encapsulates some repeated code.  To do so, it
relocates some functions from interface.cpp to omptarget.cpp.  It also
adjusts them to the LLVM coding style.

This patch is almost NFC except some `DP` messages are a bit
different.  For example, messages like "Entering target region" are
now emitted even if offload is disabled, but a subsequent "Offload is
disabled" is then emitted.

Reviewed By: jdoerfert, grokos

Differential Revision: https://reviews.llvm.org/D97908
2021-03-04 12:03:42 -05:00
Joel E. Denny bfe5452b93 [OpenMP] Fix lone target exit data
Without this patch, an `omp target exit data` before the runtime is
initialized produces a runtime error.  This patch fixes that by
changing `__tgt_target_data_end_mapper` to call `CheckDeviceAndCtors`
like many other runtime routines.

Discussed at
<https://lists.llvm.org/pipermail/openmp-dev/2021-March/003920.html>.

Reviewed By: grokos

Differential Revision: https://reviews.llvm.org/D97907
2021-03-04 12:03:42 -05:00
Joel E. Denny 10c18c69f2 [OpenMP] Fix support for device as host
Without this patch, when the offload device is set to
`omp_get_initial_device()`, the runtime fails with an error diagnostic
when entering target regions or target data regions.

However, OpenMP 5.1, sec. 2.14.5 "target Construct", "Restrictions",
p. 203, L3-5 states:

> The device clause expression must evaluate to a non-negative integer
> value that is less than or equal to the value of
> omp_get_num_devices().

Sec. 3.7.7 "omp_get_initial_device", p. 412, L2-3 states:

> The value of the device number is the value returned by the
> omp_get_num_devices routine.

Similarly, OpenMP 5.0, sec. 2.12.5 "target Construct", "Restrictions",
p. 174 L30-32 states:

> The device clause expression must evaluate to a non-negative integer
> value less than the value of omp_get_num_devices() or to the value
> of omp_get_initial_device().

This patch fixes this behavior by changing the runtime to behave as if
offloading is disabled whenever it finds the offload device (either
from a `device` clause or the default device) is set to the host
device.  In the case of mandatory offloading when
`omp_get_num_devices() == 0`, it incorporates the behavior proposed
for OpenMP 5.2 in OpenMP spec github issue 2669.

Reviewed By: grokos, RaviNarayanaswamy

Differential Revision: https://reviews.llvm.org/D97616
2021-03-04 12:03:42 -05:00
Alexey Bataev 0caf736d7e [OPENMP50]Mapping of the subcomponents with the 'default' mappers.
If the mapped structure has data members, which have 'default' mappers,
need to map these members individually using their 'default' mappers.

Differential Revision: https://reviews.llvm.org/D92195
2021-03-02 07:11:06 -08:00
Vyacheslav Zakharin 6baeeb9efa [libomptarget] Fixed MSVC build fail caused by __attribute__((used)).
Differential Revision: https://reviews.llvm.org/D97348
2021-02-24 09:59:39 -08:00
Shilei Tian e5da63d5a9 [OpenMP] Fixed a crash when offloading to x86_64 with target nowait
PR#49334 reports a crash when offloading to x86_64 with `target nowait`,
which is caused by referencing a nullptr. The root cause of the issue is, when
pushing a hidden helper task in `__kmp_push_task`, it also maps the gtid to its
shadow gtid, which is wrong.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D97329
2021-02-24 12:37:30 -05:00
Manoel Roemmer 542d9c2154 [libomptarget] Load images in order of registration
This makes sure that images are loaded in the order in which they are registered with libomptarget.

If a target can load multiple images and these images depend on each other (for example if one image contains the programs target regions and one image contains library code), then the order in which images are loaded can be important for symbol resolution (for example, in the VE plugin).
In this case: because the same code exist in the host binaries, the order in which the host linker loads them (which is also the order in which images are registered with libomptarget) is the order in which the images have to be loaded onto the device.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D95530
2021-02-24 18:15:41 +01:00
Shilei Tian f6c2984a09 [OpenMP][NVPTX] Fixed a compilation error in deviceRTLs caused by unsupported feature in release verion of LLVM
`ptx71` is not supported in release version of LLVM yet. As a result,
the support of CUDA 11.2 and CUDA 11.1 caused a compilation error as mentioned
in D97004. Since the support in D97004 is just a WA for releease, and we'll not
use it in the near future, using `ptx70` for CUDA 11 is feasible.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D97195
2021-02-23 13:20:21 -05:00
Shilei Tian 309b00a42e [OpenMP][NFC] clang-format the whole openmp project
Same script as D95318. Test files are excluded.

Reviewed By: AndreyChurbanov

Differential Revision: https://reviews.llvm.org/D97088
2021-02-20 12:46:32 -05:00
Joel E. Denny ef8b3b5ffd [OpenMP] Fix nvptx CUDA_VERSION conversion
As mentioned in PR#49250, without this patch, ptxas for CUDA 9.1 fails
in the following two tests:

- openmp/libomptarget/test/mapping/lambda_mapping.cpp
- openmp/libomptarget/test/offloading/bug49021.cpp

The error looks like:

```
ptxas /tmp/lambda_mapping-081ea9.s, line 828; error   : Not a name of any known instruction: 'activemask'
```

The problem is that our cmake script converts CUDA version strings
incorrectly: 9.1 becomes 9100, but it should be 9010, as shown in
`getCudaVersion` in `clang/lib/Driver/ToolChains/Cuda.cpp`.  Thus,
`openmp/libomptarget/deviceRTLs/nvptx/src/target_impl.cu`
inadvertently enables `activemask` because it apparently becomes
available in 9.2.  This patch fixes the conversion.

This patch does not fix the other two tests in PR#49250.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D97012
2021-02-19 11:09:26 -05:00
Joel E. Denny d2147b1a87 [OpenMP] Fix always,from and delete for data absent at exit
Without this patch, there's a runtime error for those map types at
exit from an "omp target data" or at "omp target exit data", but the
spec says the list item should be ignored.

This patch tests that fix in data_absent_at_exit.c, and it also
improves other testing for data that is not fully present at exit.

Reviewed By: grokos, RaviNarayanaswamy

Differential Revision: https://reviews.llvm.org/D96999
2021-02-19 11:09:26 -05:00
Ron Lieberman 30c0d5b4c3 [OPENMP][AMDGCN] Improvements to print_kernel_trace (bit mask)
allow bit masking to select various trace features.
  bit 0 => Launch tracing           (stderr)
  bit 1 => timing of runtime        (stdout)
  bit 2 => detailed launch tracing  (stderr)
  bit 3 => timing goes to stdout instead of stderr

  example: LIBOMPTARGET_KERNEL_TRACE=7     does it all
           LIBOMPTARGET_KERNEL_TRACE=5     Launch + details
           LIBOMPTARGET_KERNEL_TRACE=2     timings + launch to stderr
           LIBOMPTARGET_KERNEL_TRACE=10    timings + launch to stdout

Differential Revision: https://reviews.llvm.org/D96998
2021-02-19 06:47:22 -05:00
Shilei Tian 89827fd404 [OpenMP][NVPTX] Add the support for CUDA 11.2 and CUDA 11.1
CUDA 11.2 and CUDA 11.1 are all available now.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D97004
2021-02-18 21:04:39 -05:00
Jon Chesterfield 53d7fd3762 [libomptarget][amdgcn] Remove lookup of .language msgpack field 2021-02-17 23:02:16 +00:00
Alexey Bataev 60d71a286b [OPENMP50]Allow overlapping mapping in target constructs.
OpenMP 5.0 removed a lot of restriction for overlapped mapped items
comparing to OpenMP 4.5. Patch restricts the checks for overlapped data
mappings only for OpenMP 4.5 and less and reorders mapping of the
arguments so, that present and alloc mappings are processed first and
then all others.

Differential Revision: https://reviews.llvm.org/D86119
2021-02-16 14:42:08 -08:00
Johannes Doerfert 2518cc65d2 [OpenMP][FIX] Avoid use of stack allocations in asynchronous calls
As reported by Guilherme Valarini [0], we used to pass stack allocations
to calls that can nowadays be asynchronous. This is arguably a problem
and it will inevitably result in UB. To remedy the situation we
allocate the locations as part of the AsyncInfoTy object. The lifetime
of that object matches what we need for now. If the synchronization is
not tied to the AsyncInfoTy object anymore we might need to have a
different buffer construct in global space.

This should be back-ported to LLVM 12 but needs slight modifications as
it is based on refactoring patches we do not need to backport.

[0] https://lists.llvm.org/pipermail/openmp-dev/2021-February/003867.html

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D96667
2021-02-16 15:38:11 -06:00
Johannes Doerfert 758b849931 [OpenMP] Unify omptarget API and usage wrt. `__tgt_async_info`
This patch unifies our libomptarget API in two ways:
  - always pass a `__tgt_async_info` object, the Queue member decides if
    it is in use or not.
  - (almost) always synchronize in the interface layer and not in the
    omptarget layer.

A side effect is that we now put all constructor and static initializer
kernels in a stream too, if the device utilizes `__tgt_async_info`.

The patch contains a TODO which can be addressed as we add support for
asynchronous malloc and free in the plugin API. This is the only
`synchronizeAsyncInfo` left in the omptarget layer.

Site note: On a V100 system the GridMini performance for small sizes
more than doubled.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D96379
2021-02-16 15:38:06 -06:00
Johannes Doerfert a2fc0d34db [OpenMP] Move synchronization into `__tgt_async_info`
The AsyncInfo should be passed everywhere and it should offer a way to
ensure synchronization, given a libomptarget Device.

This replaces D96431.

Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D96438
2021-02-16 15:38:01 -06:00
Johannes Doerfert 942728763b [OpenMP][NFC] Unify `target` API with other by passing a `__tgt_async_info` pointer
Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D96430
2021-02-16 15:37:56 -06:00
Johannes Doerfert 44f3022cdf [OpenMP][NFC] Pass a DeviceTy, not the device number to `target`
This unifies the API of `target` relative to `targetUpdateData` and
such.

Reviewed By: tianshilei1992, grokos

Differential Revision: https://reviews.llvm.org/D96429
2021-02-16 15:37:51 -06:00
Johannes Doerfert ea9395716e [OpenMP][NFC] Clang format the libomptarget plugins
Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D96445
2021-02-16 15:37:46 -06:00
Johannes Doerfert ad94fce845 [OpenMP][NFC] Eliminate sign comparison warning via explicit casts
Reviewed By: tianshilei1992

Differential Revision: https://reviews.llvm.org/D96812
2021-02-16 15:37:41 -06:00
Johannes Doerfert 9cd1e2228c [OpenMP][NFC] Clang format libomptarget code (src & include)
The struct and enum alignments are kept by disabling clang-format for
that code region.

Reviewed By: tianshilei1992, JonChesterfield, grokos

Differential Revision: https://reviews.llvm.org/D96428
2021-02-16 15:37:35 -06:00
Jon Chesterfield 6f04addc8b [libomptarget][amdgcn] Build amdgcn devicertl as openmp
[libomptarget][amdgcn] Build amdgcn devicertl as openmp

Change cmake to build as openmp and fix up some minor errors in the code.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D96533
2021-02-12 09:51:21 +00:00
Jon Chesterfield 56c446a878 [libomptarget][amdgcn] Tolerate deadstripped device_state variable
[libomptarget][amdgcn] Tolerate deadstripped device_state variable

The device_state variable may have been deadstripped. Similar to
device_environment, leave detection of missing but used symbol to loader.

Reviewed By: pdhaliwal

Differential Revision: https://reviews.llvm.org/D96330
2021-02-09 16:29:53 +00:00
Jon Chesterfield 4756f76bce [libomptarget][amdgcn] Tolerate deadstripped env variable
[libomptarget][amdgcn] Tolerate deadstripped env variable

Discovered by Pushpinder. If the device_environment variable is unused
it can be deadstripped, in which case we should not abort due to it
missing. This change is safe in that a missing symbol which is actually
used can be reported by both linker and loader, and a missing unused
symbol is better deadstripped than left in the image.

Reviewed By: pdhaliwal

Differential Revision: https://reviews.llvm.org/D96329
2021-02-09 11:58:37 +00:00
Jon Chesterfield 2fa4186d4e [libomptarget][amdgcn] Fix language linkage post D95300, drop use of assert 2021-02-08 20:07:51 +00:00
Shilei Tian b68a6b09e6 [OpenMP][libomptarget] Fixed an issue that device sync is skipped if the kernel doesn't have any argument
Currently if there is not kernel argument, device synchronization will
be skipped. This can lead to two issues:
1. If there is any device error, it will not be captured;
2. The target region might end before the kernel is done, which is not spec
   conformant.

The test added in this patch only runs on NVPTX platform, although it will not
be executed by Phab at all. It also requires `not` which is not available on most
systems.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D96067
2021-02-04 20:14:24 -05:00
Shilei Tian 567b3f8841 [OpenMP][deviceRTLs] Drop `assert` in common parts of `deviceRTLs`
The header `assert.h` needs to be included in order to use `assert` in the code.
When building NVPTX `deviceRTLs` on a CUDA free system, it requires headers from
`gcc-multilib`, which some systems don't have. This patch drops the use of
`assert` in common parts of `deviceRTLs`. In light of
`openmp/libomptarget/deviceRTLs/amdgcn/src/target_impl.h`, a code block
```
if (!cond)
  __builtin_trap();
```
is being used. The builtin will be translated to `call void @llvm.trap()`, and
the corresponding PTX is `trap;`.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D95986
2021-02-04 12:39:43 -05:00
Shilei Tian 0f0ce3c12e [OpenMP][NVPTX] Take functions in `deviceRTLs` as `convergent`
OpenMP device compiler (similar to other SPMD compilers) assumes that
functions are convergent by default to avoid invalid transformations, such as
the bug (https://bugs.llvm.org/show_bug.cgi?id=49021).

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D95971
2021-02-03 20:58:12 -05:00
Atmn Patel b545667d0a [OpenMP][Libomptarget] Remove possible harmful copy constructor call for RTLsTy
From https://bugs.llvm.org/show_bug.cgi?id=48973, we know that
`std::call_once(PM->RTLs.initFlag, &RTLsTy::LoadRTLs, PM->RTLs)` causes compile
time problems in libstdc++v3 5.3.1. This is because there was a defect in the
standard regarding the `call_once` (LWG 2442). This was fixed in libstdc++ soon
thereafter, but there are likely other standard libraries where this will fail.

By matching this function call with the other one, we fix this bug.

Differential Revision: https://reviews.llvm.org/D95769
2021-02-01 20:13:03 -05:00
Joseph Huber fda4853998 [OpenMP] Fix seg fault in libomptarget when using Info with multiple threads
Summary:
One option for the LIBOMPTARGET_INFO environment variable is to print the current status of the device's data mappings. These are a shared resource among threads so this needs to be protected when using multiple streams.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D95786
2021-02-01 11:21:57 -05:00
Shilei Tian 26d38f6d20 [OpenMP][NVPTX] Refined CMake logic to choose compute capabilites
This patch refines the logic to choose compute capabilites via the
environment variable `LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES`. It supports the
following values (all case insensitive):
- "all": Build `deviceRTLs` for all supported compute capabilites;
- "auto": Only build for the compute capability auto detected. Note that this
  requires CUDA. If CUDA is not found, a CMake fatal error will be raised.
- "xx,yy" or "xx;yy": Build for compute capabilities `xx` and `yy`.

If `LIBOMPTARGET_NVPTX_COMPUTE_CAPABILITIES` is not set, it is equivalent to set
it to `all`.

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D95687
2021-01-30 15:14:48 -05:00
Shilei Tian 1b19c42302 [OpenMP][deviceRTLs] Separate declaration of target dependent functions from `target_impl.h`
This patch created a new header file `target_interface.h` for declarations of all target dependent functions. All future targets can get things work by simply implementing all functions declared in the header and macros/data same as each `target_impl.h`.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D95300
2021-01-28 08:14:33 -05:00
Shilei Tian 5a64794bba [OpenMP][NVPTX] Added the missing -O1 when building NVPTX bitcode libraries
In the past `-O1` was used when building NVPTX bitcode libraries. After
we switched to OpenMP, `-O1` was missing by mistake, leading to a huge performance
regression.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D95545
2021-01-28 08:13:38 -05:00