Summary:
There is currently no supported situation where the warp master is not the first thread in the warp.
This also avoids the device execution from hanging on Volta GPUs when ballot_sync is called by a number of threads that is less that the size of a warp.
Reviewers: ABataev, caomhin, grokos
Reviewed By: grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D50188
llvm-svn: 342972
Summary:
We need the support for per-team shared variables to support codegen for
lastprivates/reductions. Patch adds this support by using shared memory
if the total size of the reductions/lastprivates is <= 128 bytes,
then pre-allocated buffer in global memory if size is <= 4K bytes,or
uses malloc/free, otherwise.
Reviewers: gtbercea, kkwli0, grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D51875
llvm-svn: 342737
Summary:
Missed operation of the incrementing iterator when required just to
continue execution.
Reviewers: kkwli0, gtbercea, grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D51937
llvm-svn: 341964
This is a follow-up to r341371: The new test for PR38704 doesn't
work with Clang 6.0. It uses an UNSUPPORTED: clang-6, but that
hasn't worked because the compiler features weren't known to lit.
llvm-svn: 341448
cuDeviceGetProperties has apparently been deprecated since CUDA 5.0.
Nvidia started using annotations only in CUDA 9.2, so nobody noticed
nor cared before.
The new function returns the same values, tested with a P100.
Differential Revision: https://reviews.llvm.org/D51624
llvm-svn: 341372
* cg and HasCancel in WorkDescr were never read and can be removed.
* This eliminates the last use of priv in ThreadPrivateContext.
* CounterGroup is unused afterwards.
* Remove duplicate external declares in omptarget-nvptx.cu that are
already in the header omptarget-nvptx.h.
Differential Revision: https://reviews.llvm.org/D51622
llvm-svn: 341370
If the runtime is uninitialized the master thread must Enqueue the
state object, and ALL threads must return immediately.
Found post-commit of https://reviews.llvm.org/D51222.
llvm-svn: 341328
Summary:
Implemented simple and lightweight runtime support for SPMD mode-based
constructs. It adds support for L2 sequential parallelism wihtout full
runtime support. Also, patch fixes some use cases for
uninitialized|lightweight runtime.
Reviewers: grokos, kkwli0, Hahnfeld, gtbercea
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D51222
llvm-svn: 340944
Summary:
Removed the function that used a lock and varargs
Used the same mechanism as for debug messages
Reviewers: ABataev, gtbercea, grokos, Hahnfeld
Reviewed By: gtbercea, Hahnfeld
Subscribers: mikerice, ABataev, RaviNarayanaswamy, guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D51226
llvm-svn: 340767
Summary:
Right now, only the OMP_TARGET_OFFLOAD=DISABLED was implemented. Added support for the other MANDATORY and DEFAULT values.
Reviewers: gtbercea, ABataev, grokos, caomhin, Hahnfeld
Reviewed By: Hahnfeld
Subscribers: protze.joachim, gtbercea, AlexEichenberger, RaviNarayanaswamy, Hahnfeld, guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D50522
llvm-svn: 340542
Summary:
1. Fixed internal problem in `__kmpc_barrier` function: SPMD mode
synchronization function should be called only in L1 parallel level.
2. Removed some extra code for synchronization inside of the code, used
`__kmpc_barrier` instead.
3. Some code cleanup.
Reviewers: gtbercea, grokos
Subscribers: openmp-commits
Differential Revision: https://reviews.llvm.org/D49564
llvm-svn: 337691
This patch removes the translation code since this functionality is now implemented in the compiler.
target_data_begin and target_data_end are also patched to handle some special cases that used to be
handled by the obsolete translation function, namely ensure proper alignment of struct members when
we have partially mapped structs. Mapping a struct from a higher address (i.e. not from its beginning)
can result in distortion of the alignment for some of its member fields. Padding restores the original
(proper) alignment.
Differential revision: https://reviews.llvm.org/D44186
llvm-svn: 337455
In revision r336569 (D49036) libomptarget support for multiple nvidia images
has been fixed in case a target region resides inside one or multiple
libraries and in the compiled application. But the issues is still present
for elf images.
This fix will also support multiple images for elf.
Patch by Jannis Klinkenberg
Reviewers: protze.joachim, ABataev, grokos
Reviewed By: protze.joachim, ABataev, grokos
Subscribers: openmp-commits
Differential Revision: https://reviews.llvm.org/D49418
llvm-svn: 337355
Summary:
Should be variable name instead of variable reference. If the variable is
somehow unset, it messes up the if condition expression and causes a CMake
error.
Reviewers: jlpeyton, AndreyChurbanov, Hahnfeld
Reviewed By: Hahnfeld
Subscribers: mgorny, llvm-commits, openmp-commits
Differential Revision: https://reviews.llvm.org/D47221
llvm-svn: 337133
Summary: This patch fixes the data sharing infrastructure to work for the SPMD and non-SPMD cases.
Reviewers: ABataev, grokos, carlo.bertolli, caomhin
Reviewed By: ABataev, grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D49204
llvm-svn: 337013
Summary:
Patch fixes the next problems.
1. Removes unused functions from omptarget_nvptx_ThreadPrivateContext
class + simplified data members.
2. Fixed calculation of loop boundaries for dynamic loops with static
scheduling.
3. Introduced saving/restoring of the dynamic loop boundaries to support
several nested parallel dynamic loops.
Reviewers: grokos
Subscribers: guansong, kkwli0, openmp-commits
Differential Revision: https://reviews.llvm.org/D49241
llvm-svn: 336915
Summary:
Currently Cuda plugin supports loading of the single image, though we
may have the executable with the several images, if it has target
regions inside of the dynamically loaded library. Patch allows to load
multiple images.
Reviewers: grokos
Subscribers: guansong, openmp-commits, kkwli0
Differential Revision: https://reviews.llvm.org/D49036
llvm-svn: 336569
Summary:
Patch fixes several problems in the implementation of NVPTX RTL.
1. Detection of the last iteration for loops with static scheduling, no chunks.
2. Fixes reductions for the serialized parallel constructs.
3. Fixes handling of the barriers.
Reviewers: grokos
Reviewed By: grokos
Subscribers: Hahnfeld, guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D48480
llvm-svn: 335469
The generic entry points for static loop scheduling previously
hardcoded that the runtime was initialized. This can be wrong if
the compiler analyzes that the runtime is not needed and calls
the init functions accordingly.
This didn't affect clang-ykt because they have entry points for
different combinations of SPMD x Runtime not needed. I didn't do
measurements yet but with inlining we might get away with always
calling the generic interface and letting compiler and runtime
figure out the rest.
In any case, a correct runtime is always better than having
functions that may only be called if previous calls passed in
a specific set of arguments!
Differential Revision: https://reviews.llvm.org/D47131
llvm-svn: 333285
Introduce OPENMP_INSTALL_LIBDIR and use in all install() commands.
This also fixes installation of libomptarget-nvptx that previously
didn't honor {OPENMP,LLVM}_LIBDIR_SUFFIX.
Differential Revision: https://reviews.llvm.org/D47130
llvm-svn: 333284
The existing implementation of the dynamic scheduling
breaks the contract introduced by the original openmp
runtime and, thus, is incorrect. Patch fixes it and
introduces correct dynamic scheduling model.
Thanks to Alexey Bataev for submitting this patch.
Differential Revision: https://reviews.llvm.org/D47333
llvm-svn: 333225
We already know where the CUDA SDK is, so there is no point in
letting Clang search for it again and possibly finding no or
a different installation.
--cuda-path is supported since the beginning of CUDA support in
Clang, so making this required doesn't impose additional restrictions.
Differential Revision: https://reviews.llvm.org/D46930
llvm-svn: 332495
Move all logic related to selecting the bitcode compiler and linker
into a new file and dynamically test required compiler flags. This
also adds -fcuda-rdc for Clang trunk as previously attempted in D44992
which fixes the build.
As a result this change also enables building the library by default
if all prerequisites are met.
Differential Revision: https://reviews.llvm.org/D46901
llvm-svn: 332494
Summary: Add function to the NVPTX libomptarget library that will return true if the current target region is being executed in SPMD mode.
Reviewers: ABataev, grokos, carlo.bertolli, caomhin
Reviewed By: grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D46840
llvm-svn: 332360
Summary:
Enable the device side debug messages at compile time, use env var to control at runtime.
To achieve this, an environment data block is passed to the device lib when it is loaded.
By default, the message is off, to enable it, a user need to set LIBOMPDEVICE_DEBUG=1.
Reviewers: grokos
Reviewed By: grokos
Subscribers: openmp-commits
Tags: #openmp
Differential Revision: https://reviews.llvm.org/D46210
llvm-svn: 331550
Summary: Minor printf format correction. NVCC ignore those. Clang will give warning on these if debug is enabled.
Reviewers: grokos
Reviewed By: grokos
Subscribers: openmp-commits
Tags: #openmp
Differential Revision: https://reviews.llvm.org/D45528
llvm-svn: 330944
Summary: The LIBOMPTARGET_NVPTX_DEBUG flag is inconsistent between using nvcc to generate .a file and clang to generate .bc file. Sync the two setting so we can get debug messages from the bc file path as well.
Reviewers: grokos
Subscribers: Hahnfeld, openmp-commits, mgorny
Tags: #openmp
Differential Revision: https://reviews.llvm.org/D45530
llvm-svn: 330477
Summary:
This one line change is to remove this warning message
"warning: integer conversion resulted in a change of sign"
Reviewers: grokos
Reviewed By: grokos
Subscribers: openmp-commits
Tags: #openmp
Differential Revision: https://reviews.llvm.org/D45415
llvm-svn: 329713
Summary: The global stack initialization function may be called multiple times. The initialization of the shared memory slots should only happen when the function is called for the first time for a given warp master thread.
Reviewers: grokos, carlo.bertolli, ABataev, caomhin
Reviewed By: grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D44754
llvm-svn: 328148
Summary: The check for the master warp must take into consideration the actual number of warps: the master warp is equal to the last active warp not necessarily WARPSIZE - 1.
Reviewers: grokos, carlo.bertolli, ABataev, caomhin
Reviewed By: grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D44537
llvm-svn: 328146
Summary:
This patch allows worker to have a global memory stack managed by the runtime. This patch is needed for completeness and consistency with the globalization policy: if a worker-side variable escapes the current context it then needs to be globalized.
Until now, only the master thread was allowed to have such a stack. These global values can now potentially be shared amongst workers if the semantics of the OpenMP program require it.
Reviewers: ABataev, grokos, carlo.bertolli, caomhin
Reviewed By: grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D44487
llvm-svn: 328144
Summary:
Allow the runtime to use the existing shared memory statically allocated slots.
When a variable is globalized, the underlying memory can be either shared or global memory (both have block-wide visibility). In this case, we allow that the storage to use a limited amount of shared memory that has been statically allocated already. Only if shared memory doesn't prove to be enough do we then invoke malloc() to create a new global memory slot.
Reviewers: ABataev, carlo.bertolli, grokos, caomhin
Reviewed By: grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D44486
llvm-svn: 327639
Summary: To save on calls to malloc, this patch enables the re-use of pre-allocated global memory slots.
Reviewers: ABataev, grokos, carlo.bertolli, caomhin
Reviewed By: grokos
Subscribers: guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D44470
llvm-svn: 327637
Summary:
This patch adds support for the sharing of variables from the master thread of a team to the worker threads of the team.
The runtime uses a stack structure implemented as a doubly-linked list of slots with each slot having the exact same size as the size requested. This implementation leverages existing data structures. The runtime functions are added as separate functions to avoid interfering with the current interface.
Limitations to be addressed in future patches:
- This current patch only employs global memory. In a future patch we will enable to usage for shared memory as an optimization.
- Allow the allocation of several requested sizes in the same slot.
Reviewers: ABataev, grokos, caomhin, carlo.bertolli
Reviewed By: grokos
Subscribers: Hahnfeld, guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D44260
llvm-svn: 327440
Summary: To make the two parts of the union have the same size, the size of vect needs to be increased by 16 bits.
Reviewers: grokos, carlo.bertolli, caomhin, ABataev
Reviewed By: grokos, ABataev
Subscribers: fedor.sergeev, guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D44254
llvm-svn: 327040
Summary:
This patch reverts the changes to libomptarget that were coupled with the changes to Clang code gen for data sharing using shared memory. A similar patch exists for Clang: D43625
Shared memory is meant to be used as an optimization on top of a more general scheme. So far we didn't have a global memory implementation ready so shared memory was a solution which applied to the current level of OpenMP complexity supported by trunk on GPU devices (due to the missing NVPTX backend patch this functionality has never been exercised). Now that we have a global memory solution this patch is "in the way" and needs to be removed (for now). This patch (or an equivalent version of it) will be put out for review once the global memory scheme is in place.
Reviewers: ABataev, grokos, carlo.bertolli, caomhin
Reviewed By: grokos
Subscribers: Hahnfeld, guansong, openmp-commits
Differential Revision: https://reviews.llvm.org/D43626
llvm-svn: 326950
Summary:
Different NVIDIA GPUs support different compute capabilities. To enable the inlining of runtime functions and the best performance on different generations of NVIDIA GPUs, a bc library for each compute capability needs to be compiled. The same compiler build will then be usable in conjunction with multiple generations of NVIDIA GPUs.
To differentiate between versions of the same bc lib, the output file name will contain the compute capability ID.
Depends on D14254
Reviewers: Hahnfeld, hfinkel, carlo.bertolli, caomhin, ABataev, grokos
Reviewed By: Hahnfeld, grokos
Subscribers: guansong, mgorny, openmp-commits
Differential Revision: https://reviews.llvm.org/D41724
llvm-svn: 324904
Summary: This patch extends the libomptarget functionality in patch D14254 with support for the data sharing scheme for supporting implicitly shared variables. The runtime therefore maintains a list of references to shared variables.
Reviewers: carlo.bertolli, ABataev, Hahnfeld, grokos, caomhin, hfinkel
Reviewed By: Hahnfeld, grokos
Subscribers: guansong, llvm-commits, openmp-commits
Differential Revision: https://reviews.llvm.org/D41485
llvm-svn: 324495
https://reviews.llvm.org/D42757
The method ThreadsInTeam is used to determine the number of threads to be used in a parallel region under SPMD mode (see line 127 of supporti.h in libomptarget/deviceRTLs/nvptx/src/). This patch fixes the corresponding debug print upon initialization of the kernel in SPMD mode.
llvm-svn: 323978
That's what we really need to link the CUDA plugin against,
not the CUDA runtime API in CUDA_LIBRARIES! While the latter
comes with the CUDA SDK, the Driver API is installed with
the kernel driver and there is at most one per system. As
fallback we can use the stubs library distributed with the
CUDA SDK for linking.
Differential Revision: https://reviews.llvm.org/D42643
llvm-svn: 323787
Use equivalents for the last calls to the Runtime API. Remove
stray assert in case of an error found during review, we should
only return OFFLOAD_FAIL.
Differential Revision: https://reviews.llvm.org/D42686
llvm-svn: 323786
This patch implements the device runtime library whose interface is used in the code generation for OpenMP offloading devices.
Currently there is a single device RTL written in CUDA meant to CUDA enabled GPUs.
The interface is a variation of the kmpc interface that includes some extra calls to do thread and storage management that only make sense for a GPU target.
Differential revision: https://reviews.llvm.org/D14254
llvm-svn: 323649
We now have several options that apply for both libraries and they
shouldn't be documented in multiple files. When already merging
the two Build_With_CMake.txt documents, convert them to
reStructuredText which is used for all of LLVM's documentation.
Differential Revision: https://reviews.llvm.org/D40920
llvm-svn: 321481
This last of four patches adds a new file for the interface
functions that Clang uses during code generation. The only
change except simply moving the current code is renaming the
function CheckDeviceAndCtors() and using the correct type for
64bit device ids.
Differential Revision: https://reviews.llvm.org/D40801
llvm-svn: 319972
This third patch moves the implementation of the user-facing
OpenMP API functions into its own file. For now, the code is
only moved, no cleanups applied yet.
Differential Revision: https://reviews.llvm.org/D40800
llvm-svn: 319971
This is the second patch to split the current monolithic
implementation into separate files. Note that this change
doesn't cleanup the code yet.
Differential Revision: https://reviews.llvm.org/D40799
llvm-svn: 319970
This is the first of four patches to split the target agnostic
library into multiple (smaller) files. It only moves the code
to separate implementation files and does no cleanup (yet) except
removing unneeded headers.
Differential Revision: https://reviews.llvm.org/D40798
llvm-svn: 319969
Future patches will add (private) header files in src/ that should
not be visible to plugins, so move the "public" ones to a new
include/ directory. This is still internal in a sense that the
contained files won't be installed for the user.
Similarly, the target agnostic offloading library should be built
directly in src/. The parent directory is responsible for finding
dependencies and including all subdirectories.
Differential Revision: https://reviews.llvm.org/D40797
llvm-svn: 319968
Perform a nested CMake invocation to avoid writing our own parser
for compiler versions when we are not testing the in-tree compiler.
Use the extracted information to mark a test as unsupported that
hangs with Clang prior to version 4.0.1 and restrict tests for
libomptarget to Clang version 6.0.0 and later.
Differential Revision: https://reviews.llvm.org/D40083
llvm-svn: 319448
The code for the two OpenMP runtime libraries was very similar.
Move to common CMake file that is included and provides a simple
interface for adding testsuites. Also add a common check-openmp
target that runs all testsuites that have been registered.
Note that this renames all test options to the common OPENMP
namespace, for example OPENMP_TEST_C_COMPILER instead of
LIBOMP_TEST_COMPILER and so on.
Differential Revision: https://reviews.llvm.org/D40082
llvm-svn: 319343
These are needed by both libraries, so we can do that in a
common namespace and unify configuration parameters.
Also make sure that the user isn't requesting libomptarget
if the library cannot be built on the system. Issue an error
in that case.
Differential Revision: https://reviews.llvm.org/D40081
llvm-svn: 319342
As a first step, this allows us to generalize the detection of
standalone builds and make it fully compatible when building in
llvm/runtimes/ which automatically sets OPENMP_STANDLONE_BUILD.
Differential Revision: https://reviews.llvm.org/D40080
llvm-svn: 319341
We have just fixed the codegen of omp_is_initial_device() to reliably work
when offloading to the same device, see commit r316001. This fixes the
failing tests that were the reason why we disabled the library for 5.0.
Differential Revision: https://reviews.llvm.org/D39052
llvm-svn: 318847
This is the libomptarget-side patch which changes the __tgt_* API function signatures in preparation for the new map interface.
Changes are: Device IDs 32bits --> 64bits, Flags 32bits --> 64bits
Differential revision: https://reviews.llvm.org/D40313
llvm-svn: 318790
In standalone build, plugins where previously built in their
subdirectory in plugins/ and tests couldn't find them.
Differential Revision: https://reviews.llvm.org/D39920
llvm-svn: 317979
This is part of the renaming of data types from OpenMP TR4 to TR6
Patch by Simon Convent
Differential Revision: https://reviews.llvm.org/D39326
llvm-svn: 317338
This was never used in the upstream compiler and was responsible
for some problems with reductions in the clang-ykt fork.
Differential Revision: https://reviews.llvm.org/D39553
llvm-svn: 317214
Disable default debugging dumps for libomptarget and plugins and move dumps
under control of environment variable LIBOMPTARGET_DEBUG=<integer>. Dumps
are enabled when LIBOMPTARGET_DEBUG is set to a positive integer value.
Debugging dumps are available only in debug build; release build does not
support it.
Differential Revision: https://reviews.llvm.org/D33227
llvm-svn: 310841
Summary: On BSDs, there is no `libdl.so`, and functions like `dlopen`
are implemented in the main C library instead. Use the `CMAKE_DL_LIBS`
variable instead of hardcoding a dependency on the `dl` library.
Reviewers: grokos, joerg, emaste
Reviewed By: emaste
Subscribers: jlpeyton, mgorny, openmp-commits
Differential Revision: https://reviews.llvm.org/D34632
llvm-svn: 306319
This patch chagnes the plugin interface so that:
1) future plugins can take advantage of systems with shared CPU/device storage
2) instead of using base addresses, target regions are launched by providing target addresseds and base offsets explicitly.
Differential revision: https://reviews.llvm.org/D33028
llvm-svn: 302663
This adds AArch64 support to recently added part of the runtime responsible for offloading to target. This piece of code allows offloading-to-self on AArch64 machines.
Differential Revision: https://reviews.llvm.org/D30644
llvm-svn: 297070
Fixed bug due to which a parent struct was deallocated when one of the struct's pointers was being unmapped.
Differential Revision: https://reviews.llvm.org/D29914
llvm-svn: 295231
This change allows setting LIBOMPTARGET_LLVM_LIT_EXECUTABLE and
LIBOMPTARGET_FILECHECK_EXECUTABLE as full path. It also honors
OPENMP_LLVM_TOOLS_DIR which is meant as a common configuration
for both libomp and libomptarget.
Maybe this should be done in a common CMake module, but I'm no expert here.
Differential Revision: https://reviews.llvm.org/D29172
llvm-svn: 294284
This is the patch upstreaming the plugins part of libomptarget (CUDA, generic-elf-64).
Differential Revision: https://reviews.llvm.org/D14253
llvm-svn: 293724
glibc < 2.18 is C99 compliant and only provides the format macros in C++ if
__STDC_FORMAT_MACROS is defined. This change fixes the debug build for
GCC 4.8, GCC 6.2 and Clang 3.9.1 that were previously broken on my machine.
It shows no regression for libc++ >= 4.0.0 which has a fix since September:
http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20160926/171659.html
llvm-svn: 293468
Gheorghe-Teodor Bercea