llvm-project/openmp/libomptarget/plugins
Ye Luo c5348aecd7 [OpenMP] Use primary context in CUDA plugin
Summary:
Retaining per device primary context is preferred to creating a context owned by the plugin.

From CUDA documentation
1. Note that the use of multiple CUcontext s per device within a single process will substantially degrade performance and is strongly discouraged. Instead, it is highly recommended that the implicit one-to-one device-to-context mapping for the process provided by the CUDA Runtime API be used." from https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DRIVER.html
2. Right under cuCtxCreate. In most cases it is recommended to use cuDevicePrimaryCtxRetain. https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__CTX.html#group__CUDA__CTX_1g65dc0012348bc84810e2103a40d8e2cf
3. The primary context is unique per device and shared with the CUDA runtime API. These functions allow integration with other libraries using CUDA.  https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__PRIMARY__CTX.html#group__CUDA__PRIMARY__CTX

Two issues are addressed by this patch:
1. Not using the primary context caused interoperability issue with libraries like cublas, cusolver. CUBLAS_STATUS_EXECUTION_FAILED and cudaErrorInvalidResourceHandle
2. On OLCF summit, "Error returned from cuCtxCreate" and "CUDA error is: invalid device ordinal"

Regarding the flags of the primary context. If it is inactive, we set CU_CTX_SCHED_BLOCKING_SYNC. If it is already active, we respect the current flags.

Reviewers: grokos, ABataev, jdoerfert, protze.joachim, AndreyChurbanov, Hahnfeld

Reviewed By: jdoerfert

Subscribers: openmp-commits, yaxunl, guansong, sstefan1, tianshilei1992

Tags: #openmp

Differential Revision: https://reviews.llvm.org/D82718
2020-07-07 10:14:51 -04:00
..
aarch64 Update more file headers across all of the LLVM projects in the monorepo 2019-01-19 10:56:40 +00:00
common [Openmp][VE] Libomptarget plugin for NEC SX-Aurora 2020-05-12 10:47:30 +02:00
cuda [OpenMP] Use primary context in CUDA plugin 2020-07-07 10:14:51 -04:00
generic-elf-64bit/src [OpenMP] Put old APIs back and added new _async series for backward compatibility 2020-04-09 22:40:58 -04:00
ppc64 Update more file headers across all of the LLVM projects in the monorepo 2019-01-19 10:56:40 +00:00
ppc64le Update more file headers across all of the LLVM projects in the monorepo 2019-01-19 10:56:40 +00:00
ve [Openmp][VE] Libomptarget plugin for NEC SX-Aurora 2020-05-12 10:47:30 +02:00
x86_64 Update more file headers across all of the LLVM projects in the monorepo 2019-01-19 10:56:40 +00:00
CMakeLists.txt [Openmp][VE] Libomptarget plugin for NEC SX-Aurora 2020-05-12 10:47:30 +02:00
exports [OpenMP] Improve D2D memcpy to use more efficient driver API 2020-06-04 16:59:06 -04:00