Commit Graph

1823 Commits

Author SHA1 Message Date
Joachim Meyer 5d689cf2a6 [NFC][CUDA] Fix order of round(f) definition in __clang_cuda_math.h for non-LP64.
This broke ARM builds e.g.: https://lab.llvm.org/buildbot/#/builders/187/builds/212
2021-07-02 21:55:48 +02:00
Brian Cain 28b01c59c9 [hexagon] Add {hvx,}hexagon_{protos,circ_brev...}
Add definitions for Hexagon, Hexagon circular/bit-reverse and HVX
intrinsics.
2021-06-30 22:58:56 -05:00
Xiang1 Zhang 6d234a6908 [X86] Zero some outputs of Kelocker intrinsics in error case
Reviewed By: WangPengfei

Differential Revision: https://reviews.llvm.org/D104766
2021-06-29 13:35:40 +08:00
Nemanja Ivanovic ef906573a1 [PowerPC] Fix vec_add for 64-bit on pre-Power7 subtargets
The shift of the carry was actually incorrect.
2021-06-24 18:42:44 -05:00
Ethan Stewart 5dfdc1812d [OpenMP][AMDGCN] Apply fix for isnan, isinf and isfinite for amdgcn.
This fixes issues with various return types(bool/int) and was already
in place for nvptx headers, adjusted to work for amdgcn. This does
not affect hip as the change is guarded with OPENMP_AMDGCN.
Similar to D85879.

Reviewed By: jdoerfert, JonChesterfield, yaxunl

Differential Revision: https://reviews.llvm.org/D104677
2021-06-23 15:26:09 +01:00
Yaxun (Sam) Liu 186f2ac612 [HIP] Add support functions for C++ polymorphic types
Add runtime functions to detect invalid calls to pure or deleted virtual
functions.

Patch by: Siu Chi Chan

Reviewed by: Yaxun Liu

Differential Revision: https://reviews.llvm.org/D104392
2021-06-21 11:41:07 -04:00
Bing1 Yu 56d5c46b49 [X86] Support __tile_stream_loadd intrinsic for new AMX interface
Adding support for __tile_stream_loadd intrinsic.

Reviewed By: LuoYuanke

Differential Revision: https://reviews.llvm.org/D103784
2021-06-11 17:28:43 +08:00
Sven van Haastregt d54e7b731e [OpenCL] Add memory_scope_all_devices
Add the `memory_scope_all_devices` enum value, which is restricted to
OpenCL 3.0 or newer and the `__opencl_c_atomic_scope_all_devices`
feature.  Also guard `memory_scope_all_svm_devices` accordingly, which
is already available in OpenCL 2.0.

The `__opencl_c_atomic_scope_all_devices` feature is header-only, so
set its define to 1 in `opencl-c-base.h`.  This is done
unconditionally at the moment, as the mechanism for disabling
header-only options hasn't been decided yet.

This patch only adds a negative test for now.  Ideally adding a CL3.0
run line to atomic-ops.cl should suffice as a positive test, but we
cannot do that yet until (at least) generic address spaces and program
scope variables are supported in OpenCL 3.0 mode.

Differential Revision: https://reviews.llvm.org/D103241
2021-06-08 11:51:12 +01:00
Stuart Brady 9b14670f3c [OpenCL] Add const attribute to ctz() builtins
Reviewed By: svenvh

Differential Revision: https://reviews.llvm.org/D97725
2021-06-07 11:41:52 +01:00
Stuart Brady 86c24493ea [OpenCL][NFC] Test commit: tidy up whitespace in comment 2021-06-04 14:44:12 +01:00
Qiu Chaofan c0b3071833 [PowerPC] Fix x86 vector intrinsics wrapper compilation under C++
Reviewed By: nemanjai

Differential Revision: https://reviews.llvm.org/D103386
2021-06-01 01:19:12 +08:00
Artem Belevich 9a75c06cd9 [CUDA] Work around compatibility issue with libstdc++ 11.1.0
libstdc++ redeclares __failed_assertion multiple times and that results in the
function declared with conflicting set of attributes when we include <complex>
with __host__ __device__ attributes force-applied to all functions.

In order to work around the issue, we rename __failed_assertion within the
region with forced attributes.

See https://bugs.llvm.org/show_bug.cgi?id=50383 for the details.

Differential Revision: https://reviews.llvm.org/D102936
2021-05-24 11:07:09 -07:00
Yaxun (Sam) Liu 91dfd68e90 [NFC][HIP] fix comments in __clang_hip_cmath.h 2021-05-21 17:44:18 -04:00
Nemanja Ivanovic 7cd2833311 [PowerPC] Add vec_vupkhpx and vec_vupklpx for XL compatibility
These are old names for these functions that XL still supports.
2021-05-14 08:02:00 -05:00
Aaron En Ye Shi a249ffa421 [HIP] Clean up llvm intrinsics using __asm
Instead of using inline asm, use clang builtins
for llvm intrinsics.

Differential Revision: https://reviews.llvm.org/D102427
2021-05-13 18:55:51 +00:00
Nemanja Ivanovic 39e4676ca7 [PowerPC] Provide doubleword vector predicate form comparisons on Power7
There are two reasons this shouldn't be restricted to Power8 and up:
1. For XL compatibility
2. Because clang will expand comparison operators to these intrinsics*

*Without this patch, the following causes a selection error:

int test(vector signed long a, vector signed long b) {
  return a < b;
}

This patch provides the handling for the intrinsics in the back
end and removes the Power8 guards from the predicate functions
(vec_{all|any}_{eq|ne|gt|ge|lt|le}).
2021-05-13 04:56:56 -05:00
Anastasia Stulova 58d18dde5c [OpenCL] Remove pragma requirement from Arm dot extension.
This removed the pointless need for extension pragma since
it doesn't disable anything properly and it doesn't need to
enable anything that is not possible to disable.

The change doesn't break existing kernels since it allows to
compile more cases i.e. without pragma statements but the
pragma continues to be accepted.

Differential Revision: https://reviews.llvm.org/D100985
2021-05-12 16:25:33 +01:00
Thomas Lively 1e9c39a3f9 [WebAssembly] Use functions instead of macros for const SIMD intrinsics
To improve hygiene, consistency, and usability, it would be good to replace all
the macro intrinsics in wasm_simd128.h with functions. The reason for using
macros in the first place was to enforce the use of constants for some arguments
using `_Static_assert` with `__builtin_constant_p`. This commit switches to
using functions and uses the `__diagnose_if__` attribute rather than
`_Static_assert` to enforce constantness.

The remaining macro intrinsics cannot be made into functions until the builtin
functions they are implemented with can be replaced with normal code patterns
because the builtin functions themselves require that their arguments are
constants.

This commit also fixes a bug with the const_splat intrinsics in which the f32x4
and f64x2 variants were incorrectly producing integer vectors.

Differential Revision: https://reviews.llvm.org/D102018
2021-05-07 11:50:19 -07:00
Thomas Lively b198b9b897 [WebAssembly] Fix argument types in SIMD narrowing intrinsics
The builtins were updated to take signed parameters in 627a526955, but the
intrinsics that use those builtins were not updated as well. The intrinsic test
did not catch this sign mismatch because it is only reported as an error under
-fno-lax-vector-conversions.

This commit fixes the type mismatch and adds -fno-lax-vector-conversions to the
test to catch similar problems in the future.

Differential Revision: https://reviews.llvm.org/D101979
2021-05-06 10:07:45 -07:00
Nemanja Ivanovic 1faf3b195e [PowerPC] Re-commit ed87f512bb
This was reverted in 3761b9a234 just
as I was about to commit the fix. This patch inlcudes the
necessary fix.
2021-05-06 09:50:12 -05:00
Nico Weber 3761b9a234 Revert "[PowerPC] Provide some P8-specific altivec overloads for P7"
This reverts commit ed87f512bb.
Breaks check-clang, see e.g.
https://lab.llvm.org/buildbot/#/builders/139/builds/3818
2021-05-06 10:01:16 -04:00
Nemanja Ivanovic ed87f512bb [PowerPC] Provide some P8-specific altivec overloads for P7
This adds additional support for XL compatibility. There are a number
of functions in altivec.h that produce a single instruction (or a
very short sequence) for Power8 but can be done on Power7 without
scalarization. XL provides these implementations.
This patch adds the following overloads for doubleword vectors:
vec_add
vec_cmpeq
vec_cmpgt
vec_cmpge
vec_cmplt
vec_cmple
vec_sl
vec_sr
vec_sra
2021-05-06 08:37:36 -05:00
Johannes Doerfert 5d8d994dfb [OpenMP] Make sure classes work on the device as they do on the host
We do provide `operator delete(void*)` in `<new>` but it should be
available by default. This is mostly boilerplate to test it and the
unconditional include of `<new>` in the header we always in include
on the device.

Reviewed By: JonChesterfield

Differential Revision: https://reviews.llvm.org/D100620
2021-05-06 02:10:30 -05:00
Thomas Lively 81fce29d6e [WebAssembly] Add SIMD const_splat intrinsics
These intrinsics do not correspond to their own underlying instruction, but are
a convenience for the common case of materializing a constant vector that has
the same value in each lane.

Differential Revision: https://reviews.llvm.org/D101885
2021-05-05 13:46:45 -07:00
Thomas Lively 602f318cfd [WebAssembly] Fix constness of pointer params to load intrinsics
Update the SIMD builtin load functions to take pointers to const data and update
the intrinsics themselves to not cast away constness.

Differential Revision: https://reviews.llvm.org/D101884
2021-05-05 13:16:56 -07:00
Nemanja Ivanovic bfd60b36f8 [PowerPC] Add floating point overloads for vec_sldw
These are added for compatibility with XLC.
2021-04-30 20:29:03 -05:00
Nemanja Ivanovic c3da07d216 [PowerPC] Provide fastmath sqrt and div functions in altivec.h
This adds the long overdue implementations of these functions
that have been part of the ABI document and are now part of
the "Power Vector Intrinsic Programming Reference" (PVIPR).

The approach is to add new builtins and to emit code with
the fast flag regardless of whether fastmath was specified
on the command line.

Differential revision: https://reviews.llvm.org/D101209
2021-04-30 19:17:48 -05:00
Anastasia Stulova 3ec82e5195 [OpenCL] Prevent adding vendor extensions for all targets
Removed extension begin/end pragma as it has no effect and
it is added unconditionally for all targets.

Differential Revision: https://reviews.llvm.org/D92244
2021-04-30 14:42:51 +01:00
Wang, Pengfei e0c7db7d8c [MS] Preserve base register %rbx around cpuid
This patch copies implementation from cpuid.h, which preserve base register %rbx around cpuid. It fixes PR50133.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D101338
2021-04-30 10:16:25 +08:00
Luo, Yuanke d6c6db2fea [X86][AMX] Add description for AMX new interface.
Differential Revision: https://reviews.llvm.org/D101059
2021-04-27 16:05:11 +08:00
Thomas Lively 502f54049d [WebAssembly] Finalize wasm_simd128.h intrinsics
Adds new intrinsics for instructions that are in the final SIMD spec but did not
previously have intrinsics. Also updates the names of existing intrinsics to
reflect the final names of the underlying instructions in the spec. Keeps the
old names as deprecated functions to ease the transition to the new names.

Differential Revision: https://reviews.llvm.org/D101112
2021-04-23 13:37:27 -07:00
Nemanja Ivanovic 19b29b1ed1 [PowerPC] Provide XL-compatible builtins in altivec.h
There are some interfaces in altivec.h that are not compatible
between Clang and XL (although Clang is compatible with GCC).
Currently, we have found 3 but there may be others.

Clang/GCC signatures:

vector double vec_ctf(vector signed long long)
vector double vec_ctf(vector unsigned long long)
vector signed long long vec_cts(vector double)
vector unsigned long long vec_ctu(vector double)

XL signatures:

vector float vec_ctf(vector signed long long)
vector float vec_ctf(vector unsigned long long)
vector signed int vec_cts(vector double)
vector unsigned int vec_ctu(vector double)

This patch provides the XL behaviour under the __XL_COMPAT_ALTIVEC__
macro for users that rely on XL behaviour.

Differential revision: https://reviews.llvm.org/D101130
2021-04-23 15:13:46 -05:00
Nemanja Ivanovic 6725b90a02 [PowerPC] Add vec_ctsl and vec_ctul to altivec.h
These are added for compatibility with XLC. They are similar to
vec_cts and vec_ctu except that the result is a doubleword vector
regardless of the parameter type.
2021-04-23 11:03:38 -05:00
Wang, Pengfei e8bce83996 [X86] Enable compilation of user interrupt handlers.
Add __uintr_frame structure and use UIRET instruction for functions with
x86 interrupt calling convention when UINTR is present.

Reviewed By: LuoYuanke

Differential Revision: https://reviews.llvm.org/D99708
2021-04-23 11:43:57 +08:00
Yaxun (Sam) Liu 8baba6890d [HIP] Support overloaded math functions for hipRTC
Remove the dependence on standard C++ header
for overloaded math functions in HIP header
since standard C++ header is not available for hipRTC.

Reviewed by: Artem Belevich, Justin Lebar

Differential Revision: https://reviews.llvm.org/D100794
2021-04-22 19:06:51 -04:00
Nemanja Ivanovic 7a5641d651 [PowerPC] Add missing casts for vec_xlds and vec_load_splats
The previous commits just missed some pointer casts and ended up
producing warnings.
2021-04-22 10:31:00 -05:00
Nemanja Ivanovic 1cc1d9db28 [PowerPC] Add vec_vclz as an alias for vec_cntlz in altivec.h
Another addition for compatibility with XLC. The functions have the
same overloads so just add it as a preprocessor define.
2021-04-22 10:31:00 -05:00
Nemanja Ivanovic e43963db24 [PowerPC] Add vec_load_splats to altivec.h
Add these overloads for compatibility with XLC. This is a word
load-and-splat.
2021-04-22 10:31:00 -05:00
Nemanja Ivanovic a0e6189712 [PowerPC] Add vec_xlds to altivec.h
Add these overloads for compatibility with XLC. This is a doubleword
load-and-splat.
2021-04-22 10:31:00 -05:00
Nemanja Ivanovic a1d325af67 [PowerPC] Add vec_roundz as alias for vec_trunc in altivec.h
Add the overloads for compatibility with XLC.
2021-04-22 10:31:00 -05:00
Nemanja Ivanovic 1550c47c18 [PowerPC] Add vec_roundp as alias for vec_ceil
Add the overloads for compatibility with XLC.
2021-04-22 10:30:59 -05:00
Nemanja Ivanovic 51692c6c63 [PowerPC] Add missing VSX guard for vec_roundm with vector double
The guard was missed in the previous commit.
2021-04-22 10:30:59 -05:00
Nemanja Ivanovic 3a46667059 [PowerPC] Add vec_roundm as alias for vec_floor in altivec.h
Add the overloads for compatibility with XLC.
2021-04-22 10:30:59 -05:00
Nemanja Ivanovic 3bcd0ece43 [PowerPC] Add vec_roundc as alias for vec_rint in altivec.h
For compatibility with XLC, add these overloads.
2021-04-22 05:31:38 -05:00
Liu, Chen3 72e4bf12ee [X86] Support some missing intrinsics
Support for _mm512_i32logather_pd, _mm512_mask_i32logather_pd,
_mm512_i32logather_epi64, _mm512_mask_i32logather_epi64, _mm512_i32loscatter_pd,
_mm512_mask_i32loscatter_pd, _mm512_i32loscatter_epi64,
_mm512_mask_i32loscatter_epi64.

Differential Revision: https://reviews.llvm.org/D100368
2021-04-21 10:50:37 +08:00
Yaxun (Sam) Liu d8805574c1 [CUDA][HIP] Allow non-ODR use of host var in device
Reviewed by: Artem Belevich, Richard Smith

Differential Revision: https://reviews.llvm.org/D98193
2021-04-19 14:45:24 -04:00
Yaxun (Sam) Liu 6823af0ca8 [HIP] Support hipRTC in header
hipRTC compiles HIP device code at run time. Since the system may not
have development tools installed, when a HIP program is compiled through
hipRTC, there is no standard C or C++ header available. As such, the HIP
headers should not depend on standard C or C++ headers when used
with hipRTC. Basically when hipRTC is used, HIP headers only provides
definitions of HIP device API functions. This is in line with what nvRTC does.

This patch adds support of hipRTC to HIP headers in clang. Basically hipRTC
defines a macro __HIPCC_RTC__ when compile HIP code at run time. When
this macro is defined, HIP headers do not include standard C/C++ headers.

Reviewed by: Artem Belevich

Differential Revision: https://reviews.llvm.org/D100652
2021-04-17 11:34:52 -04:00
Sven van Haastregt 35bc7569f8 [OpenCL] Add as_size/ptrdiff/intptr/uintptr_t operators
size_t and friends are built-in scalar data types and s6.4.4.2 of the
OpenCL C Specification says the as_type() operator must be available
for these data types.

Differential Revision: https://reviews.llvm.org/D98959
2021-04-07 10:16:41 +01:00
Yaxun (Sam) Liu 85ff35a952 [HIP] remove overloaded abs in header
This function seems to be introduced by accident by
aa2b593f14

Such overloaded abs function did not exist before
the refactoring, and does not exist in
https://github.com/llvm/llvm-project/blob/main/clang/lib/Headers/__clang_cuda_cmath.h

Conceptually it also does not make sense, since it adds something like

double abs(int x) {
  return ::abs((double)x);
}

It caused regressions in CuPy.

Reviewed by: Aaron Enye Shi, Artem Belevich

Differential Revision: https://reviews.llvm.org/D99738
2021-04-01 12:23:29 -04:00
Sven van Haastregt b5995fced4 [OpenCL] Limit popcount to OpenCL 1.2 and above
s6.15.3 of the OpenCL C Specification v3.0.6 states that OpenCL 1.2 or
newer is required.
2021-03-31 09:54:18 +01:00