Commit Graph

742 Commits

Author SHA1 Message Date
Asaf Badouh 1998eb2077 [X86][AVX512BW] add convert i16 to i8 and unpack intrinsics
Differential Revision: http://reviews.llvm.org/D11564

llvm-svn: 243514
2015-07-29 12:34:20 +00:00
Asaf Badouh a6c31703ac [X86][AVX512BW] Replace attributes with __DEFAULT_FN_ATTRS
llvm-svn: 243512
2015-07-29 12:22:19 +00:00
Asaf Badouh 93aa4c808a [X86][AVX512VL] add AVX512VL intrinsics 4 out of 4
Differential Revision: http://reviews.llvm.org/D11526

llvm-svn: 243409
2015-07-28 12:04:40 +00:00
Asaf Badouh b7cf71b63d [X86][AVX512VL] add AVX512VL intrinsics 3 out of 4
http://reviews.llvm.org/D11526

llvm-svn: 243406
2015-07-28 11:14:09 +00:00
Asaf Badouh 78ee5cc8e1 [X86][AVX512VL] add AVX512VL intrinsics 2 out of 4
http://reviews.llvm.org/D11526

llvm-svn: 243402
2015-07-28 10:30:56 +00:00
Asaf Badouh 74da38706e [X86][AVX512VL] add AVX512VL intrinsics 1 out of 4
http://reviews.llvm.org/D11526

llvm-svn: 243394
2015-07-28 08:26:14 +00:00
Simon Pilgrim f81966d04b [X86] Add missing _m_prefetch intrinsic
The 3DNOW/PRFCHW cpu targets define both the PREFETCHW (set cache line modified) and PREFETCH (set cache line exclusive) instructions but only the _m_prefetchw (PREFETCHW) intrinsic is included in the header. This patch adds the missing _m_prefetch intrinsic.

I'm basing this off AMD documentation - the intel docs on the support for PREFETCHW isn't clear whether Silvermont/Broadwell properly support PREFETCH but given that the intrinsic implementation is a default __builtin_prefetch call, it is safe whatever.

Fix for PR23648

Differential Revision: http://reviews.llvm.org/D11338

llvm-svn: 243305
2015-07-27 19:01:52 +00:00
Asaf Badouh f6a58b6dff [X86][AVX512F] Add FP scalar intrinsics
intrinsics for: add/sub/mul/div/min/max in their FP scalar versions

Differential Revision: http://reviews.llvm.org/D11418

llvm-svn: 243009
2015-07-23 12:13:32 +00:00
Asaf Badouh 7d99966e91 [X86][AVX512BW] add madd and maddubs intrinsics
Differential Revision: http://reviews.llvm.org/D11420

llvm-svn: 242986
2015-07-23 07:07:25 +00:00
Asaf Badouh ffeb624483 [X86][AVX512F] add FP arithmetic intrinsics
add/div/mul/sub include rounding versions


Differential Revision: http://reviews.llvm.org/D11354

llvm-svn: 242790
2015-07-21 15:27:28 +00:00
Asaf Badouh d4419ca657 [X86][AVX512BW] add clang intrinsics for pmulhrsw / pmulhuw / pmulhw
also made minor fix in "test_mm512_maskz_permutex2var_epi16"

Differential Revision: http://reviews.llvm.org/D11336

llvm-svn: 242635
2015-07-19 08:47:31 +00:00
David Majnemer 6b8e297089 [Intrin.h] Use compiler builtins to model memory barriers
_ReadBarrier, _WriteBarrier, and _ReadWriteBarrier are essentially
memory barriers of one form or another.  Model these as
atomic_signal_fence(ATOMIC_SEQ_CST).

__faststorefence is a curious intrinsic.  It's single purpose seems to
an alternative to mfence when that instruction is slow.  However, mfence
is not always slow and is, in general, preferable to a 'lock or'
sequence on certain CPUs.  Give the compiler freedom to select the best
sequence to get a fence.

llvm-svn: 242378
2015-07-16 03:13:02 +00:00
Bill Schmidt 8da737a18a [PPC64LE] Fix vec_sld semantics for little endian
The vec_sld interface provides access to the vsldoi instruction.
Unlike most of the vec_* interfaces, we do not attempt to change the
generated code for vec_sld based on the endian mode.  It is too
difficult to correctly infer the desired semantics because of
different element types, and the corrected instruction sequence is
expensive, involving loading a permute control vector and performing a
generalized permute.

For GCC, this was implemented as "Don't touch the vec_sld"
implementation.  When it came time for the LLVM implementation, I did
the same thing.  However, this was hasty and incorrect.  In LLVM's
version of altivec.h, vec_sld was previously defined in terms of the
vec_perm interface.  Because vec_perm semantics are adjusted for
little endian, this means that leaving vec_sld untouched causes it to
generate something different for LE than for BE.  Not good.

This patch adjusts the form of vec_perm that is used for vec_sld and
vec_vsldoi, effectively undoing the modifications so that the same
vsldoi instruction will be generated for both BE and LE.

There is an accompanying back-end patch to take care of some small
ripple effects caused by these changes.

llvm-svn: 242297
2015-07-15 15:45:53 +00:00
Nemanja Ivanovic 6c363ed67a Add missing builtins to altivec.h for ABI compliance (vol. 4)
This patch corresponds to review:
http://reviews.llvm.org/D11184

A number of new interfaces for altivec.h (as mandated by the ABI):
vector float vec_cpsgn(vector float, vector float)
vector double vec_cpsgn(vector double, vector double)
vector double vec_or(vector bool long long, vector double)
vector double vec_or(vector double, vector bool long long)
vector double vec_re(vector double)
vector signed char vec_cntlz(vector signed char)
vector unsigned char vec_cntlz(vector unsigned char)
vector short vec_cntlz(vector short)
vector unsigned short vec_cntlz(vector unsigned short)
vector int vec_cntlz(vector int)
vector unsigned int vec_cntlz(vector unsigned int)
vector signed long long vec_cntlz(vector signed long long)
vector unsigned long long vec_cntlz(vector unsigned long long)
vector signed char vec_nand(vector bool signed char, vector signed char)
vector signed char vec_nand(vector signed char, vector bool signed char)
vector signed char vec_nand(vector signed char, vector signed char)
vector unsigned char vec_nand(vector bool unsigned char, vector unsigned char)
vector unsigned char vec_nand(vector unsigned char, vector bool unsigned char)
vector unsigned char vec_nand(vector unsigned char, vector unsigned char)
vector short vec_nand(vector bool short, vector short)
vector short vec_nand(vector short, vector bool short)
vector short vec_nand(vector short, vector short)
vector unsigned short vec_nand(vector bool unsigned short, vector unsigned short)
vector unsigned short vec_nand(vector unsigned short, vector bool unsigned short)
vector unsigned short vec_nand(vector unsigned short, vector unsigned short)
vector int vec_nand(vector bool int, vector int)
vector int vec_nand(vector int, vector bool int)
vector int vec_nand(vector int, vector int)
vector unsigned int vec_nand(vector bool unsigned int, vector unsigned int)
vector unsigned int vec_nand(vector unsigned int, vector bool unsigned int)
vector unsigned int vec_nand(vector unsigned int, vector unsigned int)
vector signed long long vec_nand(vector bool long long, vector signed long long)
vector signed long long vec_nand(vector signed long long, vector bool long long)
vector signed long long vec_nand(vector signed long long, vector signed long long)
vector unsigned long long vec_nand(vector bool long long, vector unsigned long long)
vector unsigned long long vec_nand(vector unsigned long long, vector bool long long)
vector unsigned long long vec_nand(vector unsigned long long, vector unsigned long long)
vector signed char vec_orc(vector bool signed char, vector signed char)
vector signed char vec_orc(vector signed char, vector bool signed char)
vector signed char vec_orc(vector signed char, vector signed char)
vector unsigned char vec_orc(vector bool unsigned char, vector unsigned char)
vector unsigned char vec_orc(vector unsigned char, vector bool unsigned char)
vector unsigned char vec_orc(vector unsigned char, vector unsigned char)
vector short vec_orc(vector bool short, vector short)
vector short vec_orc(vector short, vector bool short)
vector short vec_orc(vector short, vector short)
vector unsigned short vec_orc(vector bool unsigned short, vector unsigned short)
vector unsigned short vec_orc(vector unsigned short, vector bool unsigned short)
vector unsigned short vec_orc(vector unsigned short, vector unsigned short)
vector int vec_orc(vector bool int, vector int)
vector int vec_orc(vector int, vector bool int)
vector int vec_orc(vector int, vector int)
vector unsigned int vec_orc(vector bool unsigned int, vector unsigned int)
vector unsigned int vec_orc(vector unsigned int, vector bool unsigned int)
vector unsigned int vec_orc(vector unsigned int, vector unsigned int)
vector signed long long vec_orc(vector bool long long, vector signed long long)
vector signed long long vec_orc(vector signed long long, vector bool long long)
vector signed long long vec_orc(vector signed long long, vector signed long long)
vector unsigned long long vec_orc(vector bool long long, vector unsigned long long)
vector unsigned long long vec_orc(vector unsigned long long, vector bool long long)
vector unsigned long long vec_orc(vector unsigned long long, vector unsigned long long)
vector signed char vec_div(vector signed char, vector signed char)
vector unsigned char vec_div(vector unsigned char, vector unsigned char)
vector signed short vec_div(vector signed short, vector signed short)
vector unsigned short vec_div(vector unsigned short, vector unsigned short)
vector signed int vec_div(vector signed int, vector signed int)
vector unsigned int vec_div(vector unsigned int, vector unsigned int)
vector signed long long vec_div(vector signed long long, vector signed long long)
vector unsigned long long vec_div(vector unsigned long long, vector unsigned long long)
vector unsigned char vec_mul(vector unsigned char, vector unsigned char)
vector unsigned int vec_mul(vector unsigned int, vector unsigned int)
vector unsigned long long vec_mul(vector unsigned long long, vector unsigned long long)
vector unsigned short vec_mul(vector unsigned short, vector unsigned short)
vector signed char vec_mul(vector signed char, vector signed char)
vector signed int vec_mul(vector signed int, vector signed int)
vector signed long long vec_mul(vector signed long long, vector signed long long)
vector signed short vec_mul(vector signed short, vector signed short)
vector signed long long vec_mergeh(vector signed long long, vector signed long long)
vector signed long long vec_mergeh(vector signed long long, vector bool long long)
vector signed long long vec_mergeh(vector bool long long, vector signed long long)
vector unsigned long long vec_mergeh(vector unsigned long long, vector unsigned long long)
vector unsigned long long vec_mergeh(vector unsigned long long, vector bool long long)
vector unsigned long long vec_mergeh(vector bool long long, vector unsigned long long)
vector double vec_mergeh(vector double, vector double)
vector double vec_mergeh(vector double, vector bool long long)
vector double vec_mergeh(vector bool long long, vector double)
vector signed long long vec_mergel(vector signed long long, vector signed long long)
vector signed long long vec_mergel(vector signed long long, vector bool long long)
vector signed long long vec_mergel(vector bool long long, vector signed long long)
vector unsigned long long vec_mergel(vector unsigned long long, vector unsigned long long)
vector unsigned long long vec_mergel(vector unsigned long long, vector bool long long)
vector unsigned long long vec_mergel(vector bool long long, vector unsigned long long)
vector double vec_mergel(vector double, vector double)
vector double vec_mergel(vector double, vector bool long long)
vector double vec_mergel(vector bool long long, vector double)
vector signed int vec_pack(vector signed long long, vector signed long long)
vector unsigned int vec_pack(vector unsigned long long, vector unsigned long long)
vector bool int vec_pack(vector bool long long, vector bool long long)

llvm-svn: 242171
2015-07-14 17:50:27 +00:00
Asaf Badouh 1626545667 [x86] add 2 bit to ObjCOrBuiltinID and new intrinsics
add 2 bit to ObjCOrBuiltinID (changed from 11bits to 13bits), see discussion in
Add new intrinsics support that already covered by the BE.
All the intrinsics are covered by tests

Differential Revision: http://reviews.llvm.org/D10893

llvm-svn: 242144
2015-07-14 14:02:45 +00:00
David Majnemer e0b863f4c7 [Intrin.h] Use __ATOMIC_SEQ_CST instead of '5'
No functionality change is intended.

llvm-svn: 242087
2015-07-13 23:39:37 +00:00
David Majnemer 56e466745d [Intrin.h] Make the variable names more consistent
No functionality change intended.

llvm-svn: 242086
2015-07-13 23:38:56 +00:00
David Majnemer 8dadce78ed Intrin.h: Don't invade the program's namespace
The program is permitted to have stuff like '#define x' in it so avoid
using identifiers not reserved for the implementation.

llvm-svn: 242010
2015-07-13 02:53:23 +00:00
David Majnemer 3c8ea5f3f8 Intrin.h: Clean up our atomic intrinsics
Three things:
- The atomic intrinsics mandate memory barriers, let's start emitting
  some.
- We don't need to manually create RMW operations, we can just do
  __atomic_fetch_foo instead of performing __atomic_foo_fetch and
  undoing foo.
- Don't use inline assembly, we don't need it for these intrinsics.

This fixes PR24101.

llvm-svn: 242009
2015-07-13 02:53:19 +00:00
Nemanja Ivanovic 26c3534b84 Add missing builtins to altivec.h for ABI compliance (vol. 3)
This patch corresponds to review:
http://reviews.llvm.org/D10972

Fix for the handling of dependent features that are enabled by default
on some CPU's (such as -mvsx, -mpower8-vector).

Also provides a number of new interfaces or fixes existing ones in
altivec.h.

Changed signatures to conform to ABI:
vector short vec_perm(vector signed short, vector signed short, vector unsigned char)
vector int vec_perm(vector signed int, vector signed int, vector unsigned char)
vector long long vec_perm(vector signed long long, vector signed long long, vector unsigned char)
vector signed char vec_sld(vector signed char, vector signed char, const int)
vector unsigned char vec_sld(vector unsigned char, vector unsigned char, const int)
vector bool char vec_sld(vector bool char, vector bool char, const int)
vector unsigned short vec_sld(vector unsigned short, vector unsigned short, const int)
vector signed short vec_sld(vector signed short, vector signed short, const int)
vector signed int vec_sld(vector signed int, vector signed int, const int)
vector unsigned int vec_sld(vector unsigned int, vector unsigned int, const int)
vector float vec_sld(vector float, vector float, const int)
vector signed char vec_splat(vector signed char, const int)
vector unsigned char vec_splat(vector unsigned char, const int)
vector bool char vec_splat(vector bool char, const int)
vector signed short vec_splat(vector signed short, const int)
vector unsigned short vec_splat(vector unsigned short, const int)
vector bool short vec_splat(vector bool short, const int)
vector pixel vec_splat(vector pixel, const int)
vector signed int vec_splat(vector signed int, const int)
vector unsigned int vec_splat(vector unsigned int, const int)
vector bool int vec_splat(vector bool int, const int)
vector float vec_splat(vector float, const int)

Added a VSX path to:
vector float vec_round(vector float)

Added interfaces:
vector signed char vec_eqv(vector signed char, vector signed char)
vector signed char vec_eqv(vector bool char, vector signed char)
vector signed char vec_eqv(vector signed char, vector bool char)
vector unsigned char vec_eqv(vector unsigned char, vector unsigned char)
vector unsigned char vec_eqv(vector bool char, vector unsigned char)
vector unsigned char vec_eqv(vector unsigned char, vector bool char)
vector signed short vec_eqv(vector signed short, vector signed short)
vector signed short vec_eqv(vector bool short, vector signed short)
vector signed short vec_eqv(vector signed short, vector bool short)
vector unsigned short vec_eqv(vector unsigned short, vector unsigned short)
vector unsigned short vec_eqv(vector bool short, vector unsigned short)
vector unsigned short vec_eqv(vector unsigned short, vector bool short)
vector signed int vec_eqv(vector signed int, vector signed int)
vector signed int vec_eqv(vector bool int, vector signed int)
vector signed int vec_eqv(vector signed int, vector bool int)
vector unsigned int vec_eqv(vector unsigned int, vector unsigned int)
vector unsigned int vec_eqv(vector bool int, vector unsigned int)
vector unsigned int vec_eqv(vector unsigned int, vector bool int)
vector signed long long vec_eqv(vector signed long long, vector signed long long)
vector signed long long vec_eqv(vector bool long long, vector signed long long)
vector signed long long vec_eqv(vector signed long long, vector bool long long)
vector unsigned long long vec_eqv(vector unsigned long long, vector unsigned long long)
vector unsigned long long vec_eqv(vector bool long long, vector unsigned long long)
vector unsigned long long vec_eqv(vector unsigned long long, vector bool long long)
vector float vec_eqv(vector float, vector float)
vector float vec_eqv(vector bool int, vector float)
vector float vec_eqv(vector float, vector bool int)
vector double vec_eqv(vector double, vector double)
vector double vec_eqv(vector bool long long, vector double)
vector double vec_eqv(vector double, vector bool long long)
vector bool long long vec_perm(vector bool long long, vector bool long long, vector unsigned char)
vector double vec_round(vector double)
vector double vec_splat(vector double, const int)
vector bool long long vec_splat(vector bool long long, const int)
vector signed long long vec_splat(vector signed long long, const int)
vector unsigned long long vec_splat(vector unsigned long long,
vector bool int vec_sld(vector bool int, vector bool int, const int)
vector bool short vec_sld(vector bool short, vector bool short, const int)

llvm-svn: 241904
2015-07-10 13:11:34 +00:00
Nemanja Ivanovic e00fa61412 Add the missing return statements from revision 241399.
llvm-svn: 241405
2015-07-05 10:54:10 +00:00
Nemanja Ivanovic 1c7ad715ec Add missing builtins to altivec.h for ABI compliance (vol. 2)
This patch corresponds to review:
http://reviews.llvm.org/D10875

The bulk of the second round of additions to altivec.h.
The following interfaces were added:
vector double vec_floor(vector double)
vector double vec_madd(vector double, vector double, vector double)
vector float vec_msub(vector float, vector float, vector float)
vector double vec_msub(vector double, vector double, vector double)
vector float vec_mul(vector float, vector float)
vector double vec_mul(vector double, vector double)
vector float vec_nmadd(vector float, vector float, vector float)
vector double vec_nmadd(vector double, vector double, vector double)
vector double vec_nmsub(vector double, vector double, vector double)
vector double vec_nor(vector double, vector double)
vector double vec_or(vector double, vector double)
vector float vec_rint(vector float)
vector double vec_rint(vector double)
vector float vec_nearbyint(vector float)
vector double vec_nearbyint(vector double)
vector float vec_sqrt(vector float)
vector double vec_sqrt(vector double)
vector double vec_rsqrte(vector double)
vector double vec_sel(vector double, vector double, vector unsigned long long)
vector double vec_sel(vector double, vector double, vector unsigned long long)
vector double vec_sub(vector double, vector double)
vector double vec_trunc(vector double)
vector double vec_xor(vector double, vector double)
vector double vec_xor(vector double, vector bool long long)
vector double vec_xor(vector bool long long, vector double)

New VSX paths for the following interfaces:
vector float vec_madd(vector float, vector float, vector float)
vector float vec_nmsub(vector float, vector float, vector float)
vector float vec_rsqrte(vector float)
vector float vec_trunc(vector float)
vector float vec_floor(vector float)

llvm-svn: 241399
2015-07-05 06:40:52 +00:00
Kit Barton b61173e791 This patch adds support for the vector merge even word and vector merge odd word
instructions introduced in POWER8.

These are the Clang-related changes for http://reviews.llvm.org/D10704

All builtins are added in altivec.h and guarded with the POWER8_VECTOR macro.

Phabricator review: http://reviews.llvm.org/D10736

llvm-svn: 241293
2015-07-02 19:29:05 +00:00
Michael Kuperstein e45af54cdb [X86] Rename DEFAULT_FN_ATTR macro to __DEFAULT_FN_ATTR
llvm-svn: 241065
2015-06-30 13:36:19 +00:00
Michael Kuperstein 9101a98bd0 [X86] Add missing undef of DEFAULT_FN_ATTRS in FXSR intrinsics
llvm-svn: 241055
2015-06-30 10:18:54 +00:00
Michael Kuperstein a3c7b74208 [X86] Add FXSR intrinsics
Add intrinsics for the FXSR instructions (FXSAVE/FXSAVE64/FXRSTOR/FXRSTOR64)

These were previously declared in Intrin.h for MSVC compatibility, but now
that we have them implemented, these declarations can be removed.

llvm-svn: 241053
2015-06-30 09:45:38 +00:00
Asaf Badouh a45b7cab7b [x86][AVX512CD] Add conflict and lzcnt intrinsics in their 512bit versions
include tests

review
http://reviews.llvm.org/D10795

llvm-svn: 240941
2015-06-29 12:51:53 +00:00
Asaf Badouh 4002ce4834 [X86][AVX512BW] Add more intrinsics support:
Blend, abs, packs, adds, subs, avg, max, min, permute.
 all the intrinsics are covered by tests

review:
http://reviews.llvm.org/D10799

llvm-svn: 240937
2015-06-29 12:16:40 +00:00
Elena Demikhovsky c563c2c61a AVX-512: Implemented AVX-512 FMA intrinsics and tests.
by Igor Breger

http://reviews.llvm.org/D10797

llvm-svn: 240928
2015-06-29 09:20:57 +00:00
Nemanja Ivanovic 2f1f926e34 Add missing builtins to altivec.h for ABI compliance (vol. 1)
This patch corresponds to review:
http://reviews.llvm.org/D10637

This is the first round of additions of missing builtins listed in the ABI document. More to come (this builds onto what seurer already addes). This patch adds:
vector signed long long vec_abs(vector signed long long)
vector double vec_abs(vector double)
vector signed long long vec_add(vector signed long long, vector signed long long)
vector unsigned long long vec_add(vector unsigned long long, vector unsigned long long)
vector double vec_add(vector double, vector double)
vector double vec_and(vector bool long long, vector double)
vector double vec_and(vector double, vector bool long long)
vector double vec_and(vector double, vector double)
vector signed long long vec_and(vector signed long long, vector signed long long)
vector double vec_andc(vector bool long long, vector double)
vector double vec_andc(vector double, vector bool long long)
vector double vec_andc(vector double, vector double)
vector signed long long vec_andc(vector signed long long, vector signed long long)
vector double vec_ceil(vector double)
vector bool long long vec_cmpeq(vector double, vector double)
vector bool long long vec_cmpge(vector double, vector double)
vector bool long long vec_cmpge(vector signed long long, vector signed long long)
vector bool long long vec_cmpge(vector unsigned long long, vector unsigned long long)
vector bool long long vec_cmpgt(vector double, vector double)
vector bool long long vec_cmple(vector double, vector double)
vector bool long long vec_cmple(vector signed long long, vector signed long long)
vector bool long long vec_cmple(vector unsigned long long, vector unsigned long long)
vector bool long long vec_cmplt(vector double, vector double)
vector bool long long vec_cmplt(vector signed long long, vector signed long long)
vector bool long long vec_cmplt(vector unsigned long long, vector unsigned long long)

llvm-svn: 240821
2015-06-26 19:27:20 +00:00
Nico Weber ac64b97771 Add new file from r240741 to CMakeLists.txt.
llvm-svn: 240743
2015-06-26 00:19:32 +00:00
Nico Weber 2ca46867e1 Add an inttypes.h wrapper that fixes up some macros in Microsoft mode.
Before MSVS2015, MSVS's headers disagree about int32_t and PRIx32 and so on.
Provide a wrapper header to fix this, so that -Wformat can still be used.
Fixes PR23412.

llvm-svn: 240741
2015-06-26 00:13:18 +00:00
Sean Silva d0de76a3da Remove `requires` for x86 CPU features.
Ever since the target attributes change, we don't need to guard these
headers with `requires`. Actually it's a bit worse, because if we do
then they are included textually under the covers, causing declarations
to appear in submodules they aren't supposed to be in.

llvm-svn: 240720
2015-06-25 23:22:11 +00:00
Eric Christopher 3d920eed5d Move xtest to its own file to match the gcc header organization.
llvm-svn: 239926
2015-06-17 18:42:07 +00:00
Eric Christopher 29b78091e7 Update comments on HLE, RTM, and ADX support for intrinsics.
llvm-svn: 239925
2015-06-17 18:42:03 +00:00
Eric Christopher 9fc7fb274e Update the intel intrinsic headers to use the target attribute support.
This involved removing the conditional inclusion and replacing them
with target attributes matching the original conditional inclusion
and checks. The testcase update removes the macro checks for each
file and replaces them with usage of the __target__ attribute, e.g.:

int __attribute__((__target__(("sse3")))) foo(int a) {
  _mm_mwait(0, 0);
  return 4;
}

This usage does require the enclosing function have the requisite
__target__ attribute for inlining and code generation - also for
any macro intrinsic uses in the enclosing function. There's no change
for existing uses of the intrinsic headers.

llvm-svn: 239883
2015-06-17 07:09:32 +00:00
Eric Christopher 4d185168e9 Use a define for per-file function attributes for the Intel intrinsic headers.
This is a precursor to changing them to use the new target attribute
code.

llvm-svn: 239882
2015-06-17 07:09:20 +00:00
Eric Christopher 5a9bec104b Use a macro for the omnipresent attributes on header functions in Intrin.h.
Saves some typing and if someone wants to change them it makes it much easier.

llvm-svn: 239782
2015-06-15 23:20:35 +00:00
Luke Cheeseman 59b2d83909 This patch implements clang support for the ACLE special register intrinsics
in section 10.1, __arm_{w,r}sr{,p,64}.

This includes arm_acle.h definitions with builtins and codegen to support
these, the intrinsics are implemented by generating read/write_register calls
which get appropriately lowered in the backend based on the register string
provided. SemaChecking is also implemented to fault invalid parameters.

Differential Revision: http://reviews.llvm.org/D9697

llvm-svn: 239737
2015-06-15 17:51:01 +00:00
Nemanja Ivanovic b17f1129fa Clang support for vector quad bit permute and gather instructions through builtins
This patch corresponds to review:
http://reviews.llvm.org/D10095

This is for just two instructions and related builtins:
vbpermq
vgbbd

llvm-svn: 239506
2015-06-11 06:25:36 +00:00
Bill Seurer 703e8486ec [PowerPC] Reformat altivec.h with clang-format
This revision just fixes the formatting of altivec.h.

llvm-svn: 239408
2015-06-09 14:39:47 +00:00
David Majnemer 81ecbf45d4 Revert accidental commit
This change was unrelated to r239170.

llvm-svn: 239176
2015-06-05 18:24:55 +00:00
David Majnemer cdffc36c11 [AST] There is no message for C++1z-style static_assert
We would crash in the DeclPrinter trying to pretty-print the
static_assert message.  C++1z-style assertions don't have a message so
we would crash.

This fixes PR23756.

llvm-svn: 239170
2015-06-05 18:03:58 +00:00
Bill Seurer 8be14f11ce [PowerPC] This revision adds 68 of the missing "Predefined Functions for Vector Programming" from appendix A of the OpenPOWER ABI for Linux Supplement document.
I also added tests for the new functions and updated another test that was looking for specific line numbers in error messages from altivec.h.

https://llvm.org/bugs/show_bug.cgi?id=23679

http://reviews.llvm.org/D10131

llvm-svn: 239066
2015-06-04 18:45:44 +00:00
Ekaterina Romanova 2e81434552 Added doxygen comments for the intrinsics.
llvm-svn: 238386
2015-05-28 01:25:25 +00:00
John Thompson b7892ffc69 It appears these exports are needed, as wmmintrin.h includes them.
llvm-svn: 238345
2015-05-27 18:26:41 +00:00
Kit Barton 5944ee2179 This patch adds support for the vector quadword add/sub instructions introduced
in POWER8.

These are the Clang-related changes for http://reviews.llvm.org/D9081

vadduqm
vaddeuqm
vaddcuq
vaddecuq
vsubuqm
vsubeuqm
vsubcuq
vsubecuq
All builtins are added in altivec.h, and guarded with the POWER8_VECTOR and
powerpc64 macros.



http://reviews.llvm.org/D9903

llvm-svn: 238145
2015-05-25 15:52:45 +00:00
Michael Kuperstein 7619004211 [X86] Add _mm256_set_m128 and its 5 variants.
Differential Revision: http://reviews.llvm.org/D9855

llvm-svn: 237778
2015-05-20 07:46:52 +00:00
Michael Kuperstein 877f3cbe84 [X86] Add _mm_broadcastsd_pd intrinsic
_mm_broadcastsd_pd is basically an alias for _mm_movedup_pd, however the alias is only available from AVX2 forward.

llvm-svn: 237698
2015-05-19 14:49:14 +00:00
Michael Kuperstein 6168183e04 [X86] Added _mm256_bslli_epi128 and _mm256_bsrli_epi128.
These two intrinsics are alternative names for _mm256_slli_si256 and _mm256_srli_si256, respectively.

llvm-svn: 237693
2015-05-19 13:05:46 +00:00
Bill Schmidt 41e14c4dfa [PPC64] Add vector pack/unpack support from ISA 2.07
This patch adds support for the following new instructions in the
Power ISA 2.07:

  vpksdss
  vpksdus
  vpkudus
  vpkudum
  vupkhsw
  vupklsw

These instructions are available through the vec_packs, vec_packsu,
vec_unpackh, and vec_unpackl built-in interfaces.  These are
lane-sensitive instructions, so the built-ins have different
implementations for big- and little-endian, and the instructions must
be marked as killing the vector swap optimization for now.

The first three instructions perform saturating pack operations.  The
fourth performs a modulo pack operation, which means it can be
represented with a vector shuffle, and conversely the appropriate
vector shuffles may cause this instruction to be generated.  The other
instructions are only generated via built-in support for now.

I noticed during patch preparation that the macro __VSX__ was not
previously predefined when the power8-vector or direct-move features
are requested.  This is an error, and I've corrected that here as
well.

Appropriate tests have been added.

There is a companion patch to llvm for the rest of this support.

llvm-svn: 237500
2015-05-16 01:02:25 +00:00
Richard Smith 23d8d0338e [modules] Fix a #include cycle when building a module for our builtin headers.
xmmintrin.h includes emmintrin.h and vice versa if SSE2 is enabled. We break
this cycle for a modules build, and instead make the xmmintrin.h module
re-export the immintrin.h module. Also included is a fix for an assert in the
serialization code if a module exports another module that was declared later
in the same module map.

llvm-svn: 237321
2015-05-14 00:45:20 +00:00
Elena Demikhovsky bd5c8b9be9 AVX-512: FP compare intrinsics - changed type of CC parameter from i8 to i32 according to the spec.
Added FP compare intrinsics for SKX.

llvm-svn: 236715
2015-05-07 11:26:36 +00:00
Elena Demikhovsky e7d4c2e229 AVX-512: Added AVX-512 intrinsics and tests
by Asaf Badouh (asaf.badouh@intel.com)

llvm-svn: 236218
2015-04-30 09:24:29 +00:00
Elena Demikhovsky 35dc8c0944 AVX-512: added intrinsics for KNL and SKX
by Asaf Badouh (asaf.badouh@intel.com)

llvm-svn: 235986
2015-04-28 13:28:01 +00:00
Artem Belevich 4e192df778 [cuda] Added support for CUDA built-in variables.
Added cuda_builtin_vars.h which implements built-in CUDA variables
using __declattr(property).

Fields of built-in variables (except for warpSize) are implemented
using __declattr(property) which replaces read/write of a member field
with a call to a getter/setter member function, in this case with
appropriate NVPTX builtin.

Added a test case to check diagnostics on attempt to construct or
improperly access a built-in variable.

Differential Revision: http://reviews.llvm.org/D9064

llvm-svn: 235448
2015-04-21 22:14:13 +00:00
Artem Belevich a050112bba Revert r235398 "[cuda] Added support for CUDA built-in variables."
r235398 was causing buildbot break due to missing Makefile changes.

llvm-svn: 235401
2015-04-21 18:36:42 +00:00
Artem Belevich d0a2ae054f [cuda] Added support for CUDA built-in variables.
Added cuda_builtin_vars.h which implements built-in CUDA variables
using __declattr(property).

Fields of built-in variables (except for warpSize) are implemented
using __declattr(property) which replaces read/write of a member field
with a call to a getter/setter member function, in this case with
appropriate NVPTX builtin.

Added a test case to check diagnostics on attempt to construct or
improperly access a built-in variable.

Differential Revision: http://reviews.llvm.org/D9064

llvm-svn: 235398
2015-04-21 17:39:06 +00:00
Ekaterina Romanova b929ad7b17 _mm256_blend_epi16 is being cast to __m256d instead of __m256i. Fixing this.
llvm-svn: 234560
2015-04-10 02:39:45 +00:00
Ulrich Weigand cc67344a86 [SystemZ] Add header files to Makefile / module.modulemap
This should fix build-bot failures after r233804.

The patch also adds a "systemz" feature, and renames the
"transactional-execution" feature to "htm", since it turns
out "-" is not a legal character in module feature names.

llvm-svn: 233807
2015-04-01 14:15:35 +00:00
Ulrich Weigand 3a610ebf1e [SystemZ] Support transactional execution on zEC12
The zEC12 provides the transactional-execution facility.  This is exposed
to users via a set of builtin routines on other compilers.  This patch
adds clang support to enable those builtins.  In partciular, the patch:

- enables the transactional-execution feature by default on zEC12
- allows to override presence of that feature via the -mhtm/-mno-htm options
- adds a predefined macro __HTM__ if the feature is enabled
- adds support for the transactional-execution GCC builtins
- adds Sema checking to verify the __builtin_tabort abort code
- adds the s390intrin.h header file (for GCC compatibility)
- adds s390 sections to the htmintrin.h and htmxlintrin.h header files

Since this is first use of target-specific intrinsics on the platform,
the patch creates the include/clang/Basic/BuiltinsSystemZ.def file and
hooks it up in TargetBuiltins.h and lib/Basic/Targets.cpp.

An associated LLVM patch adds the required LLVM IR intrinsics.

For reference, the transactional-execution instructions are documented
in the z/Architecture Principles of Operation for the zEC12:
http://publibfp.boulder.ibm.com/cgi-bin/bookmgr/download/DZ9ZR009.pdf
The associated builtins are documented in the GCC manual:
http://gcc.gnu.org/onlinedocs/gcc/S_002f390-System-z-Built-in-Functions.html
The htmxlintrin.h intrinsics provided for compatibility with the IBM XL
compiler are documented in the "z/OS XL C/C++ Programming Guide".

llvm-svn: 233804
2015-04-01 12:54:25 +00:00
Elena Demikhovsky 29da2fba46 AVX-512: added clang intrinsics for logical and, or xor for 512 bits
by Asaf Badouh (asaf.badouh@intel.com)

llvm-svn: 233794
2015-04-01 06:54:16 +00:00
Kit Barton 8246f28237 Add Hardware Transactional Memory (HTM) Support
This patch adds Hardware Transaction Memory (HTM) support supported by ISA 2.07
(POWER8). The intrinsic support is based on GCC one [1], with both 'PowerPC HTM
Low Level Built-in Functions' and 'PowerPC HTM High Level Inline Functions'
implemented.

Along with builtins a new driver switch is added to enable/disable HTM
instruction support (-mhtm) and a header with common definitions (mostly to
parse the TFHAR register value). The HTM switch also sets a preprocessor builtin
HTM.

The HTM usage requires a recently newer kernel with PPC HTM enabled. Tested on
powerpc64 and powerpc64le.

This is send along a llvm patch to enabled the builtins and option switch.

[1]
https://gcc.gnu.org/onlinedocs/gcc/PowerPC-Hardware-Transactional-Memory-Built-in-Functions.html

Phabricator Review: http://reviews.llvm.org/D8248

llvm-svn: 233205
2015-03-25 19:41:41 +00:00
Sanjay Patel 0a6da5de55 [X86, AVX2] Replace inserti128 and extracti128 intrinsics with generic shuffles
This is nearly identical to the v*f128_si256 parts of r231792 and r232052.

AVX2 introduced proper integer variants of the hacked integer insert/extract
C intrinsics that were created for this same functionality with AVX1.

This should complete the front end fixes for insert/extract128 intrinsics. 
Corresponding LLVM patch to follow.

llvm-svn: 232109
2015-03-12 21:54:24 +00:00
Sanjay Patel f204b00940 Replace second (hopefully unused) access of macro input argument with zero vector to be safer.
Suggested by Craig Topper in D8275.

This is a follow-on to r232052.

llvm-svn: 232061
2015-03-12 17:23:46 +00:00
Sanjay Patel 0c351aba25 [X86, AVX] replace vextractf128 intrinsics with generic shuffles
This is very much like D8088 (checked in at r231792).

Now that we've replaced the vinsertf128 intrinsics,
do the same for their extract twins.

Differential Revision: http://reviews.llvm.org/D8275

llvm-svn: 232052
2015-03-12 15:50:36 +00:00
Kit Barton 8553bec911 Add builtins for the 64-bit vector integer arithmetic instructions added in POWER8.
These are the Clang-related changes for the instructions added to LLVM in http://reviews.llvm.org/D7959.

Phabricator review: http://reviews.llvm.org/D8041

llvm-svn: 231931
2015-03-11 15:57:19 +00:00
Sanjay Patel 7f6aa52e93 [X86, AVX] Replace vinsertf128 intrinsics with generic shuffles.
We want to replace as much custom x86 shuffling via intrinsics
as possible because pushing the code down the generic shuffle
optimization path allows for better codegen and less complexity
in LLVM.

This is the sibling patch for the LLVM half of this change:
http://reviews.llvm.org/D8086

Differential Revision: http://reviews.llvm.org/D8088

llvm-svn: 231792
2015-03-10 15:19:26 +00:00
Nemanja Ivanovic 55e757db4a Add Clang support for PPC cryptography builtins
Review: http://reviews.llvm.org/D7951

llvm-svn: 231291
2015-03-04 21:48:22 +00:00
Juergen Ributzka 9baa03fc07 Lower _mm256_broadcastsi128_si256 directly to a vector shuffle.
Originally we were using the same GCC builtins to lower this AVX2 vector
intrinsic. Instead we will now lower it directly to a vector shuffle.

This will not only allow LLVM to generate better code, but it will also allow us
to remove the GCC intrinsics.

Reviewed by Andrea

This is related to rdar://problem/18742778.

llvm-svn: 231081
2015-03-03 17:22:53 +00:00
Dmitri Gribenko a586ea13a4 Restore the libc++ definition of max_align_t on Apple platforms
Clang has introduced ::max_align_t in stddef.h in r201729, but libc++ was
already defining std::max_align_t on Darwin because there was none in the
global namespace.  After that Clang commit though, libc++ started defining
std::max_align_t to be a typedef for ::max_align_t, which has a different
definition.  This changed the ABI.  This commit restores the previous
definition.

rdar://19919394 rdar://18557982

llvm-svn: 230292
2015-02-24 01:06:22 +00:00
Filipe Cabecinhas d74002965e Make the _mm256_insert_epi64 definition more consistent
Use long long for the epi64 argument, like the other intrinsics.
NFC since this is only defined in 64-bit mode, not in 32-bit.

Fix suggested by H. J. Lu!

llvm-svn: 229886
2015-02-19 19:00:33 +00:00
Filipe Cabecinhas 54a2ba8b76 [Headers] Add tests for _mm256_insert_epi64 and fix its definition
Summary:
The definition for _mm256_insert_epi64 was taking an int, which would get
truncated before being inserted in the vector.

Original patch by Joshua Magee!

Reviewers: bruno, craig.topper

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D7179

llvm-svn: 229811
2015-02-19 03:02:33 +00:00
Craig Topper a462482d98 [X86] Add _mm_bslli_si128 and _mm_bsrli_si128 as aliases of _mm_slli_si128 and _mm_srli_si128. This matches Intel documentation and gcc.
llvm-svn: 229066
2015-02-13 06:04:45 +00:00
Craig Topper 51e47418d4 [X86] Simplify some code and remove some -Wshadow disables from intrinsic header.
llvm-svn: 229065
2015-02-13 06:04:43 +00:00
Filipe Cabecinhas 2177fc1732 Make the byte-shift SSE intrinsics emit vector shuffles which we know the backend can handle.
Also removed unused builtins.

Original patch by Andrea Di Biagio!

Reviewers: craig.topper, nadav

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D7199

llvm-svn: 228481
2015-02-07 01:37:09 +00:00
David Majnemer 1cf22e690d Headers: Don't use attribute keywords which aren't reserved
Instead of using 'unavailable', use '__unavailable__'

llvm-svn: 228087
2015-02-04 00:26:10 +00:00
Craig Topper 53565c60e7 [X86] Add other flavors of AVX512 cmpps/cmppd intrinsics.
llvm-svn: 227773
2015-02-01 22:27:40 +00:00
Craig Topper 2a898bfc67 [X86] Add the AVX512 exp2a23 intrinsics.
llvm-svn: 227769
2015-02-01 21:34:11 +00:00
Craig Topper da97c20128 [X86] Add all intrinsics for scalar rsqrt28/rcp28 to avx512erintrin.h. Add parentheses around all macro arguments.
llvm-svn: 227722
2015-02-01 10:15:11 +00:00
Craig Topper c4b852a909 [X86] Flesh out more of the avx512erintrin.h file.
llvm-svn: 227719
2015-02-01 08:52:55 +00:00
Craig Topper b01fc317c1 [X86] Use macros in AVX512ER header to allow ICE to be checked for immediate argument.
llvm-svn: 227716
2015-02-01 08:05:12 +00:00
Craig Topper 67826a5883 [X86] Rename _mm512_valign_epi64/32 intrinsics to _mm512_alignr_epi64/32 to match Intel docs. Make immediate argument to them an ICE. Fix mask size for the alignd version.
llvm-svn: 227713
2015-02-01 07:35:40 +00:00
Craig Topper 72c7d51251 [X86] Change rounding parameter of all the AVX512 builtins to an ICE.
llvm-svn: 227712
2015-02-01 07:35:35 +00:00
Craig Topper 9fee8ab4f9 [x86] Remove tab characters from avxintrin.h. NFC.
llvm-svn: 227676
2015-01-31 06:33:59 +00:00
Craig Topper 459554f164 [X86] Make order consistent between 'const' and 'int' in one of the intrinsic header files. NFC
llvm-svn: 227675
2015-01-31 06:31:30 +00:00
Richard Smith 99335be950 Don't use BCPL comments here, in case someone wants to use <stdatomic.h> from C89 mode.
llvm-svn: 227417
2015-01-29 03:34:39 +00:00
Hans Wennborg 2e56d950ff Intrin.h: define _XCR_XFEATURE_ENABLED_MASK
Users expect to be able to use this with _xgetbv.

llvm-svn: 227270
2015-01-27 23:34:35 +00:00
Craig Topper 335e218760 [X86] Add intrinsics for AVX512 128 and 256 bit integer comparison of word and byte vectors.
llvm-svn: 227186
2015-01-27 09:16:29 +00:00
Craig Topper b4789096c0 [X86] Add AVX512 integer comparison intrinsics for word and byte vectors.
llvm-svn: 227079
2015-01-26 09:24:10 +00:00
Craig Topper 2f25a5a875 [X86] Add more of the AVX512 integer comparision intrinsics. This adds 128 and 256 bit vectors of dwords and qwords.
llvm-svn: 227075
2015-01-26 08:11:49 +00:00
Craig Topper 4cac1c2318 [X86] Add AVX512F integer comparision intrinsics to header file.
llvm-svn: 227067
2015-01-25 23:30:07 +00:00
Adam Nemet f893edeaea [AVX512] Add sub-vector FP extracts
Analogous to AVX2, these need to be implemented as macros to properly
propagate the immediate index operand.

Part of <rdar://problem/17688758>

llvm-svn: 226496
2015-01-19 20:12:05 +00:00
Craig Topper f557b09f14 [x86] Mark that the AVX-512 cmpps/cmppd builtins need an ICE for the comparison immediate. This requires converting to a macro in the header file.
llvm-svn: 226421
2015-01-19 01:18:19 +00:00
Adam Nemet c0cff244fc [AVX512] Add intrinsics for masked aligned FP loads and stores
Part of <rdar://problem/17688758>

llvm-svn: 226298
2015-01-16 18:51:50 +00:00
Adam Nemet 63a951eb1c [AVX512] Add FP unpack intrinsics
These are implemented with __builtin_shufflevector just like AVX.

We have some tests on the LLVM side to assert that these shufflevectors do
indeed generate the corresponding unpck instruction.

Part of <rdar://problem/17688758>

llvm-svn: 225922
2015-01-14 01:31:17 +00:00
Ben Langmuir c67a774e17 Add [extern_c] attribute to _Builtin_intrinsics module
This allows users to import this module inside an extern "C" {} block.

llvm-svn: 225835
2015-01-13 21:54:32 +00:00
Chandler Carruth 032d422d2e Effectively revert r151058 which caused Clang's unwind.h to defer to
libunwind in all cases when installed.

At the time, Clang's unwind.h didn't provide huge chunks of the
LSB-specified unwind interface, and was generally too aenemic to use for
real software. However, it has since then become a strict superset of
the APIs provided by libunwind on Linux. Notably, you cannot compile
llgo's libgo library against libunwind, but you can against Clang's
unwind.h. So let's just use our header. =] I've checked pretty
thoroughly for any incompatibilities, and I am not aware of any.

An open question is whether or not we should continue to munge
GNU_SOURCE here. I didn't touch that as it potentially has compatibility
implications on systems I cannot easily test -- Darwin. If a Darwin
maintainer can verify that this is in fact unnecessary and remove it,
cool. Until then, leaving it in makes this change a no-op there, and
only really relevant on Linux systems where it is pretty clearly the
right way to go.

llvm-svn: 224934
2014-12-29 13:29:38 +00:00
Chandler Carruth f3cabbd424 Add a missing declaration to our unwind.h implementation. This is
necessary to be fully compatible with existing software that calls into
the linux unwind code. You can find documentation of this API and why it
exists in the discussion abot NPTL here:
https://gcc.gnu.org/ml/gcc-patches/2003-09/msg00154.html

llvm-svn: 224933
2014-12-29 13:29:36 +00:00
Chandler Carruth 28daca211c [x86] Also add the missing type casts on the returns in the sha
intrinsic header file. Along with r224822, this should restore the build
bots to passing.

llvm-svn: 224883
2014-12-27 11:50:51 +00:00