Commit Graph

92 Commits

Author SHA1 Message Date
Simon Pilgrim 99ed27053d [X86][SSE] Add _mm_set_pd1 (PR32827)
Matches _mm_set_ps1 implementation

llvm-svn: 301637
2017-04-28 10:28:32 +00:00
Ekaterina Romanova 6a5702a093 [DOXYGEN] Improvements to smmintrin.h and emmintrin.h intrinsics.
I made some small changes in smmintrin.h and emmintrin.h intrinsics.
 - changed some regular comments '//' into doxygen-style comments '///' where necessary
 - removed some trailing spaces in doxygen comments.

I got an OK from Eric Christopher to commit doxygen comments without prior code
review upstream.

llvm-svn: 298371
2017-03-21 13:34:06 +00:00
Oren Ben Simhon 259b091669 [X86] DAZ Macros Relocation
The DAZ feature introduces the denormal zero support for x86.
Currently the definitions are located under SSE3 header, however there are some SSE2 targets that support the feature as well.

Differential Revision: https://reviews.llvm.org/D30194

llvm-svn: 296296
2017-02-26 11:58:15 +00:00
Ekaterina Romanova ff266f5236 Added doxygen comments to smmintrin.h's intrinsics.
Note: The doxygen comments are automatically generated based on Sony's intrinsic
s document.

I got an OK from Eric Christopher to commit doxygen comments without prior code
review upstream.

llvm-svn: 295404
2017-02-17 02:49:50 +00:00
Ekaterina Romanova 2e041c9c20 [DOXYGEN] Documentation for the newly added x86 intrinsics.
Added doxygen comments for the newly added intrinsics in avxintrin.h, namely _mm256_cvtsd_f64, _mm256_cvtsi256_si32 and _mm256_cvtss_f32 

Added doxygen comments for the new intrinsics in emmintrin.h, namely _mm_loadu_si64 and _mm_load_sd.

Explicit parameter names were added for _mm_clflush and _mm_setcsr 

The rest of the changes are editorial, removing trailing spaces at the end of the lines.

Differential Revision: https://reviews.llvm.org/D28503

llvm-svn: 291876
2017-01-13 01:14:08 +00:00
Ekaterina Romanova dffe45b3e6 [DOXYGEN] Improved doxygen comments for x86 intrinsics.
Improved doxygen comments for the following intrinsics headers:  __wmmintrin_pclmul.h, bmiintrin.h, emmintrin.h, f16cintrin.h, immintrin.h, mmintrin.h, pmmintrin.h, tmmintrin.h

Added \n commands to insert a line breaks where necessary, since one long line of documentation is nearly unreadable.
Formatted comments to fit into 80 chars.
In some cases added \a command in front of the parameter names to display them in italics.

llvm-svn: 290561
2016-12-27 00:49:38 +00:00
Ekaterina Romanova 0c1c3bbc78 [DOXYGEN] Improved doxygen comments for x86 intrinsics headers.
Tagged instruction names with <c> INSTR_NAME </c> to display them in typewriter font.

In the past, \c command was used, unfortunately it applied to only one word. 
<c> .. </c> has the same meaning, but applies to all words in between the tags.

llvm-svn: 289249
2016-12-09 18:35:50 +00:00
Ekaterina Romanova 797b0ebf2d [DOXYGEN] Improved doxygen comments for emmintrin.h intrinsics.
Tagged parameter names with \a doxygen command to display parameters in italics.
Formatted comments to fit into 80 chars.

llvm-svn: 289116
2016-12-08 22:10:51 +00:00
Ekaterina Romanova 2174b6fe72 Minor changes in x86 intrinsics headers; NFC
I made several changes for consistency with the rest of x86 instrinsics header files. Some of these changes help to render doxygen comments better.

1. avxintrin.h –  Moved the opening bracket on a separate line for several 
                  intrinsics (for consistency with the rest of the intrinsics).

2. emmintrin.h -  Moved the doxygen comment next to the body of the function;
               -  Added braces after extern "C"  even though there is only 
                  one declaration each time

3. xmmintrin.h -  Moved the doxygen comment next to the body of the function;
               -  Added intrinsic prototypes for a couple of macro definitions
                  into the doxygen comment;
               -  Added braces after extern "C"  even though there is only one
                  declaration each time

4. ammintrin.h –  Removed extra line between the doxygen comment and the body
                  of the functions (for consistency with the rest of the files).

Desk reviewed by Paul Robinson.

llvm-svn: 287278
2016-11-17 23:02:00 +00:00
Ekaterina Romanova 64adc38e51 Doxygen comments for avxintrin.h.
Added doxygen comments to avxintrin.h's intrinsics. As of now, around 75% of the
intrinsics in this file are documented here. The patches for the other 25% will be se
nt out later.

Removed extra spaces in emmitrin.h.

Note: The doxygen comments are automatically generated based on Sony's intrinsics document.

I got an OK from Eric Christopher to commit doxygen comments without prior code
review upstream.

llvm-svn: 286336
2016-11-09 03:58:30 +00:00
Ekaterina Romanova 06477bf035 Add more doxygen comments to emmintrin.h's intrinsics.
With this patch, all intrinsics in this file (with an exception of a handful of a recently added ones) will be documented. I will send out a patch for 4 missining intrisics later.

The doxygen comments are automatically generated based on Sony's intrinsics document.

I got an OK from Eric Christopher to commit doxygen comments without prior code
review upstream. This patch was internally reviewed by Yunzhong Gao.

llvm-svn: 284934
2016-10-23 07:30:50 +00:00
Ekaterina Romanova 493091fdef Add more doxygen comments to emmintrin.h's intrinsics.
With this patch, 75% of the intrinsics in this file will be documented now. The patches for the rest of the intrisics in this file will be send out later.

The doxygen comments are automatically generated based on Sony's intrinsics document.

I got an OK from Eric Christopher to commit doxygen comments without prior code review upstream. This patch was internally reviewed by Yunzhong Gao.

llvm-svn: 284754
2016-10-20 17:59:15 +00:00
Albert Gutowski 727ab8a803 Add some MS aliases for existing intrinsics
Reviewers: thakis, compnerd, majnemer, rsmith, rnk

Subscribers: alexshap, cfe-commits

Differential Revision: https://reviews.llvm.org/D24330

llvm-svn: 281540
2016-09-14 21:19:43 +00:00
Albert Gutowski 9918cb6573 Reverse commit 281375 (breaks building Chromium)
llvm-svn: 281399
2016-09-13 21:24:51 +00:00
Albert Gutowski ae3fb3113f Add some MS aliases for existing intrinsics
Reviewers: thakis, compnerd, majnemer, rsmith, rnk

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D24330

llvm-svn: 281375
2016-09-13 19:26:42 +00:00
Craig Topper d0681d528d [X86] Use v2i64 vectors to implement _mm_and/andn/or/xor_pd.
These will be reused when removing some builtins from avx512vldqintrin.h and this will make the tests for that change show a better number of vector elements.

llvm-svn: 280196
2016-08-31 05:38:55 +00:00
Ekaterina Romanova a84c24f39c Add doxygen comments to emmintrin.h's intrinsics.
Only around 50% of the intrinsics in this file are documented now. The patches for the rest of the intrisics in this file will be send out later.

The doxygen comments are automatically generated based on Sony's intrinsics docu
ment.

I got an OK from Eric Christopher to commit doxygen comments without prior code
review upstream. This patch was internally reviewed by Paul Robinson.

llvm-svn: 276499
2016-07-22 23:49:37 +00:00
Simon Pilgrim e3b9ee0645 [X86][SSE] Reimplement SSE fp2si conversion intrinsics instead of using generic IR
D20859 and D20860 attempted to replace the SSE (V)CVTTPS2DQ and VCVTTPD2DQ truncating conversions with generic IR instead.

It turns out that the behaviour of these intrinsics is different enough from generic IR that this will cause problems, INF/NAN/out of range values are guaranteed to result in a 0x80000000 value - which plays havoc with constant folding which converts them to either zero or UNDEF. This is also an issue with the scalar implementations (which were already generic IR and what I was trying to match).

This patch changes both scalar and packed versions back to using x86-specific builtins.

It also deals with the other scalar conversion cases that are runtime rounding mode dependent and can have similar issues with constant folding.

Differential Revision: https://reviews.llvm.org/D22105

llvm-svn: 276102
2016-07-20 10:18:01 +00:00
Craig Topper 2a383c9273 [X86] Use undefined instead of setzero in shufflevector based intrinsics when the second source is unused. Rewrite immediate extractions in shuffle intrinsics to be in ((c >> x) & y) form instead of ((c & z) >> x). This way only x varies between each use instead of having to vary x and z.
llvm-svn: 274525
2016-07-04 22:18:01 +00:00
Asaf Badouh 57819aa185 [X86] add _mm_loadu_si64
Differential Revision: http://reviews.llvm.org/D21504

llvm-svn: 273812
2016-06-26 13:51:54 +00:00
Craig Topper 50e3dfe9d0 [X86] Fix pslldq/psrldq intrinsics to not fail compilation with immediates larger than 16. This was accidentally broken in r272246.
llvm-svn: 273775
2016-06-25 07:31:14 +00:00
Simon Pilgrim beca5f295c [Clang][X86] Convert non-temporal store builtins to generic __builtin_nontemporal_store in headers
We can now use __builtin_nontemporal_store instead of target specific builtins for naturally aligned nontemporal stores which avoids the need for handling in CGBuiltin.cpp

The scalar integer nontemporal (unaligned) store builtins will have to wait as __builtin_nontemporal_store currently assumes natural alignment and doesn't accept the 'packed struct' trick that we use for normal unaligned load/stores.

The nontemporal loads require further backend support before we can safely convert them to __builtin_nontemporal_load

Differential Revision: http://reviews.llvm.org/D21272

llvm-svn: 272540
2016-06-13 09:57:52 +00:00
Craig Topper 2769bb5753 [X86] Handle AVX2 pslldqi and psrldqi intrinsics shufflevector creation directly in the header file instead of in CGBuiltin.cpp. Simplify the sse2 equivalents as well.
llvm-svn: 272246
2016-06-09 05:15:12 +00:00
Craig Topper 3a0c7260f4 [X86] Add void to the argument list of intrinsics that don't take arguments since empty argument list mean something else in C.
llvm-svn: 272244
2016-06-09 05:14:28 +00:00
Craig Topper 6a77b62640 [X86] Use unsigned types for vector arithmetic in intrinsics to avoid undefined behavior for signed integer overflow.
This is really only needed for addition, subtraction, and multiplication, but I did the bitwise ops too for overall consistency. Clang currently doesn't set NSW for signed vector operations so the undefined behavior shouldn't happen today.

llvm-svn: 271778
2016-06-04 05:43:41 +00:00
Simon Pilgrim 00880511b1 [X86][SSE] Replace (V)CVTTPS2DQ and VCVTTPD2DQ truncating (round to zero) f32/f64 to i32 with generic IR (clang)
The 'cvtt' truncation (round to zero) conversions can be safely represented as generic __builtin_convertvector (fptosi) calls instead of x86 intrinsics. We already do this (implicitly) for the scalar equivalents.

Note: I looked at updating _mm_cvttpd_epi32 as well but this still requires a lot more backend work to correctly lower (both for debug and optimized builds).

Differential Revision: http://reviews.llvm.org/D20859

llvm-svn: 271436
2016-06-01 21:46:51 +00:00
Simon Pilgrim 645e1ad33a [X86][SSE] _mm_store1_ps/_mm_store1_pd should require an aligned pointer
According to the gcc headers, intel intrinsics docs and msdn codegen the _mm_store1_pd (and its _mm_store_pd1 equivalent) should use an aligned pointer - the clang headers are the only implementation I can find that assume non-aligned stores (by storing with _mm_storeu_pd).

Additionally, according to the intel intrinsics docs and msdn codegen the _mm_store1_ps (_mm_store_ps1) requires a similarly aligned pointer.

This patch raises the alignment requirements to match the other implementations by calling _mm_store_ps/_mm_store_pd instead.

I've also added the missing _mm_store_pd1 intrinsic (which maps to _mm_store1_pd like _mm_store_ps1 does to _mm_store1_ps).

As a followup I'll update the llvm fast-isel tests to match this codegen.

Differential Revision: http://reviews.llvm.org/D20617

llvm-svn: 271218
2016-05-30 17:55:25 +00:00
Craig Topper 09175dab31 [X86] Replace unaligned store builtins in SSE/AVX intrinsic files with code that will compile to a native unaligned store. Remove the builtins since they are no longer used.
Intrinsics will be removed from llvm in a future commit.

llvm-svn: 271214
2016-05-30 17:10:30 +00:00
Simon Pilgrim 6d1a0c4c75 [X86][SSE] Make unsigned integer vector types generally available
As discussed on http://reviews.llvm.org/D20684, move the unsigned integer vector types used for zero extension to make them available for general use.

llvm-svn: 271187
2016-05-29 18:49:08 +00:00
Simon Pilgrim 90770c7c76 [X86][SSE] Replace lossless i32/f32 to f64 conversion intrinsics with generic IR
Both the (V)CVTDQ2PD(Y) (i32 to f64) and (V)CVTPS2PD(Y) (f32 to f64) conversion instructions are lossless and can be safely represented as generic __builtin_convertvector calls instead of x86 intrinsics without affecting final codegen.

This patch removes the clang builtins and their use in the sse2/avx headers - a future patch will deal with removing the llvm intrinsics, but that will require a bit more work.

Differential Revision: http://reviews.llvm.org/D20528

llvm-svn: 270499
2016-05-23 22:13:02 +00:00
Craig Topper 1aa231e3aa [X86] Add typecasts to remove most assumptions about what __m128i/__m256i is defined as. Add similar typecasts for the fp types as well.
llvm-svn: 269632
2016-05-16 06:38:42 +00:00
Ekaterina Romanova f2ed62027d Add doxygen comments to emmintrin.h's intrinsics. Only around 25% of the intrinsics in this file are documented now. The patches for the rest of the intrisics in this file will be send out later.
The doxygen comments are automatically generated based on Sony's intrinsics document.

I got an OK from Eric Christopher to commit doxygen comments without prior code review upstream. This patch was internally reviewed by Paul Robinson.

llvm-svn: 265844
2016-04-08 20:45:48 +00:00
Craig Topper fb79b5f273 [X86] Add 'pause' builtin that's already in llvm and use it instead of inline assembly to implement _mm_pause.
llvm-svn: 252712
2015-11-11 08:13:33 +00:00
Craig Topper a5455524c2 [X86] Use __builtin_ia32_paddq and __builtin_ia32_psubq to implement a couple intrinsics that were supposed to operate on MMX registers. Otherwise we end up operating on GPRs. Throw in a test for _mm_mul_su32 while I was there.
llvm-svn: 252711
2015-11-11 08:00:41 +00:00
Craig Topper d619eaaae4 [X86] Add missing typecasts in intrinsic macros. This should make them more robust against inputs that aren't already the right type.
llvm-svn: 252700
2015-11-11 03:47:10 +00:00
Craig Topper fd778eebac [X86] Use setzero instead of set1(0) in a few places in intrinsic headers.
llvm-svn: 252587
2015-11-10 05:08:08 +00:00
Chandler Carruth cbe6411401 Fix the SSE4 byte sign extension in a cleaner way, and more thoroughly
test that our intrinsics behave the same under -fsigned-char and
-funsigned-char.

This further testing uncovered that AVX-2 has a broken cmpgt for 8-bit
elements, and has for a long time. This is fixed in the same way as
SSE4 handles the case.

The other ISA extensions currently work correctly because they use
specific instruction intrinsics. As soon as they are rewritten in terms
of generic IR, they will need to add these special casts. I've added the
necessary testing to catch this however, so we shouldn't have to chase
it down again.

I considered changing the core typedef to be signed, but that seems like
a bad idea. Notably, it would be an ABI break if anyone is reaching into
the innards of the intrinsic headers and passing __v16qi on an API
boundary. I can't be completely confident that this wouldn't happen due
to a macro expanding in a lambda, etc., so it seems much better to leave
it alone. It also matches GCC's behavior exactly.

A fun side note is that for both GCC and Clang, -funsigned-char really
does change the semantics of __v16qi. To observe this, consider:

  % cat x.cc
  #include <smmintrin.h>
  #include <iostream>

  int main() {
    __v16qi a = { 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
    __v16qi b = _mm_set1_epi8(-1);
    std::cout << (int)(a / b)[0] << ", " << (int)(a / b)[1] << '\n';
  }
  % clang++ -o x x.cc && ./x
  -1, 1
  % clang++ -funsigned-char -o x x.cc && ./x
  0, 1

However, while this may be surprising, both Clang and GCC agree.

Differential Revision: http://reviews.llvm.org/D13324

llvm-svn: 249097
2015-10-01 23:40:12 +00:00
Michael Kuperstein a10dff946e [X86] Make f16c intrinsics accessible through emmintrin.h, per Intel docs
Differential Revision: http://reviews.llvm.org/D13015

llvm-svn: 248156
2015-09-21 13:34:47 +00:00
Michael Kuperstein 5c2cb0eee2 [X86] Fix some non-reserved parameter names in intrinsic headers
Differential Revision: http://reviews.llvm.org/D13009

llvm-svn: 248150
2015-09-21 11:45:27 +00:00
Simon Pilgrim 5aba9925c0 [X86][SSE] Add _mm_undefined_* intrinsics
Added missing SSE/AVX 'undefined' intrinsics (PR24040):

_mm_undefined_pd, _mm_undefined_ps + _mm_undefined_si128
_mm256_undefined_pd, _mm256_undefined_ps + _mm256_undefined_si256
_mm512_undefined, _mm512_undefined_ps, _mm512_undefined_pd + _mm512_undefined_epi32

Added builtin intrinsicss:

__builtin_ia32_undef128, __builtin_ia32_undef256 + __builtin_ia32_undef512

Differential Revision: http://reviews.llvm.org/D12052

llvm-svn: 246083
2015-08-26 21:17:12 +00:00
Michael Kuperstein e45af54cdb [X86] Rename DEFAULT_FN_ATTR macro to __DEFAULT_FN_ATTR
llvm-svn: 241065
2015-06-30 13:36:19 +00:00
Eric Christopher 9fc7fb274e Update the intel intrinsic headers to use the target attribute support.
This involved removing the conditional inclusion and replacing them
with target attributes matching the original conditional inclusion
and checks. The testcase update removes the macro checks for each
file and replaces them with usage of the __target__ attribute, e.g.:

int __attribute__((__target__(("sse3")))) foo(int a) {
  _mm_mwait(0, 0);
  return 4;
}

This usage does require the enclosing function have the requisite
__target__ attribute for inlining and code generation - also for
any macro intrinsic uses in the enclosing function. There's no change
for existing uses of the intrinsic headers.

llvm-svn: 239883
2015-06-17 07:09:32 +00:00
Eric Christopher 4d185168e9 Use a define for per-file function attributes for the Intel intrinsic headers.
This is a precursor to changing them to use the new target attribute
code.

llvm-svn: 239882
2015-06-17 07:09:20 +00:00
Craig Topper a462482d98 [X86] Add _mm_bslli_si128 and _mm_bsrli_si128 as aliases of _mm_slli_si128 and _mm_srli_si128. This matches Intel documentation and gcc.
llvm-svn: 229066
2015-02-13 06:04:45 +00:00
Craig Topper 51e47418d4 [X86] Simplify some code and remove some -Wshadow disables from intrinsic header.
llvm-svn: 229065
2015-02-13 06:04:43 +00:00
Filipe Cabecinhas 2177fc1732 Make the byte-shift SSE intrinsics emit vector shuffles which we know the backend can handle.
Also removed unused builtins.

Original patch by Andrea Di Biagio!

Reviewers: craig.topper, nadav

Subscribers: cfe-commits

Differential Revision: http://reviews.llvm.org/D7199

llvm-svn: 228481
2015-02-07 01:37:09 +00:00
David Majnemer 1cf22e690d Headers: Don't use attribute keywords which aren't reserved
Instead of using 'unavailable', use '__unavailable__'

llvm-svn: 228087
2015-02-04 00:26:10 +00:00
Craig Topper 2094d8fe88 [x86] Add the (v)cmpps/pd/ss/sd builtins to match gcc. Use them in the sse intrinsic files.
This still lower to the same intrinsics as before.

This is preparation for bounds checking the immediate on the avx version of the builtin so we don't pass illegal immediates into the backend. Since SSE uses a smaller size immediate its not possible to bounds check when using a shared builtin. Rather than creating a clang specific builtin for the different immediate, I decided (after consulting with Chandler) that it was better to match gcc.

llvm-svn: 224879
2014-12-27 06:59:57 +00:00
Alp Toker d480b1bf34 Fix a SSE2 intrinsics typo
Full discourse at:

  http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20131104/092514.html
  http://lists.cs.uiuc.edu/pipermail/llvmdev/2013-November/068124.html

Patch by Dimitry Andric and Alexey Dokuchaev!

llvm-svn: 195558
2013-11-23 22:11:57 +00:00
Manman Ren be38b9e15f _mm_extract_epi16: use "& 7" when index is out of bound.
This is in line with implementation of _mm_extract_pi16.
rdar://15250497

llvm-svn: 193187
2013-10-22 19:24:42 +00:00