maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/clang/lib/Headers/immintrin.h

509 lines
15 KiB
C
Raw Normal View History

Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
/*===---- immintrin.h - Intel intrinsics -----------------------------------===
*
Move the builtin headers to use the new license file header. Summary: These all had somewhat custom file headers with different text from the ones I searched for previously, and so I missed them. Thanks to Hal and Kristina and others who prompted me to fix this, and sorry it took so long. Reviewers: hfinkel Subscribers: mcrosier, javed.absar, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60406 llvm-svn: 357941
2019-04-09 04:51:30 +08:00
* Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
* See https://llvm.org/LICENSE.txt for license information.
* SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
*
*===-----------------------------------------------------------------------===
*/
#ifndef __IMMINTRIN_H
#define __IMMINTRIN_H
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__MMX__)
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#include <mmintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__SSE__)
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#include <xmmintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__SSE2__)
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#include <emmintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__SSE3__)
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#include <pmmintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__SSSE3__)
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#include <tmmintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__SSE4_2__) || defined(__SSE4_1__))
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#include <smmintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AES__) || defined(__PCLMUL__))
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#include <wmmintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
[Clang][Feature] Adding CLFLUSHOPT feature and intrinsic to clang Differential Revision: http://reviews.llvm.org/D21792 llvm-svn: 274559
2016-07-05 23:56:03 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__CLFLUSHOPT__)
#include <clflushoptintrin.h>
#endif
[X86] Add CLWB intrinsic. clang part Reviewers: RKSimon, zvi, igorb Reviewed By: RKSimon Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D38781 llvm-svn: 315607
2017-10-13 02:57:15 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__CLWB__)
#include <clwbintrin.h>
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX__)
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#include <avxintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX2__)
Begin adding AVX2 intrinsics. Necessitated increasing the number of bits used to store builtinID when serializing identifier table. llvm-svn: 146855
2011-12-19 13:04:33 +08:00
#include <avx2intrin.h>
[X86] Add two missing #endif directives to immintrin.h that should have been in r333014. llvm-svn: 333023
2018-05-23 04:33:04 +08:00
#endif
Begin adding AVX2 intrinsics. Necessitated increasing the number of bits used to store builtinID when serializing identifier table. llvm-svn: 146855
2011-12-19 13:04:33 +08:00
[X86] Move 128-bit f16c intrinsics to __emmintrin_f16c.h include from emmintrin.h. Move 256-bit f16c intrinsics back to f16cintrin.h Intel documents the 128-bit versions as being in emmintrin.h and the 256-bit version as being in immintrin.h. This patch makes a new __emmtrin_f16c.h to hold the 128-bit versions to be included from emmintrin.h. And makes the existing f16cintrin.h contain the 256-bit versions and include it from immintrin.h with an error if its included directly. Differential Revision: https://reviews.llvm.org/D47174 llvm-svn: 333014
2018-05-23 02:54:19 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__F16C__)
#include <f16cintrin.h>
[X86] Add two missing #endif directives to immintrin.h that should have been in r333014. llvm-svn: 333023
2018-05-23 04:33:04 +08:00
#endif
Move _mm256_cvtps_ph and _mm256_cvtph_ps to immintrin.h. This more closely matches their locations as described by Intel documentation, and lets us remove a pair of redundant typedefs. Differential Revision: http://reviews.llvm.org/D15127 llvm-svn: 254528
2015-12-03 02:41:52 +08:00
[x86][icelake][vpclmulqdq] added vpclmulqdq feature recognition added intrinsics support for vpclmulqdq instructions _mm256_clmulepi64_epi128 _mm512_clmulepi64_epi128 matching a similar work on the backend (D40101) Differential Revision: https://reviews.llvm.org/D41573 llvm-svn: 321480
2017-12-27 17:00:31 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__VPCLMULQDQ__)
#include <vpclmulqdqintrin.h>
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__BMI__)
Add intrinsics for lzcnt and tzcnt instructions. llvm-svn: 147263
2011-12-25 14:25:37 +08:00
#include <bmiintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add intrinsics for lzcnt and tzcnt instructions. llvm-svn: 147263
2011-12-25 14:25:37 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__BMI2__)
Add BMI2 intrinsics. llvm-svn: 147275
2011-12-26 10:31:10 +08:00
#include <bmi2intrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add BMI2 intrinsics. llvm-svn: 147275
2011-12-26 10:31:10 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__LZCNT__)
Add intrinsics for lzcnt and tzcnt instructions. llvm-svn: 147263
2011-12-25 14:25:37 +08:00
#include <lzcntintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add intrinsics for lzcnt and tzcnt instructions. llvm-svn: 147263
2011-12-25 14:25:37 +08:00
[X86] Move all Intel defined intrinsic includes into immintrin.h This matches the Intel documentation which shows them available by importing immintrin.h. x86intrin.h also includes immintrin.h so anyone including x86intrin.h will still get them. This is different than gcc, but I don't think we were a perfect match there already. I'm unclear what gcc's policy is about how they choose which to add things to. Differential Revision: https://reviews.llvm.org/D47182 llvm-svn: 333110
2018-05-24 02:32:58 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__POPCNT__)
#include <popcntintrin.h>
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__FMA__)
Add fma3 intrinsic header file. llvm-svn: 157913
2012-06-04 11:42:47 +08:00
#include <fmaintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add fma3 intrinsic header file. llvm-svn: 157913
2012-06-04 11:42:47 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512F__)
AVX-512: Added intrinsics to clang. The set is small, that what I have right now. Everybody is welcome to add more. llvm-svn: 213641
2014-07-22 19:31:39 +08:00
#include <avx512fintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
AVX-512: Added intrinsics to clang. The set is small, that what I have right now. Everybody is welcome to add more. llvm-svn: 213641
2014-07-22 19:31:39 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512VL__)
[AVX512] Added VPCMPEQ intrinisics to headers. Added tests. Patch by Maxim Blumenthal <maxim.blumenthal@intel.com> llvm-svn: 219319
2014-10-09 01:18:13 +08:00
#include <avx512vlintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[AVX512] Added VPCMPEQ intrinisics to headers. Added tests. Patch by Maxim Blumenthal <maxim.blumenthal@intel.com> llvm-svn: 219319
2014-10-09 01:18:13 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512BW__)
[AVX512] Added VPCMPEQ intrinisics to headers. Added tests. Patch by Maxim Blumenthal <maxim.blumenthal@intel.com> llvm-svn: 219319
2014-10-09 01:18:13 +08:00
#include <avx512bwintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[AVX512] Added VPCMPEQ intrinisics to headers. Added tests. Patch by Maxim Blumenthal <maxim.blumenthal@intel.com> llvm-svn: 219319
2014-10-09 01:18:13 +08:00
[x86][icelake][bitalg] added bitalg feature recognition added intrinsics support for bitalg instructions _mm512_popcnt_epi16 _mm512_mask_popcnt_epi16 _mm512_maskz_popcnt_epi16 _mm512_popcnt_epi8 _mm512_mask_popcnt_epi8 _mm512_maskz_popcnt_epi8 _mm512_mask_bitshuffle_epi64_mask _mm512_bitshuffle_epi64_mask _mm256_popcnt_epi16 _mm256_mask_popcnt_epi16 _mm256_maskz_popcnt_epi16 _mm128_popcnt_epi16 _mm128_mask_popcnt_epi16 _mm128_maskz_popcnt_epi16 _mm256_popcnt_epi8 _mm256_mask_popcnt_epi8 _mm256_maskz_popcnt_epi8 _mm128_popcnt_epi8 _mm128_mask_popcnt_epi8 _mm128_maskz_popcnt_epi8 _mm256_mask_bitshuffle_epi32_mask _mm256_bitshuffle_epi32_mask _mm128_mask_bitshuffle_epi16_mask _mm128_bitshuffle_epi16_mask matching a similar work on the backend (D40222) Differential Revision: https://reviews.llvm.org/D41564 llvm-svn: 321483
2017-12-27 18:01:00 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512BITALG__)
#include <avx512bitalgintrin.h>
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512CD__)
[x86][AVX512CD] Add conflict and lzcnt intrinsics in their 512bit versions include tests review http://reviews.llvm.org/D10795 llvm-svn: 240941
2015-06-29 20:51:53 +08:00
#include <avx512cdintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[x86][AVX512CD] Add conflict and lzcnt intrinsics in their 512bit versions include tests review http://reviews.llvm.org/D10795 llvm-svn: 240941
2015-06-29 20:51:53 +08:00
[X86] Adding avx512_vpopcntdq feature set and its intrinsics AVX512_VPOPCNTDQ is a new feature set that was published by Intel. The patch represents the Clang side of the addition of six intrinsics for two new machine instructions (vpopcntd and vpopcntq). It also includes the addition of the new feature set. Differential Revision: https://reviews.llvm.org/D33170 llvm-svn: 303857
2017-05-25 21:44:11 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512VPOPCNTDQ__)
#include <avx512vpopcntdqintrin.h>
#endif
[X86] Add builtins and tests for 128 and 256 bit vpopcntdq. llvm-svn: 320915
2017-12-16 14:02:31 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VL__) && defined(__AVX512VPOPCNTDQ__))
#include <avx512vpopcntdqvlintrin.h>
#endif
[x86][icelake][vnni] added vnni feature recognition added intrinsics support for VNNI instructions _mm256_mask_dpbusd_epi32 _mm256_maskz_dpbusd_epi32 _mm256_dpbusd_epi32 _mm256_mask_dpbusds_epi32 _mm256_maskz_dpbusds_epi32 _mm256_dpbusds_epi32 _mm256_mask_dpwssd_epi32 _mm256_maskz_dpwssd_epi32 _mm256_dpwssd_epi32 _mm256_mask_dpwssds_epi32 _mm256_maskz_dpwssds_epi32 _mm256_dpwssds_epi32 _mm128_mask_dpbusd_epi32 _mm128_maskz_dpbusd_epi32 _mm128_dpbusd_epi32 _mm128_mask_dpbusds_epi32 _mm128_maskz_dpbusds_epi32 _mm128_dpbusds_epi32 _mm128_mask_dpwssd_epi32 _mm128_maskz_dpwssd_epi32 _mm128_dpwssd_epi32 _mm128_mask_dpwssds_epi32 _mm128_maskz_dpwssds_epi32 _mm128_dpwssds_epi32 _mm512_mask_dpbusd_epi32 _mm512_maskz_dpbusd_epi32 _mm512_dpbusd_epi32 _mm512_mask_dpbusds_epi32 _mm512_maskz_dpbusds_epi32 _mm512_dpbusds_epi32 _mm512_mask_dpwssd_epi32 _mm512_maskz_dpwssd_epi32 _mm512_dpwssd_epi32 _mm512_mask_dpwssds_epi32 _mm512_maskz_dpwssds_epi32 _mm512_dpwssds_epi32 matching a similar work on the backend (D40208) Differential Revision: https://reviews.llvm.org/D41558 llvm-svn: 321484
2017-12-27 18:37:51 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512VNNI__)
#include <avx512vnniintrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VL__) && defined(__AVX512VNNI__))
#include <avx512vlvnniintrin.h>
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512DQ__)
AVX-512: Added AVX-512 intrinsics and tests by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236218
2015-04-30 17:24:29 +08:00
#include <avx512dqintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
AVX-512: Added AVX-512 intrinsics and tests by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236218
2015-04-30 17:24:29 +08:00
[x86][icelake][bitalg] added bitalg feature recognition added intrinsics support for bitalg instructions _mm512_popcnt_epi16 _mm512_mask_popcnt_epi16 _mm512_maskz_popcnt_epi16 _mm512_popcnt_epi8 _mm512_mask_popcnt_epi8 _mm512_maskz_popcnt_epi8 _mm512_mask_bitshuffle_epi64_mask _mm512_bitshuffle_epi64_mask _mm256_popcnt_epi16 _mm256_mask_popcnt_epi16 _mm256_maskz_popcnt_epi16 _mm128_popcnt_epi16 _mm128_mask_popcnt_epi16 _mm128_maskz_popcnt_epi16 _mm256_popcnt_epi8 _mm256_mask_popcnt_epi8 _mm256_maskz_popcnt_epi8 _mm128_popcnt_epi8 _mm128_mask_popcnt_epi8 _mm128_maskz_popcnt_epi8 _mm256_mask_bitshuffle_epi32_mask _mm256_bitshuffle_epi32_mask _mm128_mask_bitshuffle_epi16_mask _mm128_bitshuffle_epi16_mask matching a similar work on the backend (D40222) Differential Revision: https://reviews.llvm.org/D41564 llvm-svn: 321483
2017-12-27 18:01:00 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VL__) && defined(__AVX512BITALG__))
#include <avx512vlbitalgintrin.h>
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VL__) && defined(__AVX512BW__))
[AVX512] Added VPCMPEQ intrinisics to headers. Added tests. Patch by Maxim Blumenthal <maxim.blumenthal@intel.com> llvm-svn: 219319
2014-10-09 01:18:13 +08:00
#include <avx512vlbwintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[AVX512] Added VPCMPEQ intrinisics to headers. Added tests. Patch by Maxim Blumenthal <maxim.blumenthal@intel.com> llvm-svn: 219319
2014-10-09 01:18:13 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VL__) && defined(__AVX512CD__))
[Clang][BuiltIn][AVX512] Adding intrinsics without mask for VBROADCAST and VPBROADCAST instruction set . Differential Revision: http://reviews.llvm.org/D19196 llvm-svn: 267696
2016-04-27 19:43:14 +08:00
#include <avx512vlcdintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[Clang][BuiltIn][AVX512] Adding intrinsics without mask for VBROADCAST and VPBROADCAST instruction set . Differential Revision: http://reviews.llvm.org/D19196 llvm-svn: 267696
2016-04-27 19:43:14 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VL__) && defined(__AVX512DQ__))
AVX-512: Added AVX-512 intrinsics and tests by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236218
2015-04-30 17:24:29 +08:00
#include <avx512vldqintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
AVX-512: Added AVX-512 intrinsics and tests by Asaf Badouh (asaf.badouh@intel.com) llvm-svn: 236218
2015-04-30 17:24:29 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512ER__)
AVX-512: Added intrinsics to clang. The set is small, that what I have right now. Everybody is welcome to add more. llvm-svn: 213641
2014-07-22 19:31:39 +08:00
#include <avx512erintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
AVX-512: Added intrinsics to clang. The set is small, that what I have right now. Everybody is welcome to add more. llvm-svn: 213641
2014-07-22 19:31:39 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512IFMA__)
[CLANG][AVX512][BUILTIN] Adding new feature flag header file and new builtin vpmadd52{h|l}uq{128|256|512}{mask|maskz} Differential Revision: http://reviews.llvm.org/D17915 llvm-svn: 262820
2016-03-07 17:55:55 +08:00
#include <avx512ifmaintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[CLANG][AVX512][BUILTIN] Adding new feature flag header file and new builtin vpmadd52{h|l}uq{128|256|512}{mask|maskz} Differential Revision: http://reviews.llvm.org/D17915 llvm-svn: 262820
2016-03-07 17:55:55 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512IFMA__) && defined(__AVX512VL__))
[CLANG][AVX512][BUILTIN] Adding new feature flag header file and new builtin vpmadd52{h|l}uq{128|256|512}{mask|maskz} Differential Revision: http://reviews.llvm.org/D17915 llvm-svn: 262820
2016-03-07 17:55:55 +08:00
#include <avx512ifmavlintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[CLANG][AVX512][BUILTIN] Adding new feature flag header file and new builtin vpmadd52{h|l}uq{128|256|512}{mask|maskz} Differential Revision: http://reviews.llvm.org/D17915 llvm-svn: 262820
2016-03-07 17:55:55 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512VBMI__)
[CLANG][AVX512][BUILTIN] Adding new feature flag headed files and new BUILTIN vpermi2varq{i|t}{128|256|512}{mask|maskz} Differential Revision: http://reviews.llvm.org/D17917 llvm-svn: 262834
2016-03-08 01:04:11 +08:00
#include <avx512vbmiintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[CLANG][AVX512][BUILTIN] Adding new feature flag headed files and new BUILTIN vpermi2varq{i|t}{128|256|512}{mask|maskz} Differential Revision: http://reviews.llvm.org/D17917 llvm-svn: 262834
2016-03-08 01:04:11 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VBMI__) && defined(__AVX512VL__))
[CLANG][AVX512][BUILTIN] Adding new feature flag headed files and new BUILTIN vpermi2varq{i|t}{128|256|512}{mask|maskz} Differential Revision: http://reviews.llvm.org/D17917 llvm-svn: 262834
2016-03-08 01:04:11 +08:00
#include <avx512vbmivlintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[CLANG][AVX512][BUILTIN] Adding new feature flag headed files and new BUILTIN vpermi2varq{i|t}{128|256|512}{mask|maskz} Differential Revision: http://reviews.llvm.org/D17917 llvm-svn: 262834
2016-03-08 01:04:11 +08:00
[x86][icelake][vbmi2] added vbmi2 feature recognition added intrinsics support for vbmi2 instructions _mm[128,256,512]_mask[z]_compress_epi[16,32] _mm[128,256,512]_mask_compressstoreu_epi[16,32] _mm[128,256,512]_mask[z]_expand_epi[16,32] _mm[128,256,512]_mask[z]_expandloadu_epi[16,32] _mm[128,256,512]_mask[z]_sh[l,r]di_epi[16,32,64] _mm[128,256,512]_mask_sh[l,r]dv_epi[16,32,64] matching a similar work on the backend (D40206) Differential Revision: https://reviews.llvm.org/D41557 llvm-svn: 321487
2017-12-27 19:25:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512VBMI2__)
#include <avx512vbmi2intrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VBMI2__) && defined(__AVX512VL__))
#include <avx512vlvbmi2intrin.h>
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512PF__)
[Clang][AVX512][BuiltIn] Adding intrinsics of VGATHER{DPS|DPD} , VPGATHER{QD|QQ|DD|DQ} and VGATHERPF{0|1}{DPS|QPS|DPD|QPD} instruction set . Differential Revision: http://reviews.llvm.org/D19224 llvm-svn: 266983
2016-04-21 20:47:27 +08:00
#include <avx512pfintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[Clang][AVX512][BuiltIn] Adding intrinsics of VGATHER{DPS|DPD} , VPGATHER{QD|QQ|DD|DQ} and VGATHERPF{0|1}{DPS|QPS|DPD|QPD} instruction set . Differential Revision: http://reviews.llvm.org/D19224 llvm-svn: 266983
2016-04-21 20:47:27 +08:00
Enable intrinsics of AVX512_BF16, which are supported for BFLOAT16 in Cooper Lake Summary: 1. Enable infrastructure of AVX512_BF16, which is supported for BFLOAT16 in Cooper Lake; 2. Enable intrinsics for VCVTNE2PS2BF16, VCVTNEPS2BF16 and DPBF16PS instructions, which are Vector Neural Network Instructions supporting BFLOAT16 inputs and conversion instructions from IEEE single precision. For more details about BF16 intrinsic, please refer to the latest ISE document: https://software.intel.com/en-us/download/intel-architecture-instruction-set-extensions-programming-reference Patch by LiuTianle Reviewers: craig.topper, smaslov, LuoYuanke, wxiao3, annita.zhang, spatel, RKSimon Reviewed By: craig.topper Subscribers: mgorny, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D60552 llvm-svn: 360018
2019-05-06 16:25:11 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__AVX512BF16__)
#include <avx512bf16intrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || \
(defined(__AVX512VL__) && defined(__AVX512BF16__))
#include <avx512vlbf16intrin.h>
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__PKU__)
[X86][PKU] add clang intrinsic for {RD|WR}PKRU Differential Revision: http://reviews.llvm.org/D15837 llvm-svn: 256672
2015-12-31 22:14:07 +08:00
#include <pkuintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[X86][PKU] add clang intrinsic for {RD|WR}PKRU Differential Revision: http://reviews.llvm.org/D15837 llvm-svn: 256672
2015-12-31 22:14:07 +08:00
[x86][icelake][vaes] added vaes feature recognition added intrinsics support for vaes instructions, matching a similar work on the backend (D40078) _mm256_aesenc_epi128 _mm512_aesenc_epi128 _mm256_aesenclast_epi128 _mm512_aesenclast_epi128 _mm256_aesdec_epi128 _mm512_aesdec_epi128 _mm256_aesdeclast_epi128 _mm512_aesdeclast_epi128 llvm-svn: 321474
2017-12-27 16:16:54 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__VAES__)
#include <vaesintrin.h>
#endif
[x86][icelake][gfni] added gfni feature recognition added intrinsics support for gfni instructions _mm_gf2p8affineinv_epi64_epi8 _mm_mask_gf2p8affineinv_epi64_epi8 _mm_maskz_gf2p8affineinv_epi64_epi8 _mm256_gf2p8affineinv_epi64_epi8 _mm256_mask_gf2p8affineinv_epi64_epi8 _mm256_maskz_gf2p8affineinv_epi64_epi8 _mm512_gf2p8affineinv_epi64_epi8 _mm512_mask_gf2p8affineinv_epi64_epi8 _mm512_maskz_gf2p8affineinv_epi64_epi8 _mm_gf2p8affine_epi64_epi8 _mm_mask_gf2p8affine_epi64_epi8 _mm_maskz_gf2p8affine_epi64_epi8 _mm256_gf2p8affine_epi64_epi8 _mm256_mask_gf2p8affine_epi64_epi8 _mm256_maskz_gf2p8affine_epi64_epi8 _mm512_gf2p8affine_epi64_epi8 _mm512_mask_gf2p8affine_epi64_epi8 _mm512_maskz_gf2p8affine_epi64_epi8 _mm_gf2p8mul_epi8 _mm_mask_gf2p8mul_epi8 _mm_maskz_gf2p8mul_epi8 _mm256_gf2p8mul_epi8 _mm256_mask_gf2p8mul_epi8 _mm256_maskz_gf2p8mul_epi8 _mm512_gf2p8mul_epi8 _mm512_mask_gf2p8mul_epi8 _mm512_maskz_gf2p8mul_epi8 matching a similar work on the backend (D40373) Differential Revision: https://reviews.llvm.org/D41582 llvm-svn: 321477
2017-12-27 16:37:47 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__GFNI__)
#include <gfniintrin.h>
#endif
[X86] Add rdpid command line option and intrinsics. Summary: This patch adds -mrdpid/-mno-rdpid and the rdpid intrinsic. The corresponding LLVM commit has already been made. Reviewers: RKSimon, spatel, zvi, AndreiGrischenko Reviewed By: RKSimon Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D42272 llvm-svn: 323047
2018-01-21 02:36:52 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__RDPID__)
Remove \brief commands from doxygen comments. This is similar to the LLVM change https://reviews.llvm.org/D46290. We've been running doxygen with the autobrief option for a couple of years now. This makes the \brief markers into our comments redundant. Since they are a visual distraction and we don't want to encourage more \brief markers in new code either, this patch removes them all. Patch produced by for i in $(git grep -l '\@brief'); do perl -pi -e 's/\@brief //g' $i & done for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done Differential Revision: https://reviews.llvm.org/D46320 llvm-svn: 331834
2018-05-09 09:00:01 +08:00
/// Returns the value of the IA32_TSC_AUX MSR (0xc0000103).
[X86] Add rdpid command line option and intrinsics. Summary: This patch adds -mrdpid/-mno-rdpid and the rdpid intrinsic. The corresponding LLVM commit has already been made. Reviewers: RKSimon, spatel, zvi, AndreiGrischenko Reviewed By: RKSimon Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D42272 llvm-svn: 323047
2018-01-21 02:36:52 +08:00
///
/// \headerfile <immintrin.h>
///
/// This intrinsic corresponds to the <c> RDPID </c> instruction.
static __inline__ unsigned int __attribute__((__always_inline__, __nodebug__, __target__("rdpid")))
_rdpid_u32(void) {
return __builtin_ia32_rdpid();
}
#endif // __RDPID__
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__RDRND__)
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ int __attribute__((__always_inline__, __nodebug__, __target__("rdrnd")))
Add _rdrand{16,32,64}_step intrinsics to immintrin.h llvm-svn: 160118
2012-07-12 17:33:03 +08:00
_rdrand16_step(unsigned short *__p)
{
return __builtin_ia32_rdrand16_step(__p);
}
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ int __attribute__((__always_inline__, __nodebug__, __target__("rdrnd")))
Add _rdrand{16,32,64}_step intrinsics to immintrin.h llvm-svn: 160118
2012-07-12 17:33:03 +08:00
_rdrand32_step(unsigned int *__p)
{
return __builtin_ia32_rdrand32_step(__p);
}
X86 Intrinsics: _bit_scan_forward should not be under #ifdef __RDRND__ Summary: The _bit_scan_forward and _bit_scan_reverse intrinsics were accidentally masked under the preprocessor checks that prune intrinsics definitions for the benefit of faster compile-time on Windows. This patch moves the definitons out of that region. Fixes pr33722 Reviewers: craig.topper, aaboud, thakis Reviewed By: craig.topper Subscribers: cfe-commits Differential Revision: https://reviews.llvm.org/D35184 llvm-svn: 307524
2017-07-10 15:13:56 +08:00
#ifdef __x86_64__
static __inline__ int __attribute__((__always_inline__, __nodebug__, __target__("rdrnd")))
_rdrand64_step(unsigned long long *__p)
{
return __builtin_ia32_rdrand64_step(__p);
}
#endif
#endif /* __RDRND__ */
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__FSGSBASE__)
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
#ifdef __x86_64__
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ unsigned int __attribute__((__always_inline__, __nodebug__, __target__("fsgsbase")))
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
_readfsbase_u32(void)
{
return __builtin_ia32_rdfsbase32();
}
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ unsigned long long __attribute__((__always_inline__, __nodebug__, __target__("fsgsbase")))
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
_readfsbase_u64(void)
{
return __builtin_ia32_rdfsbase64();
}
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ unsigned int __attribute__((__always_inline__, __nodebug__, __target__("fsgsbase")))
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
_readgsbase_u32(void)
{
return __builtin_ia32_rdgsbase32();
}
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ unsigned long long __attribute__((__always_inline__, __nodebug__, __target__("fsgsbase")))
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
_readgsbase_u64(void)
{
return __builtin_ia32_rdgsbase64();
}
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ void __attribute__((__always_inline__, __nodebug__, __target__("fsgsbase")))
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
_writefsbase_u32(unsigned int __V)
{
[X86] Remove 'return' from a bunch of intrinsics that return void and use a builtin that returns void. Found by running the intrinsic headers through -pedantic -ansi. llvm-svn: 333563
2018-05-31 01:23:45 +08:00
__builtin_ia32_wrfsbase32(__V);
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
}
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ void __attribute__((__always_inline__, __nodebug__, __target__("fsgsbase")))
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
_writefsbase_u64(unsigned long long __V)
{
[X86] Remove 'return' from a bunch of intrinsics that return void and use a builtin that returns void. Found by running the intrinsic headers through -pedantic -ansi. llvm-svn: 333563
2018-05-31 01:23:45 +08:00
__builtin_ia32_wrfsbase64(__V);
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
}
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ void __attribute__((__always_inline__, __nodebug__, __target__("fsgsbase")))
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
_writegsbase_u32(unsigned int __V)
{
[X86] Remove 'return' from a bunch of intrinsics that return void and use a builtin that returns void. Found by running the intrinsic headers through -pedantic -ansi. llvm-svn: 333563
2018-05-31 01:23:45 +08:00
__builtin_ia32_wrgsbase32(__V);
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
}
Update the intel intrinsic headers to use the target attribute support. This involved removing the conditional inclusion and replacing them with target attributes matching the original conditional inclusion and checks. The testcase update removes the macro checks for each file and replaces them with usage of the __target__ attribute, e.g.: int __attribute__((__target__(("sse3")))) foo(int a) { _mm_mwait(0, 0); return 4; } This usage does require the enclosing function have the requisite __target__ attribute for inlining and code generation - also for any macro intrinsic uses in the enclosing function. There's no change for existing uses of the intrinsic headers. llvm-svn: 239883
2015-06-17 15:09:32 +08:00
static __inline__ void __attribute__((__always_inline__, __nodebug__, __target__("fsgsbase")))
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
_writegsbase_u64(unsigned long long __V)
{
[X86] Remove 'return' from a bunch of intrinsics that return void and use a builtin that returns void. Found by running the intrinsic headers through -pedantic -ansi. llvm-svn: 333563
2018-05-31 01:23:45 +08:00
__builtin_ia32_wrgsbase64(__V);
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
}
Adding front-end support to several intrinsics (bit scanning, conversion and state reading intrinsics) Adding LLVM front-end support to two intrinsics dealing with bit scan: _bit_scan_forward and _bit_scan_reverse. Their functionality is as described in Intel intrinsics guide: https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_bit_scan_forward&expand=371,370 https://software.intel.com/sites/landingpage/IntrinsicsGuide/#text=_bit_scan_reverse&expand=371,370 Furthermore, adding clang front-end support to these conversion intrinsics: _mm256_cvtsd_f64, _mm256_cvtsi256_si32 and _mm256_cvtss_f32. Finally, adding tests to all of the above, as well as to the state reading intrinsics _rdpmc and _rdtsc. Their functionality is also specified in the Intel intrinsics guide. Commit on behalf of Omer Paparo Bivas llvm-svn: 271387
2016-06-01 20:21:00 +08:00
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
#endif
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif /* __FSGSBASE__ */
Add FSGSBASE intrinsics to x86 intrinsic headers. llvm-svn: 221130
2014-11-03 14:51:41 +08:00
[X86] Add the movbe instruction intrinsics from icc. These intrinsics exist in icc. They can be found on the Intel Intrinsics Guide website. All the backend support is in place to pattern match a load+bswap or a bswap+store pattern to the MOVBE instructions. So we just need to get the frontend to emit the correct IR. The pointer arguments in icc are declared as void so I had to jump through a packed struct to forcing a specific alignment on the load/store. Same trick we use in the unaligned vector load/store intrinsics Differential Revision: https://reviews.llvm.org/D52586 llvm-svn: 343343
2018-09-29 01:09:51 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__MOVBE__)
/* The structs used below are to force the load/store to be unaligned. This
* is accomplished with the __packed__ attribute. The __may_alias__ prevents
* tbaa metadata from being generated based on the struct and the type of the
* field inside of it.
*/
static __inline__ short __attribute__((__always_inline__, __nodebug__, __target__("movbe")))
_loadbe_i16(void const * __P) {
struct __loadu_i16 {
short __v;
} __attribute__((__packed__, __may_alias__));
return __builtin_bswap16(((struct __loadu_i16*)__P)->__v);
}
static __inline__ void __attribute__((__always_inline__, __nodebug__, __target__("movbe")))
_storebe_i16(void * __P, short __D) {
struct __storeu_i16 {
short __v;
} __attribute__((__packed__, __may_alias__));
((struct __storeu_i16*)__P)->__v = __builtin_bswap16(__D);
}
static __inline__ int __attribute__((__always_inline__, __nodebug__, __target__("movbe")))
_loadbe_i32(void const * __P) {
struct __loadu_i32 {
int __v;
} __attribute__((__packed__, __may_alias__));
return __builtin_bswap32(((struct __loadu_i32*)__P)->__v);
}
static __inline__ void __attribute__((__always_inline__, __nodebug__, __target__("movbe")))
_storebe_i32(void * __P, int __D) {
struct __storeu_i32 {
int __v;
} __attribute__((__packed__, __may_alias__));
((struct __storeu_i32*)__P)->__v = __builtin_bswap32(__D);
}
#ifdef __x86_64__
static __inline__ long long __attribute__((__always_inline__, __nodebug__, __target__("movbe")))
_loadbe_i64(void const * __P) {
struct __loadu_i64 {
long long __v;
} __attribute__((__packed__, __may_alias__));
return __builtin_bswap64(((struct __loadu_i64*)__P)->__v);
}
static __inline__ void __attribute__((__always_inline__, __nodebug__, __target__("movbe")))
_storebe_i64(void * __P, long long __D) {
struct __storeu_i64 {
long long __v;
} __attribute__((__packed__, __may_alias__));
((struct __storeu_i64*)__P)->__v = __builtin_bswap64(__D);
}
#endif
#endif /* __MOVBE */
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__RTM__)
Add clang support of RTM from TSX - New options '-mrtm'/'-mno-rtm' are added to enable/disable RTM feature - Builtin macro '__RTM__' is defined if RTM feature is enabled - RTM intrinsic header is added and introduces 3 new intrinsics, namely '_xbegin', '_xend', and '_xabort'. - 3 new builtins are added to keep compatible with gcc, namely '__builtin_ia32_xbegin', '__builtin_ia32_xend', and '__builtin_ia32_xabort'. - Test cases for pre-defined macro and new intrinsic codegen are added. llvm-svn: 167665
2012-11-10 13:17:46 +08:00
#include <rtmintrin.h>
Move xtest to its own file to match the gcc header organization. llvm-svn: 239926
2015-06-18 02:42:07 +08:00
#include <xtestintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Add XTEST intrinsic defined in TSX extension llvm-svn: 178330
2013-03-29 13:14:06 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__SHA__)
Move sha intrinsics to immintrin.h This is consistent with ICC and Intel's SHA-enabled GCC version. llvm-svn: 191002
2013-09-19 22:00:22 +08:00
#include <shaintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
Move sha intrinsics to immintrin.h This is consistent with ICC and Intel's SHA-enabled GCC version. llvm-svn: 191002
2013-09-19 22:00:22 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__FXSR__)
[X86] Add FXSR intrinsics Add intrinsics for the FXSR instructions (FXSAVE/FXSAVE64/FXRSTOR/FXRSTOR64) These were previously declared in Intrin.h for MSVC compatibility, but now that we have them implemented, these declarations can be removed. llvm-svn: 241053
2015-06-30 17:45:38 +08:00
#include <fxsrintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[X86] Add FXSR intrinsics Add intrinsics for the FXSR instructions (FXSAVE/FXSAVE64/FXRSTOR/FXRSTOR64) These were previously declared in Intrin.h for MSVC compatibility, but now that we have them implemented, these declarations can be removed. llvm-svn: 241053
2015-06-30 17:45:38 +08:00
Recommit r351160 "[X86] Make _xgetbv/_xsetbv on non-windows platforms" V8 has been fixed now. llvm-svn: 351391
2019-01-17 06:56:25 +08:00
/* No feature check desired due to internal MSC_VER checks */
[X86] Add XSAVE intrinsic family Add intrinsics for the XSAVE instructions (XSAVE/XSAVE64/XRSTOR/XRSTOR64) XSAVEOPT instructions (XSAVEOPT/XSAVEOPT64) XSAVEC instructions (XSAVEC/XSAVEC64) XSAVES instructions (XSAVES/XSAVES64/XRSTORS/XRSTORS64) Differential Revision: http://reviews.llvm.org/D13014 llvm-svn: 250158
2015-10-13 20:29:35 +08:00
#include <xsaveintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__XSAVEOPT__)
[X86] Add XSAVE intrinsic family Add intrinsics for the XSAVE instructions (XSAVE/XSAVE64/XRSTOR/XRSTOR64) XSAVEOPT instructions (XSAVEOPT/XSAVEOPT64) XSAVEC instructions (XSAVEC/XSAVEC64) XSAVES instructions (XSAVES/XSAVES64/XRSTORS/XRSTORS64) Differential Revision: http://reviews.llvm.org/D13014 llvm-svn: 250158
2015-10-13 20:29:35 +08:00
#include <xsaveoptintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[X86] Add XSAVE intrinsic family Add intrinsics for the XSAVE instructions (XSAVE/XSAVE64/XRSTOR/XRSTOR64) XSAVEOPT instructions (XSAVEOPT/XSAVEOPT64) XSAVEC instructions (XSAVEC/XSAVEC64) XSAVES instructions (XSAVES/XSAVES64/XRSTORS/XRSTORS64) Differential Revision: http://reviews.llvm.org/D13014 llvm-svn: 250158
2015-10-13 20:29:35 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__XSAVEC__)
[X86] Add XSAVE intrinsic family Add intrinsics for the XSAVE instructions (XSAVE/XSAVE64/XRSTOR/XRSTOR64) XSAVEOPT instructions (XSAVEOPT/XSAVEOPT64) XSAVEC instructions (XSAVEC/XSAVEC64) XSAVES instructions (XSAVES/XSAVES64/XRSTORS/XRSTORS64) Differential Revision: http://reviews.llvm.org/D13014 llvm-svn: 250158
2015-10-13 20:29:35 +08:00
#include <xsavecintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[X86] Add XSAVE intrinsic family Add intrinsics for the XSAVE instructions (XSAVE/XSAVE64/XRSTOR/XRSTOR64) XSAVEOPT instructions (XSAVEOPT/XSAVEOPT64) XSAVEC instructions (XSAVEC/XSAVEC64) XSAVES instructions (XSAVES/XSAVES64/XRSTORS/XRSTORS64) Differential Revision: http://reviews.llvm.org/D13014 llvm-svn: 250158
2015-10-13 20:29:35 +08:00
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__XSAVES__)
[X86] Add XSAVE intrinsic family Add intrinsics for the XSAVE instructions (XSAVE/XSAVE64/XRSTOR/XRSTOR64) XSAVEOPT instructions (XSAVEOPT/XSAVEOPT64) XSAVEC instructions (XSAVEC/XSAVEC64) XSAVES instructions (XSAVES/XSAVES64/XRSTORS/XRSTORS64) Differential Revision: http://reviews.llvm.org/D13014 llvm-svn: 250158
2015-10-13 20:29:35 +08:00
#include <xsavesintrin.h>
[ms] Reintroduce feature guards in intrinsic headers in Microsoft mode Visual Studio's C++ standard library headers include intrin.h, so the intrinsic headers get included a lot more often in Microsoft mode than elsewhere. The AVX512 intrinsics are a lot of code (0.7 MB, causing 30% compile time overhead for small programs including e.g. <string> and 6% compile time overhead for larger projects like e.g. v8). Since multiversioning can't be relied on in Microsoft mode (cl.exe doesn't support it), having faster compiles seems like the much better tradeoff until we have a better intrinsic story going forward (which we'll need for e.g. PR19898). Actually using intrinsics on Windows already requires the right /arch: settings, so this patch should have no big behavior change. See also thread "The intrinsics headers (especially avx512) are too big. What to do about it?" on cfe-dev. http://reviews.llvm.org/D20291 llvm-svn: 269675
2016-05-17 02:14:07 +08:00
#endif
[X86] Add XSAVE intrinsic family Add intrinsics for the XSAVE instructions (XSAVE/XSAVE64/XRSTOR/XRSTOR64) XSAVEOPT instructions (XSAVEOPT/XSAVEOPT64) XSAVEC instructions (XSAVEC/XSAVEC64) XSAVES instructions (XSAVES/XSAVES64/XRSTORS/XRSTORS64) Differential Revision: http://reviews.llvm.org/D13014 llvm-svn: 250158
2015-10-13 20:29:35 +08:00
Control-Flow Enforcement Technology - Shadow Stack and Indirect Branch Tracking support (Clang side) Shadow stack solution introduces a new stack for return addresses only. The stack has a Shadow Stack Pointer (SSP) that points to the last address to which we expect to return. If we return to a different address an exception is triggered. This patch includes shadow stack intrinsics as well as the corresponding CET header. It includes CET clang flags for shadow stack and Indirect Branch Tracking. For more information, please see the following: https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf Differential Revision: https://reviews.llvm.org/D40224 Change-Id: I79ad0925a028bbc94c8ecad75f6daa2f214171f1 llvm-svn: 318995
2017-11-26 20:34:54 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__SHSTK__)
#include <cetintrin.h>
#endif
Update comments on HLE, RTM, and ADX support for intrinsics. llvm-svn: 239925
2015-06-18 02:42:03 +08:00
/* Some intrinsics inside adxintrin.h are available only on processors with ADX,
* whereas others are also available at all times. */
[x86] Added _addcarryx_u32, _addcarryx_u64 intrinsics llvm-svn: 218117
2014-09-19 18:17:06 +08:00
#include <adxintrin.h>
[X86] Move all Intel defined intrinsic includes into immintrin.h This matches the Intel documentation which shows them available by importing immintrin.h. x86intrin.h also includes immintrin.h so anyone including x86intrin.h will still get them. This is different than gcc, but I don't think we were a perfect match there already. I'm unclear what gcc's policy is about how they choose which to add things to. Differential Revision: https://reviews.llvm.org/D47182 llvm-svn: 333110
2018-05-24 02:32:58 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__RDSEED__)
#include <rdseedintrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__WBNOINVD__)
#include <wbnoinvdintrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__CLDEMOTE__)
#include <cldemoteintrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__WAITPKG__)
#include <waitpkgintrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || \
defined(__MOVDIRI__) || defined(__MOVDIR64B__)
#include <movdirintrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__PCONFIG__)
#include <pconfigintrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__SGX__)
#include <sgxintrin.h>
#endif
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__PTWRITE__)
#include <ptwriteintrin.h>
#endif
[x86] invpcid intrinsic An intrinsic for an old instruction, as described in the Intel SDM. Reviewers: craig.topper, rnk Reviewed By: craig.topper, rnk Differential Revision: https://reviews.llvm.org/D47142 llvm-svn: 333256
2018-05-25 14:34:42 +08:00
#if !defined(_MSC_VER) || __has_feature(modules) || defined(__INVPCID__)
#include <invpcidintrin.h>
#endif
[X86] Add inline assembly versions of _InterlockedExchange_HLEAcquire/Release and _InterlockedCompareExchange_HLEAcquire/Release for MSVC compatibility. Clang/LLVM doesn't have a way to pass an HLE hint through to the X86 backend to emit HLE prefixed instructions. So this is a good short term fix. Differential Revision: https://reviews.llvm.org/D47672 llvm-svn: 334751
2018-06-15 02:43:52 +08:00
#ifdef _MSC_VER
/* Define the default attributes for these intrinsics */
#define __DEFAULT_FN_ATTRS __attribute__((__always_inline__, __nodebug__))
#ifdef __cplusplus
extern "C" {
#endif
/*----------------------------------------------------------------------------*\
|* Interlocked Exchange HLE
\*----------------------------------------------------------------------------*/
#if defined(__i386__) || defined(__x86_64__)
static __inline__ long __DEFAULT_FN_ATTRS
_InterlockedExchange_HLEAcquire(long volatile *_Target, long _Value) {
__asm__ __volatile__(".byte 0xf2 ; lock ; xchg %0, %1"
: "+r" (_Value), "+m" (*_Target) :: "memory");
return _Value;
}
static __inline__ long __DEFAULT_FN_ATTRS
_InterlockedExchange_HLERelease(long volatile *_Target, long _Value) {
__asm__ __volatile__(".byte 0xf3 ; lock ; xchg %0, %1"
: "+r" (_Value), "+m" (*_Target) :: "memory");
return _Value;
}
#endif
#if defined(__x86_64__)
static __inline__ __int64 __DEFAULT_FN_ATTRS
_InterlockedExchange64_HLEAcquire(__int64 volatile *_Target, __int64 _Value) {
__asm__ __volatile__(".byte 0xf2 ; lock ; xchg %0, %1"
: "+r" (_Value), "+m" (*_Target) :: "memory");
return _Value;
}
static __inline__ __int64 __DEFAULT_FN_ATTRS
_InterlockedExchange64_HLERelease(__int64 volatile *_Target, __int64 _Value) {
__asm__ __volatile__(".byte 0xf3 ; lock ; xchg %0, %1"
: "+r" (_Value), "+m" (*_Target) :: "memory");
return _Value;
}
#endif
/*----------------------------------------------------------------------------*\
|* Interlocked Compare Exchange HLE
\*----------------------------------------------------------------------------*/
#if defined(__i386__) || defined(__x86_64__)
static __inline__ long __DEFAULT_FN_ATTRS
_InterlockedCompareExchange_HLEAcquire(long volatile *_Destination,
long _Exchange, long _Comparand) {
__asm__ __volatile__(".byte 0xf2 ; lock ; cmpxchg %2, %1"
: "+a" (_Comparand), "+m" (*_Destination)
: "r" (_Exchange) : "memory");
return _Comparand;
}
static __inline__ long __DEFAULT_FN_ATTRS
_InterlockedCompareExchange_HLERelease(long volatile *_Destination,
long _Exchange, long _Comparand) {
__asm__ __volatile__(".byte 0xf3 ; lock ; cmpxchg %2, %1"
: "+a" (_Comparand), "+m" (*_Destination)
: "r" (_Exchange) : "memory");
return _Comparand;
}
#endif
#if defined(__x86_64__)
static __inline__ __int64 __DEFAULT_FN_ATTRS
_InterlockedCompareExchange64_HLEAcquire(__int64 volatile *_Destination,
__int64 _Exchange, __int64 _Comparand) {
__asm__ __volatile__(".byte 0xf2 ; lock ; cmpxchg %2, %1"
: "+a" (_Comparand), "+m" (*_Destination)
: "r" (_Exchange) : "memory");
return _Comparand;
}
static __inline__ __int64 __DEFAULT_FN_ATTRS
_InterlockedCompareExchange64_HLERelease(__int64 volatile *_Destination,
__int64 _Exchange, __int64 _Comparand) {
__asm__ __volatile__(".byte 0xf3 ; lock ; cmpxchg %2, %1"
: "+a" (_Comparand), "+m" (*_Destination)
: "r" (_Exchange) : "memory");
return _Comparand;
}
#endif
#ifdef __cplusplus
}
#endif
#undef __DEFAULT_FN_ATTRS
#endif /* _MSC_VER */
Add immintrin meta header. - This is the official way to get AVX intrinsics, we might want to disallow direct inclusion of avxintrin.h, just like GCC does. llvm-svn: 111660
2010-08-21 02:04:07 +08:00
#endif /* __IMMINTRIN_H */