maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/fast-isel-nontemporal.ll

829 lines
22 KiB
LLVM
Raw Normal View History

Make utils/update_llc_test_checks.py note that the assertions are autogenerated. Also update existing test cases which appear to be generated by it and weren't modified (other than addition of the header) by rerunning it. llvm-svn: 253917
2015-11-24 05:33:58 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[X86][FastISel] Make sure we use the right register class when we select stores. llvm-svn: 267806
2016-04-28 06:33:42 +08:00
; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+sse2 -fast-isel -O0 < %s | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE2
; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+sse4a -fast-isel -O0 < %s | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE4A
[X86][SSE] Added SSE41/AVX2 non-temporal tests Useful for when we add MOVNTDQA support llvm-svn: 271552
2016-06-03 02:01:21 +08:00
; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+sse4.1 -fast-isel -O0 < %s | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE41
; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+avx -fast-isel -O0 < %s | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX1
; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+avx2 -fast-isel -O0 < %s | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX2
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+avx512f -fast-isel -O0 < %s | FileCheck %s --check-prefix=ALL --check-prefix=AVX512 --check-prefix=AVX512F
; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+avx512bw -fast-isel -O0 < %s | FileCheck %s --check-prefix=ALL --check-prefix=AVX512 --check-prefix=AVX512BW
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
; Scalar Stores
;
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
define void @test_nti32(i32* nocapture %ptr, i32 %X) {
; ALL-LABEL: test_nti32:
; ALL: # BB#0: # %entry
; ALL-NEXT: movntil %esi, (%rdi)
; ALL-NEXT: retq
entry:
store i32 %X, i32* %ptr, align 4, !nontemporal !1
ret void
}
define void @test_nti64(i64* nocapture %ptr, i64 %X) {
; ALL-LABEL: test_nti64:
; ALL: # BB#0: # %entry
; ALL-NEXT: movntiq %rsi, (%rdi)
; ALL-NEXT: retq
entry:
store i64 %X, i64* %ptr, align 8, !nontemporal !1
ret void
}
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
define void @test_ntfloat(float* nocapture %ptr, float %X) {
; SSE2-LABEL: test_ntfloat:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movss %xmm0, (%rdi)
; SSE2-NEXT: retq
;
; SSE4A-LABEL: test_ntfloat:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movntss %xmm0, (%rdi)
; SSE4A-NEXT: retq
;
[X86][SSE] Added SSE41/AVX2 non-temporal tests Useful for when we add MOVNTDQA support llvm-svn: 271552
2016-06-03 02:01:21 +08:00
; SSE41-LABEL: test_ntfloat:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movss %xmm0, (%rdi)
; SSE41-NEXT: retq
;
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; AVX-LABEL: test_ntfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovss %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_ntfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovss %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
entry:
store float %X, float* %ptr, align 4, !nontemporal !1
ret void
}
define void @test_ntdouble(double* nocapture %ptr, double %X) {
; SSE2-LABEL: test_ntdouble:
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
; SSE2: # BB#0: # %entry
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; SSE2-NEXT: movsd %xmm0, (%rdi)
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
; SSE2-NEXT: retq
;
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; SSE4A-LABEL: test_ntdouble:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movntsd %xmm0, (%rdi)
; SSE4A-NEXT: retq
;
[X86][SSE] Added SSE41/AVX2 non-temporal tests Useful for when we add MOVNTDQA support llvm-svn: 271552
2016-06-03 02:01:21 +08:00
; SSE41-LABEL: test_ntdouble:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movsd %xmm0, (%rdi)
; SSE41-NEXT: retq
;
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; AVX-LABEL: test_ntdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovsd %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_ntdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovsd %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
entry:
store double %X, double* %ptr, align 8, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
; 128-bit Vector Stores
;
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
define void @test_nt4xfloat(<4 x float>* nocapture %ptr, <4 x float> %X) {
; SSE-LABEL: test_nt4xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntps %xmm0, (%rdi)
; SSE-NEXT: retq
;
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
; AVX-LABEL: test_nt4xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntps %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt4xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntps %xmm0, (%rdi)
; AVX512-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
entry:
store <4 x float> %X, <4 x float>* %ptr, align 16, !nontemporal !1
ret void
}
define void @test_nt2xdouble(<2 x double>* nocapture %ptr, <2 x double> %X) {
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; SSE-LABEL: test_nt2xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntpd %xmm0, (%rdi)
; SSE-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
;
; AVX-LABEL: test_nt2xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntpd %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt2xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntpd %xmm0, (%rdi)
; AVX512-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
entry:
store <2 x double> %X, <2 x double>* %ptr, align 16, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal store tests for all 128-bit integer vector types llvm-svn: 271389
2016-06-01 21:05:00 +08:00
define void @test_nt16xi8(<16 x i8>* nocapture %ptr, <16 x i8> %X) {
; SSE-LABEL: test_nt16xi8:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt16xi8:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt16xi8:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 128-bit integer vector types llvm-svn: 271389
2016-06-01 21:05:00 +08:00
entry:
store <16 x i8> %X, <16 x i8>* %ptr, align 16, !nontemporal !1
ret void
}
define void @test_nt8xi16(<8 x i16>* nocapture %ptr, <8 x i16> %X) {
; SSE-LABEL: test_nt8xi16:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xi16:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt8xi16:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 128-bit integer vector types llvm-svn: 271389
2016-06-01 21:05:00 +08:00
entry:
store <8 x i16> %X, <8 x i16>* %ptr, align 16, !nontemporal !1
ret void
}
define void @test_nt4xi32(<4 x i32>* nocapture %ptr, <4 x i32> %X) {
; SSE-LABEL: test_nt4xi32:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt4xi32:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt4xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 128-bit integer vector types llvm-svn: 271389
2016-06-01 21:05:00 +08:00
entry:
store <4 x i32> %X, <4 x i32>* %ptr, align 16, !nontemporal !1
ret void
}
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
define void @test_nt2xi64(<2 x i64>* nocapture %ptr, <2 x i64> %X) {
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; SSE-LABEL: test_nt2xi64:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
;
; AVX-LABEL: test_nt2xi64:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt2xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %xmm0, (%rdi)
; AVX512-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
entry:
store <2 x i64> %X, <2 x i64>* %ptr, align 16, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; 128-bit Vector Loads
;
define <4 x float> @test_load_nt4xfloat(<4 x float>* nocapture %ptr) {
; SSE-LABEL: test_load_nt4xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movaps (%rdi), %xmm0
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt4xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovaps (%rdi), %xmm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt4xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovaps (%rdi), %xmm0
; AVX512-NEXT: retq
entry:
%0 = load <4 x float>, <4 x float>* %ptr, align 16, !nontemporal !1
ret <4 x float> %0
}
define <2 x double> @test_load_nt2xdouble(<2 x double>* nocapture %ptr) {
; SSE-LABEL: test_load_nt2xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movapd (%rdi), %xmm0
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt2xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovapd (%rdi), %xmm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt2xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovapd (%rdi), %xmm0
; AVX512-NEXT: retq
entry:
%0 = load <2 x double>, <2 x double>* %ptr, align 16, !nontemporal !1
ret <2 x double> %0
}
define <16 x i8> @test_load_nt16xi8(<16 x i8>* nocapture %ptr) {
; SSE-LABEL: test_load_nt16xi8:
; SSE: # BB#0: # %entry
; SSE-NEXT: movdqa (%rdi), %xmm0
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt16xi8:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovdqa (%rdi), %xmm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt16xi8:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovdqa (%rdi), %xmm0
; AVX512-NEXT: retq
entry:
%0 = load <16 x i8>, <16 x i8>* %ptr, align 16, !nontemporal !1
ret <16 x i8> %0
}
define <8 x i16> @test_load_nt8xi16(<8 x i16>* nocapture %ptr) {
; SSE-LABEL: test_load_nt8xi16:
; SSE: # BB#0: # %entry
; SSE-NEXT: movdqa (%rdi), %xmm0
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt8xi16:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovdqa (%rdi), %xmm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt8xi16:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovdqa (%rdi), %xmm0
; AVX512-NEXT: retq
entry:
%0 = load <8 x i16>, <8 x i16>* %ptr, align 16, !nontemporal !1
ret <8 x i16> %0
}
define <4 x i32> @test_load_nt4xi32(<4 x i32>* nocapture %ptr) {
; SSE-LABEL: test_load_nt4xi32:
; SSE: # BB#0: # %entry
; SSE-NEXT: movdqa (%rdi), %xmm0
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt4xi32:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovdqa (%rdi), %xmm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt4xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovdqa (%rdi), %xmm0
; AVX512-NEXT: retq
entry:
%0 = load <4 x i32>, <4 x i32>* %ptr, align 16, !nontemporal !1
ret <4 x i32> %0
}
define <2 x i64> @test_load_nt2xi64(<2 x i64>* nocapture %ptr) {
; SSE-LABEL: test_load_nt2xi64:
; SSE: # BB#0: # %entry
; SSE-NEXT: movdqa (%rdi), %xmm0
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt2xi64:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovdqa (%rdi), %xmm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt2xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovdqa (%rdi), %xmm0
; AVX512-NEXT: retq
entry:
%0 = load <2 x i64>, <2 x i64>* %ptr, align 16, !nontemporal !1
ret <2 x i64> %0
}
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
; 256-bit Vector Stores
;
define void @test_nt8xfloat(<8 x float>* nocapture %ptr, <8 x float> %X) {
; SSE-LABEL: test_nt8xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntps %xmm0, (%rdi)
; SSE-NEXT: movntps %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntps %ymm0, (%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt8xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntps %ymm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <8 x float> %X, <8 x float>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt4xdouble(<4 x double>* nocapture %ptr, <4 x double> %X) {
; SSE-LABEL: test_nt4xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntpd %xmm0, (%rdi)
; SSE-NEXT: movntpd %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt4xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntpd %ymm0, (%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt4xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntpd %ymm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <4 x double> %X, <4 x double>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt32xi8(<32 x i8>* nocapture %ptr, <32 x i8> %X) {
; SSE-LABEL: test_nt32xi8:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt32xi8:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt32xi8:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %ymm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <32 x i8> %X, <32 x i8>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt16xi16(<16 x i16>* nocapture %ptr, <16 x i16> %X) {
; SSE-LABEL: test_nt16xi16:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt16xi16:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt16xi16:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %ymm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <16 x i16> %X, <16 x i16>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt8xi32(<8 x i32>* nocapture %ptr, <8 x i32> %X) {
; SSE-LABEL: test_nt8xi32:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xi32:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt8xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %ymm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <8 x i32> %X, <8 x i32>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt4xi64(<4 x i64>* nocapture %ptr, <4 x i64> %X) {
; SSE-LABEL: test_nt4xi64:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt4xi64:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt4xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %ymm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <4 x i64> %X, <4 x i64>* %ptr, align 32, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; 256-bit Vector Loads
;
define <8 x float> @test_load_nt8xfloat(<8 x float>* nocapture %ptr) {
; SSE-LABEL: test_load_nt8xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movaps (%rdi), %xmm0
; SSE-NEXT: movaps 16(%rdi), %xmm1
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt8xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovaps (%rdi), %ymm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt8xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovaps (%rdi), %ymm0
; AVX512-NEXT: retq
entry:
%0 = load <8 x float>, <8 x float>* %ptr, align 32, !nontemporal !1
ret <8 x float> %0
}
define <4 x double> @test_load_nt4xdouble(<4 x double>* nocapture %ptr) {
; SSE-LABEL: test_load_nt4xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movapd (%rdi), %xmm0
; SSE-NEXT: movapd 16(%rdi), %xmm1
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt4xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovapd (%rdi), %ymm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt4xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovapd (%rdi), %ymm0
; AVX512-NEXT: retq
entry:
%0 = load <4 x double>, <4 x double>* %ptr, align 32, !nontemporal !1
ret <4 x double> %0
}
define <32 x i8> @test_load_nt32xi8(<32 x i8>* nocapture %ptr) {
; SSE-LABEL: test_load_nt32xi8:
; SSE: # BB#0: # %entry
; SSE-NEXT: movaps (%rdi), %xmm0
; SSE-NEXT: movaps 16(%rdi), %xmm1
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt32xi8:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovdqa (%rdi), %ymm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt32xi8:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovdqa (%rdi), %ymm0
; AVX512-NEXT: retq
entry:
%0 = load <32 x i8>, <32 x i8>* %ptr, align 32, !nontemporal !1
ret <32 x i8> %0
}
define <16 x i16> @test_load_nt16xi16(<16 x i16>* nocapture %ptr) {
; SSE-LABEL: test_load_nt16xi16:
; SSE: # BB#0: # %entry
; SSE-NEXT: movaps (%rdi), %xmm0
; SSE-NEXT: movaps 16(%rdi), %xmm1
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt16xi16:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovdqa (%rdi), %ymm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt16xi16:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovdqa (%rdi), %ymm0
; AVX512-NEXT: retq
entry:
%0 = load <16 x i16>, <16 x i16>* %ptr, align 32, !nontemporal !1
ret <16 x i16> %0
}
define <8 x i32> @test_load_nt8xi32(<8 x i32>* nocapture %ptr) {
; SSE-LABEL: test_load_nt8xi32:
; SSE: # BB#0: # %entry
; SSE-NEXT: movaps (%rdi), %xmm0
; SSE-NEXT: movaps 16(%rdi), %xmm1
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt8xi32:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovdqa (%rdi), %ymm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt8xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovdqa (%rdi), %ymm0
; AVX512-NEXT: retq
entry:
%0 = load <8 x i32>, <8 x i32>* %ptr, align 32, !nontemporal !1
ret <8 x i32> %0
}
define <4 x i64> @test_load_nt4xi64(<4 x i64>* nocapture %ptr) {
; SSE-LABEL: test_load_nt4xi64:
; SSE: # BB#0: # %entry
; SSE-NEXT: movaps (%rdi), %xmm0
; SSE-NEXT: movaps 16(%rdi), %xmm1
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt4xi64:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovdqa (%rdi), %ymm0
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt4xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovdqa (%rdi), %ymm0
; AVX512-NEXT: retq
entry:
%0 = load <4 x i64>, <4 x i64>* %ptr, align 32, !nontemporal !1
ret <4 x i64> %0
}
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
;
; 512-bit Vector Stores
;
define void @test_nt16xfloat(<16 x float>* nocapture %ptr, <16 x float> %X) {
; SSE-LABEL: test_nt16xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntps %xmm0, (%rdi)
; SSE-NEXT: movntps %xmm1, 16(%rdi)
; SSE-NEXT: movntps %xmm2, 32(%rdi)
; SSE-NEXT: movntps %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt16xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntps %ymm0, (%rdi)
; AVX-NEXT: vmovntps %ymm1, 32(%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512-LABEL: test_nt16xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntps %zmm0, (%rdi)
; AVX512-NEXT: retq
entry:
store <16 x float> %X, <16 x float>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt8xdouble(<8 x double>* nocapture %ptr, <8 x double> %X) {
; SSE-LABEL: test_nt8xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntpd %xmm0, (%rdi)
; SSE-NEXT: movntpd %xmm1, 16(%rdi)
; SSE-NEXT: movntpd %xmm2, 32(%rdi)
; SSE-NEXT: movntpd %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntpd %ymm0, (%rdi)
; AVX-NEXT: vmovntpd %ymm1, 32(%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512-LABEL: test_nt8xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntpd %zmm0, (%rdi)
; AVX512-NEXT: retq
entry:
store <8 x double> %X, <8 x double>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt64xi8(<64 x i8>* nocapture %ptr, <64 x i8> %X) {
; SSE-LABEL: test_nt64xi8:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: movntdq %xmm2, 32(%rdi)
; SSE-NEXT: movntdq %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt64xi8:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vmovntdq %ymm1, 32(%rdi)
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512F-LABEL: test_nt64xi8:
; AVX512F: # BB#0: # %entry
; AVX512F-NEXT: vmovntdq %ymm0, (%rdi)
; AVX512F-NEXT: vmovntdq %ymm1, 32(%rdi)
; AVX512F-NEXT: retq
;
; AVX512BW-LABEL: test_nt64xi8:
; AVX512BW: # BB#0: # %entry
; AVX512BW-NEXT: vmovntdq %zmm0, (%rdi)
; AVX512BW-NEXT: retq
entry:
store <64 x i8> %X, <64 x i8>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt32xi16(<32 x i16>* nocapture %ptr, <32 x i16> %X) {
; SSE-LABEL: test_nt32xi16:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: movntdq %xmm2, 32(%rdi)
; SSE-NEXT: movntdq %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt32xi16:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vmovntdq %ymm1, 32(%rdi)
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512F-LABEL: test_nt32xi16:
; AVX512F: # BB#0: # %entry
; AVX512F-NEXT: vmovntdq %ymm0, (%rdi)
; AVX512F-NEXT: vmovntdq %ymm1, 32(%rdi)
; AVX512F-NEXT: retq
;
; AVX512BW-LABEL: test_nt32xi16:
; AVX512BW: # BB#0: # %entry
; AVX512BW-NEXT: vmovntdq %zmm0, (%rdi)
; AVX512BW-NEXT: retq
entry:
store <32 x i16> %X, <32 x i16>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt16xi32(<16 x i32>* nocapture %ptr, <16 x i32> %X) {
; SSE-LABEL: test_nt16xi32:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: movntdq %xmm2, 32(%rdi)
; SSE-NEXT: movntdq %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt16xi32:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vmovntdq %ymm1, 32(%rdi)
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512-LABEL: test_nt16xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %zmm0, (%rdi)
; AVX512-NEXT: retq
entry:
store <16 x i32> %X, <16 x i32>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt8xi64(<8 x i64>* nocapture %ptr, <8 x i64> %X) {
; SSE-LABEL: test_nt8xi64:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: movntdq %xmm2, 32(%rdi)
; SSE-NEXT: movntdq %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xi64:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vmovntdq %ymm1, 32(%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512-LABEL: test_nt8xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %zmm0, (%rdi)
; AVX512-NEXT: retq
entry:
store <8 x i64> %X, <8 x i64>* %ptr, align 64, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; 512-bit Vector Loads
;
define <16 x float> @test_load_nt16xfloat(<16 x float>* nocapture %ptr) {
; SSE-LABEL: test_load_nt16xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movaps (%rdi), %xmm0
; SSE-NEXT: movaps 16(%rdi), %xmm1
; SSE-NEXT: movaps 32(%rdi), %xmm2
; SSE-NEXT: movaps 48(%rdi), %xmm3
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt16xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovaps (%rdi), %ymm0
; AVX-NEXT: vmovaps 32(%rdi), %ymm1
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt16xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovaps (%rdi), %zmm0
; AVX512-NEXT: retq
entry:
%0 = load <16 x float>, <16 x float>* %ptr, align 64, !nontemporal !1
ret <16 x float> %0
}
define <8 x double> @test_load_nt8xdouble(<8 x double>* nocapture %ptr) {
; SSE-LABEL: test_load_nt8xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movapd (%rdi), %xmm0
; SSE-NEXT: movapd 16(%rdi), %xmm1
; SSE-NEXT: movapd 32(%rdi), %xmm2
; SSE-NEXT: movapd 48(%rdi), %xmm3
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt8xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovapd (%rdi), %ymm0
; AVX-NEXT: vmovapd 32(%rdi), %ymm1
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt8xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovapd (%rdi), %zmm0
; AVX512-NEXT: retq
entry:
%0 = load <8 x double>, <8 x double>* %ptr, align 64, !nontemporal !1
ret <8 x double> %0
}
; TODO - 512-bit integer vector loads
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
!1 = !{i32 1}