maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/fast-isel-nontemporal.ll

1222 lines
34 KiB
LLVM
Raw Normal View History

Make utils/update_llc_test_checks.py note that the assertions are autogenerated. Also update existing test cases which appear to be generated by it and weren't modified (other than addition of the header) by rerunning it. llvm-svn: 253917
2015-11-24 05:33:58 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[X86][MMX] Test showing failure to create MMX non-temporal store llvm-svn: 299640
2017-04-06 18:32:30 +08:00
; RUN: llc < %s -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+mmx,+sse2 -fast-isel -O0 | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE2
; RUN: llc < %s -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+mmx,+sse4a -fast-isel -O0 | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE4A
; RUN: llc < %s -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+mmx,+sse4.1 -fast-isel -O0 | FileCheck %s --check-prefix=ALL --check-prefix=SSE --check-prefix=SSE41
; RUN: llc < %s -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+mmx,+avx -fast-isel -O0 | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX1
; RUN: llc < %s -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+mmx,+avx2 -fast-isel -O0 | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX2
; RUN: llc < %s -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+mmx,+avx512f -fast-isel -O0 | FileCheck %s --check-prefix=ALL --check-prefix=AVX512 --check-prefix=AVX512F
; RUN: llc < %s -verify-machineinstrs -mtriple=x86_64-unknown-unknown -mattr=+mmx,+avx512bw -fast-isel -O0 | FileCheck %s --check-prefix=ALL --check-prefix=AVX512 --check-prefix=AVX512BW
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
; Scalar Stores
;
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
define void @test_nti32(i32* nocapture %ptr, i32 %X) {
; ALL-LABEL: test_nti32:
; ALL: # BB#0: # %entry
; ALL-NEXT: movntil %esi, (%rdi)
; ALL-NEXT: retq
entry:
store i32 %X, i32* %ptr, align 4, !nontemporal !1
ret void
}
define void @test_nti64(i64* nocapture %ptr, i64 %X) {
; ALL-LABEL: test_nti64:
; ALL: # BB#0: # %entry
; ALL-NEXT: movntiq %rsi, (%rdi)
; ALL-NEXT: retq
entry:
store i64 %X, i64* %ptr, align 8, !nontemporal !1
ret void
}
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
define void @test_ntfloat(float* nocapture %ptr, float %X) {
; SSE2-LABEL: test_ntfloat:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movss %xmm0, (%rdi)
; SSE2-NEXT: retq
;
; SSE4A-LABEL: test_ntfloat:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movntss %xmm0, (%rdi)
; SSE4A-NEXT: retq
;
[X86][SSE] Added SSE41/AVX2 non-temporal tests Useful for when we add MOVNTDQA support llvm-svn: 271552
2016-06-03 02:01:21 +08:00
; SSE41-LABEL: test_ntfloat:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movss %xmm0, (%rdi)
; SSE41-NEXT: retq
;
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; AVX-LABEL: test_ntfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovss %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_ntfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovss %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
entry:
store float %X, float* %ptr, align 4, !nontemporal !1
ret void
}
define void @test_ntdouble(double* nocapture %ptr, double %X) {
; SSE2-LABEL: test_ntdouble:
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
; SSE2: # BB#0: # %entry
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; SSE2-NEXT: movsd %xmm0, (%rdi)
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
; SSE2-NEXT: retq
;
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; SSE4A-LABEL: test_ntdouble:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movntsd %xmm0, (%rdi)
; SSE4A-NEXT: retq
;
[X86][SSE] Added SSE41/AVX2 non-temporal tests Useful for when we add MOVNTDQA support llvm-svn: 271552
2016-06-03 02:01:21 +08:00
; SSE41-LABEL: test_ntdouble:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movsd %xmm0, (%rdi)
; SSE41-NEXT: retq
;
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; AVX-LABEL: test_ntdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovsd %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_ntdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovsd %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
entry:
store double %X, double* %ptr, align 8, !nontemporal !1
ret void
}
[X86][MMX] Test showing failure to create MMX non-temporal store llvm-svn: 299640
2017-04-06 18:32:30 +08:00
;
; MMX Store
;
define void @test_mmx(x86_mmx* nocapture %a0, x86_mmx* nocapture %a1) {
; ALL-LABEL: test_mmx:
; ALL: # BB#0: # %entry
; ALL-NEXT: movq (%rdi), %mm0
; ALL-NEXT: psrlq $3, %mm0
[X86][MMX] Add fast-isel support for MMX non-temporal writes Differential Revision: https://reviews.llvm.org/D31754 llvm-svn: 299852
2017-04-11 00:58:07 +08:00
; ALL-NEXT: movntq %mm0, (%rsi)
[X86][MMX] Test showing failure to create MMX non-temporal store llvm-svn: 299640
2017-04-06 18:32:30 +08:00
; ALL-NEXT: retq
entry:
%0 = load x86_mmx, x86_mmx* %a0
%1 = call x86_mmx @llvm.x86.mmx.psrli.q(x86_mmx %0, i32 3)
store x86_mmx %1, x86_mmx* %a1, align 8, !nontemporal !1
ret void
}
declare x86_mmx @llvm.x86.mmx.psrli.q(x86_mmx, i32) nounwind readnone
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
; 128-bit Vector Stores
;
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
define void @test_nt4xfloat(<4 x float>* nocapture %ptr, <4 x float> %X) {
; SSE-LABEL: test_nt4xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntps %xmm0, (%rdi)
; SSE-NEXT: retq
;
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
; AVX-LABEL: test_nt4xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntps %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt4xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntps %xmm0, (%rdi)
; AVX512-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
entry:
store <4 x float> %X, <4 x float>* %ptr, align 16, !nontemporal !1
ret void
}
define void @test_nt2xdouble(<2 x double>* nocapture %ptr, <2 x double> %X) {
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; SSE-LABEL: test_nt2xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntpd %xmm0, (%rdi)
; SSE-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
;
; AVX-LABEL: test_nt2xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntpd %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt2xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntpd %xmm0, (%rdi)
; AVX512-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
entry:
store <2 x double> %X, <2 x double>* %ptr, align 16, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal store tests for all 128-bit integer vector types llvm-svn: 271389
2016-06-01 21:05:00 +08:00
define void @test_nt16xi8(<16 x i8>* nocapture %ptr, <16 x i8> %X) {
; SSE-LABEL: test_nt16xi8:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt16xi8:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt16xi8:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 128-bit integer vector types llvm-svn: 271389
2016-06-01 21:05:00 +08:00
entry:
store <16 x i8> %X, <16 x i8>* %ptr, align 16, !nontemporal !1
ret void
}
define void @test_nt8xi16(<8 x i16>* nocapture %ptr, <8 x i16> %X) {
; SSE-LABEL: test_nt8xi16:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xi16:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt8xi16:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 128-bit integer vector types llvm-svn: 271389
2016-06-01 21:05:00 +08:00
entry:
store <8 x i16> %X, <8 x i16>* %ptr, align 16, !nontemporal !1
ret void
}
define void @test_nt4xi32(<4 x i32>* nocapture %ptr, <4 x i32> %X) {
; SSE-LABEL: test_nt4xi32:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt4xi32:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt4xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %xmm0, (%rdi)
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 128-bit integer vector types llvm-svn: 271389
2016-06-01 21:05:00 +08:00
entry:
store <4 x i32> %X, <4 x i32>* %ptr, align 16, !nontemporal !1
ret void
}
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
define void @test_nt2xi64(<2 x i64>* nocapture %ptr, <2 x i64> %X) {
[X86][FastISel] Teach how to select SSE4A nontemporal stores. Add FastISel support for SSE4A scalar float / double non-temporal stores Follow up to D13698 Differential Revision: http://reviews.llvm.org/D13773 llvm-svn: 250610
2015-10-17 21:04:42 +08:00
; SSE-LABEL: test_nt2xi64:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
;
; AVX-LABEL: test_nt2xi64:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %xmm0, (%rdi)
; AVX-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt2xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %xmm0, (%rdi)
; AVX512-NEXT: retq
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
entry:
store <2 x i64> %X, <2 x i64>* %ptr, align 16, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; 128-bit Vector Loads
;
define <4 x float> @test_load_nt4xfloat(<4 x float>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt4xfloat:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: retq
;
; SSE4A-LABEL: test_load_nt4xfloat:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt4xfloat:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX-LABEL: test_load_nt4xfloat:
; AVX: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt4xfloat:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <4 x float>, <4 x float>* %ptr, align 16, !nontemporal !1
ret <4 x float> %0
}
define <2 x double> @test_load_nt2xdouble(<2 x double>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt2xdouble:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movapd (%rdi), %xmm0
; SSE2-NEXT: retq
;
; SSE4A-LABEL: test_load_nt2xdouble:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movapd (%rdi), %xmm0
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt2xdouble:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX-LABEL: test_load_nt2xdouble:
; AVX: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt2xdouble:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <2 x double>, <2 x double>* %ptr, align 16, !nontemporal !1
ret <2 x double> %0
}
define <16 x i8> @test_load_nt16xi8(<16 x i8>* nocapture %ptr) {
; SSE-LABEL: test_load_nt16xi8:
; SSE: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE-NEXT: movntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt16xi8:
; AVX: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt16xi8:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <16 x i8>, <16 x i8>* %ptr, align 16, !nontemporal !1
ret <16 x i8> %0
}
define <8 x i16> @test_load_nt8xi16(<8 x i16>* nocapture %ptr) {
; SSE-LABEL: test_load_nt8xi16:
; SSE: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE-NEXT: movntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt8xi16:
; AVX: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt8xi16:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <8 x i16>, <8 x i16>* %ptr, align 16, !nontemporal !1
ret <8 x i16> %0
}
define <4 x i32> @test_load_nt4xi32(<4 x i32>* nocapture %ptr) {
; SSE-LABEL: test_load_nt4xi32:
; SSE: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE-NEXT: movntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt4xi32:
; AVX: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt4xi32:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <4 x i32>, <4 x i32>* %ptr, align 16, !nontemporal !1
ret <4 x i32> %0
}
define <2 x i64> @test_load_nt2xi64(<2 x i64>* nocapture %ptr) {
; SSE-LABEL: test_load_nt2xi64:
; SSE: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE-NEXT: movntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; SSE-NEXT: retq
;
; AVX-LABEL: test_load_nt2xi64:
; AVX: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX-NEXT: retq
;
; AVX512-LABEL: test_load_nt2xi64:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %xmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <2 x i64>, <2 x i64>* %ptr, align 16, !nontemporal !1
ret <2 x i64> %0
}
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
;
; 256-bit Vector Stores
;
define void @test_nt8xfloat(<8 x float>* nocapture %ptr, <8 x float> %X) {
; SSE-LABEL: test_nt8xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntps %xmm0, (%rdi)
; SSE-NEXT: movntps %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntps %ymm0, (%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt8xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntps %ymm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <8 x float> %X, <8 x float>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt4xdouble(<4 x double>* nocapture %ptr, <4 x double> %X) {
; SSE-LABEL: test_nt4xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntpd %xmm0, (%rdi)
; SSE-NEXT: movntpd %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt4xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntpd %ymm0, (%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt4xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntpd %ymm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <4 x double> %X, <4 x double>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt32xi8(<32 x i8>* nocapture %ptr, <32 x i8> %X) {
; SSE-LABEL: test_nt32xi8:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt32xi8:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt32xi8:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %ymm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <32 x i8> %X, <32 x i8>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt16xi16(<16 x i16>* nocapture %ptr, <16 x i16> %X) {
; SSE-LABEL: test_nt16xi16:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt16xi16:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt16xi16:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %ymm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <16 x i16> %X, <16 x i16>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt8xi32(<8 x i32>* nocapture %ptr, <8 x i32> %X) {
; SSE-LABEL: test_nt8xi32:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xi32:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt8xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %ymm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <8 x i32> %X, <8 x i32>* %ptr, align 32, !nontemporal !1
ret void
}
define void @test_nt4xi64(<4 x i64>* nocapture %ptr, <4 x i64> %X) {
; SSE-LABEL: test_nt4xi64:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt4xi64:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-LABEL: test_nt4xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %ymm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
[X86][SSE] Added non-temporal store tests for all 256-bit vector types Also added KNL AVX-512 checks llvm-svn: 271391
2016-06-01 21:20:25 +08:00
entry:
store <4 x i64> %X, <4 x i64>* %ptr, align 32, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; 256-bit Vector Loads
;
define <8 x float> @test_load_nt8xfloat(<8 x float>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt8xfloat:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE4A-LABEL: test_load_nt8xfloat:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt8xfloat:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt8xfloat:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovaps (%rdi), %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt8xfloat:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX512-LABEL: test_load_nt8xfloat:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %ymm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <8 x float>, <8 x float>* %ptr, align 32, !nontemporal !1
ret <8 x float> %0
}
define <4 x double> @test_load_nt4xdouble(<4 x double>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt4xdouble:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movapd (%rdi), %xmm0
; SSE2-NEXT: movapd 16(%rdi), %xmm1
; SSE2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE4A-LABEL: test_load_nt4xdouble:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movapd (%rdi), %xmm0
; SSE4A-NEXT: movapd 16(%rdi), %xmm1
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt4xdouble:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt4xdouble:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovapd (%rdi), %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt4xdouble:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX512-LABEL: test_load_nt4xdouble:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %ymm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <4 x double>, <4 x double>* %ptr, align 32, !nontemporal !1
ret <4 x double> %0
}
define <32 x i8> @test_load_nt32xi8(<32 x i8>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt32xi8:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE4A-LABEL: test_load_nt32xi8:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt32xi8:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt32xi8:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovdqa (%rdi), %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt32xi8:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX512-LABEL: test_load_nt32xi8:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %ymm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <32 x i8>, <32 x i8>* %ptr, align 32, !nontemporal !1
ret <32 x i8> %0
}
define <16 x i16> @test_load_nt16xi16(<16 x i16>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt16xi16:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE4A-LABEL: test_load_nt16xi16:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt16xi16:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt16xi16:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovdqa (%rdi), %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt16xi16:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX512-LABEL: test_load_nt16xi16:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %ymm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <16 x i16>, <16 x i16>* %ptr, align 32, !nontemporal !1
ret <16 x i16> %0
}
define <8 x i32> @test_load_nt8xi32(<8 x i32>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt8xi32:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE4A-LABEL: test_load_nt8xi32:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt8xi32:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt8xi32:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovdqa (%rdi), %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt8xi32:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX512-LABEL: test_load_nt8xi32:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %ymm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <8 x i32>, <8 x i32>* %ptr, align 32, !nontemporal !1
ret <8 x i32> %0
}
define <4 x i64> @test_load_nt4xi64(<4 x i64>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt4xi64:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE4A-LABEL: test_load_nt4xi64:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt4xi64:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt4xi64:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovdqa (%rdi), %ymm0
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt4xi64:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX512-LABEL: test_load_nt4xi64:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %ymm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <4 x i64>, <4 x i64>* %ptr, align 32, !nontemporal !1
ret <4 x i64> %0
}
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
;
; 512-bit Vector Stores
;
define void @test_nt16xfloat(<16 x float>* nocapture %ptr, <16 x float> %X) {
; SSE-LABEL: test_nt16xfloat:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntps %xmm0, (%rdi)
; SSE-NEXT: movntps %xmm1, 16(%rdi)
; SSE-NEXT: movntps %xmm2, 32(%rdi)
; SSE-NEXT: movntps %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt16xfloat:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntps %ymm0, (%rdi)
; AVX-NEXT: vmovntps %ymm1, 32(%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512-LABEL: test_nt16xfloat:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntps %zmm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
entry:
store <16 x float> %X, <16 x float>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt8xdouble(<8 x double>* nocapture %ptr, <8 x double> %X) {
; SSE-LABEL: test_nt8xdouble:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntpd %xmm0, (%rdi)
; SSE-NEXT: movntpd %xmm1, 16(%rdi)
; SSE-NEXT: movntpd %xmm2, 32(%rdi)
; SSE-NEXT: movntpd %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xdouble:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntpd %ymm0, (%rdi)
; AVX-NEXT: vmovntpd %ymm1, 32(%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512-LABEL: test_nt8xdouble:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntpd %zmm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
entry:
store <8 x double> %X, <8 x double>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt64xi8(<64 x i8>* nocapture %ptr, <64 x i8> %X) {
; SSE-LABEL: test_nt64xi8:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: movntdq %xmm2, 32(%rdi)
; SSE-NEXT: movntdq %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt64xi8:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vmovntdq %ymm1, 32(%rdi)
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512F-LABEL: test_nt64xi8:
; AVX512F: # BB#0: # %entry
; AVX512F-NEXT: vmovntdq %ymm0, (%rdi)
; AVX512F-NEXT: vmovntdq %ymm1, 32(%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512F-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512F-NEXT: retq
;
; AVX512BW-LABEL: test_nt64xi8:
; AVX512BW: # BB#0: # %entry
; AVX512BW-NEXT: vmovntdq %zmm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512BW-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512BW-NEXT: retq
entry:
store <64 x i8> %X, <64 x i8>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt32xi16(<32 x i16>* nocapture %ptr, <32 x i16> %X) {
; SSE-LABEL: test_nt32xi16:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: movntdq %xmm2, 32(%rdi)
; SSE-NEXT: movntdq %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt32xi16:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vmovntdq %ymm1, 32(%rdi)
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512F-LABEL: test_nt32xi16:
; AVX512F: # BB#0: # %entry
; AVX512F-NEXT: vmovntdq %ymm0, (%rdi)
; AVX512F-NEXT: vmovntdq %ymm1, 32(%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512F-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512F-NEXT: retq
;
; AVX512BW-LABEL: test_nt32xi16:
; AVX512BW: # BB#0: # %entry
; AVX512BW-NEXT: vmovntdq %zmm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512BW-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512BW-NEXT: retq
entry:
store <32 x i16> %X, <32 x i16>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt16xi32(<16 x i32>* nocapture %ptr, <16 x i32> %X) {
; SSE-LABEL: test_nt16xi32:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: movntdq %xmm2, 32(%rdi)
; SSE-NEXT: movntdq %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt16xi32:
; AVX: # BB#0: # %entry
[X86] Add AVX 256-bit load and stores to fast isel. I'm not sure why this was missing for so long. This also exposed that we were picking floating point 256-bit VMOVNTPS for some integer types in normal isel for AVX1 even though VMOVNTDQ is available. In practice it doesn't matter due to the execution dependency fix pass, but it required extra isel patterns. Fixing that in a follow up commit. llvm-svn: 271481
2016-06-02 12:19:45 +08:00
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vmovntdq %ymm1, 32(%rdi)
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512-LABEL: test_nt16xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %zmm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
entry:
store <16 x i32> %X, <16 x i32>* %ptr, align 64, !nontemporal !1
ret void
}
define void @test_nt8xi64(<8 x i64>* nocapture %ptr, <8 x i64> %X) {
; SSE-LABEL: test_nt8xi64:
; SSE: # BB#0: # %entry
; SSE-NEXT: movntdq %xmm0, (%rdi)
; SSE-NEXT: movntdq %xmm1, 16(%rdi)
; SSE-NEXT: movntdq %xmm2, 32(%rdi)
; SSE-NEXT: movntdq %xmm3, 48(%rdi)
; SSE-NEXT: retq
;
; AVX-LABEL: test_nt8xi64:
; AVX: # BB#0: # %entry
; AVX-NEXT: vmovntdq %ymm0, (%rdi)
; AVX-NEXT: vmovntdq %ymm1, 32(%rdi)
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
;
; AVX512-LABEL: test_nt8xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdq %zmm0, (%rdi)
[X86] Generate VZEROUPPER for Skylake-avx512. VZEROUPPER should not be issued on Knights Landing (KNL), but on Skylake-avx512 it should be. Differential Revision: https://reviews.llvm.org/D29874 llvm-svn: 296859
2017-03-03 17:03:24 +08:00
; AVX512-NEXT: vzeroupper
[X86][SSE] Added non-temporal store tests for all 512-bit vector types llvm-svn: 271393
2016-06-01 21:58:00 +08:00
; AVX512-NEXT: retq
entry:
store <8 x i64> %X, <8 x i64>* %ptr, align 64, !nontemporal !1
ret void
}
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; 512-bit Vector Loads
;
define <16 x float> @test_load_nt16xfloat(<16 x float>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt16xfloat:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: movaps 32(%rdi), %xmm2
; SSE2-NEXT: movaps 48(%rdi), %xmm3
; SSE2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE4A-LABEL: test_load_nt16xfloat:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: movaps 32(%rdi), %xmm2
; SSE4A-NEXT: movaps 48(%rdi), %xmm3
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt16xfloat:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: movntdqa 32(%rdi), %xmm2
; SSE41-NEXT: movntdqa 48(%rdi), %xmm3
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt16xfloat:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovaps (%rdi), %ymm0
; AVX1-NEXT: vmovaps 32(%rdi), %ymm1
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt16xfloat:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: vmovntdqa 32(%rdi), %ymm1
; AVX2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX512-LABEL: test_load_nt16xfloat:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %zmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <16 x float>, <16 x float>* %ptr, align 64, !nontemporal !1
ret <16 x float> %0
}
define <8 x double> @test_load_nt8xdouble(<8 x double>* nocapture %ptr) {
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE2-LABEL: test_load_nt8xdouble:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movapd (%rdi), %xmm0
; SSE2-NEXT: movapd 16(%rdi), %xmm1
; SSE2-NEXT: movapd 32(%rdi), %xmm2
; SSE2-NEXT: movapd 48(%rdi), %xmm3
; SSE2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; SSE4A-LABEL: test_load_nt8xdouble:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movapd (%rdi), %xmm0
; SSE4A-NEXT: movapd 16(%rdi), %xmm1
; SSE4A-NEXT: movapd 32(%rdi), %xmm2
; SSE4A-NEXT: movapd 48(%rdi), %xmm3
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt8xdouble:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: movntdqa 32(%rdi), %xmm2
; SSE41-NEXT: movntdqa 48(%rdi), %xmm3
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt8xdouble:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovapd (%rdi), %ymm0
; AVX1-NEXT: vmovapd 32(%rdi), %ymm1
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt8xdouble:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: vmovntdqa 32(%rdi), %ymm1
; AVX2-NEXT: retq
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
;
; AVX512-LABEL: test_load_nt8xdouble:
; AVX512: # BB#0: # %entry
[X86][SSE] Add general lowering of nontemporal vector loads Currently the only way to use the (V)MOVNTDQA nontemporal vector loads instructions is through the int_x86_sse41_movntdqa style builtins. This patch adds support for lowering nontemporal loads from general IR, allowing us to remove the movntdqa builtins in a future patch. We currently still fold nontemporal loads into suitable instructions, we should probably look at removing this (and nontemporal stores as well) or at least make the target's folding implementation aware that its dealing with a nontemporal memory transaction. There is also an issue that VMOVNTDQA only acts on 128-bit vectors on pre-AVX2 hardware - so currently a normal ymm load is still used on AVX1 targets. Differential Review: http://reviews.llvm.org/D20965 llvm-svn: 272010
2016-06-07 21:34:24 +08:00
; AVX512-NEXT: vmovntdqa (%rdi), %zmm0
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
; AVX512-NEXT: retq
entry:
%0 = load <8 x double>, <8 x double>* %ptr, align 64, !nontemporal !1
ret <8 x double> %0
}
[X86][AVX512] Added 512-bit integer vector non-temporal load tests llvm-svn: 272016
2016-06-07 23:12:47 +08:00
define <64 x i8> @test_load_nt64xi8(<64 x i8>* nocapture %ptr) {
; SSE2-LABEL: test_load_nt64xi8:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: movaps 32(%rdi), %xmm2
; SSE2-NEXT: movaps 48(%rdi), %xmm3
; SSE2-NEXT: retq
;
; SSE4A-LABEL: test_load_nt64xi8:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: movaps 32(%rdi), %xmm2
; SSE4A-NEXT: movaps 48(%rdi), %xmm3
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt64xi8:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: movntdqa 32(%rdi), %xmm2
; SSE41-NEXT: movntdqa 48(%rdi), %xmm3
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt64xi8:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovaps (%rdi), %ymm0
; AVX1-NEXT: vmovaps 32(%rdi), %ymm1
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt64xi8:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: vmovntdqa 32(%rdi), %ymm1
; AVX2-NEXT: retq
;
; AVX512F-LABEL: test_load_nt64xi8:
; AVX512F: # BB#0: # %entry
; AVX512F-NEXT: vmovntdqa (%rdi), %ymm0
; AVX512F-NEXT: vmovntdqa 32(%rdi), %ymm1
; AVX512F-NEXT: retq
;
; AVX512BW-LABEL: test_load_nt64xi8:
; AVX512BW: # BB#0: # %entry
; AVX512BW-NEXT: vmovntdqa (%rdi), %zmm0
; AVX512BW-NEXT: retq
entry:
%0 = load <64 x i8>, <64 x i8>* %ptr, align 64, !nontemporal !1
ret <64 x i8> %0
}
define <32 x i16> @test_load_nt32xi16(<32 x i16>* nocapture %ptr) {
; SSE2-LABEL: test_load_nt32xi16:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: movaps 32(%rdi), %xmm2
; SSE2-NEXT: movaps 48(%rdi), %xmm3
; SSE2-NEXT: retq
;
; SSE4A-LABEL: test_load_nt32xi16:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: movaps 32(%rdi), %xmm2
; SSE4A-NEXT: movaps 48(%rdi), %xmm3
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt32xi16:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: movntdqa 32(%rdi), %xmm2
; SSE41-NEXT: movntdqa 48(%rdi), %xmm3
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt32xi16:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovaps (%rdi), %ymm0
; AVX1-NEXT: vmovaps 32(%rdi), %ymm1
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt32xi16:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: vmovntdqa 32(%rdi), %ymm1
; AVX2-NEXT: retq
;
; AVX512F-LABEL: test_load_nt32xi16:
; AVX512F: # BB#0: # %entry
; AVX512F-NEXT: vmovntdqa (%rdi), %ymm0
; AVX512F-NEXT: vmovntdqa 32(%rdi), %ymm1
; AVX512F-NEXT: retq
;
; AVX512BW-LABEL: test_load_nt32xi16:
; AVX512BW: # BB#0: # %entry
; AVX512BW-NEXT: vmovntdqa (%rdi), %zmm0
; AVX512BW-NEXT: retq
entry:
%0 = load <32 x i16>, <32 x i16>* %ptr, align 64, !nontemporal !1
ret <32 x i16> %0
}
define <16 x i32> @test_load_nt16xi32(<16 x i32>* nocapture %ptr) {
; SSE2-LABEL: test_load_nt16xi32:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: movaps 32(%rdi), %xmm2
; SSE2-NEXT: movaps 48(%rdi), %xmm3
; SSE2-NEXT: retq
;
; SSE4A-LABEL: test_load_nt16xi32:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: movaps 32(%rdi), %xmm2
; SSE4A-NEXT: movaps 48(%rdi), %xmm3
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt16xi32:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: movntdqa 32(%rdi), %xmm2
; SSE41-NEXT: movntdqa 48(%rdi), %xmm3
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt16xi32:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovaps (%rdi), %ymm0
; AVX1-NEXT: vmovaps 32(%rdi), %ymm1
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt16xi32:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: vmovntdqa 32(%rdi), %ymm1
; AVX2-NEXT: retq
;
; AVX512-LABEL: test_load_nt16xi32:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdqa (%rdi), %zmm0
; AVX512-NEXT: retq
entry:
%0 = load <16 x i32>, <16 x i32>* %ptr, align 64, !nontemporal !1
ret <16 x i32> %0
}
define <8 x i64> @test_load_nt8xi64(<8 x i64>* nocapture %ptr) {
; SSE2-LABEL: test_load_nt8xi64:
; SSE2: # BB#0: # %entry
; SSE2-NEXT: movaps (%rdi), %xmm0
; SSE2-NEXT: movaps 16(%rdi), %xmm1
; SSE2-NEXT: movaps 32(%rdi), %xmm2
; SSE2-NEXT: movaps 48(%rdi), %xmm3
; SSE2-NEXT: retq
;
; SSE4A-LABEL: test_load_nt8xi64:
; SSE4A: # BB#0: # %entry
; SSE4A-NEXT: movaps (%rdi), %xmm0
; SSE4A-NEXT: movaps 16(%rdi), %xmm1
; SSE4A-NEXT: movaps 32(%rdi), %xmm2
; SSE4A-NEXT: movaps 48(%rdi), %xmm3
; SSE4A-NEXT: retq
;
; SSE41-LABEL: test_load_nt8xi64:
; SSE41: # BB#0: # %entry
; SSE41-NEXT: movntdqa (%rdi), %xmm0
; SSE41-NEXT: movntdqa 16(%rdi), %xmm1
; SSE41-NEXT: movntdqa 32(%rdi), %xmm2
; SSE41-NEXT: movntdqa 48(%rdi), %xmm3
; SSE41-NEXT: retq
;
; AVX1-LABEL: test_load_nt8xi64:
; AVX1: # BB#0: # %entry
; AVX1-NEXT: vmovaps (%rdi), %ymm0
; AVX1-NEXT: vmovaps 32(%rdi), %ymm1
; AVX1-NEXT: retq
;
; AVX2-LABEL: test_load_nt8xi64:
; AVX2: # BB#0: # %entry
; AVX2-NEXT: vmovntdqa (%rdi), %ymm0
; AVX2-NEXT: vmovntdqa 32(%rdi), %ymm1
; AVX2-NEXT: retq
;
; AVX512-LABEL: test_load_nt8xi64:
; AVX512: # BB#0: # %entry
; AVX512-NEXT: vmovntdqa (%rdi), %zmm0
; AVX512-NEXT: retq
entry:
%0 = load <8 x i64>, <8 x i64>* %ptr, align 64, !nontemporal !1
ret <8 x i64> %0
}
[X86][SSE] Added non-temporal load tests for vector types These currently lower to regular loads instead of MOVNTDQA llvm-svn: 271516
2016-06-02 21:51:50 +08:00
[x86][FastISel] Teach how to select nontemporal stores. This patch teaches x86 fast-isel how to select nontemporal stores. On x86, we can use MOVNTI for nontemporal stores of doublewords/quadwords. Instructions (V)MOVNTPS/PD/DQ can be used for SSE2/AVX aligned nontemporal vector stores. Before this patch, fast-isel always selected 'movd/movq' instead of 'movnti' for doubleword/quadword nontemporal stores. In the case of nontemporal stores of aligned vectors, fast-isel always selected movaps/movapd/movdqa instead of movntps/movntpd/movntdq. With this patch, if we use SSE2/AVX intrinsics for nontemporal stores we now always get the expected (V)MOVNT instructions. The lack of fast-isel support for nontemporal stores was spotted when analyzing the -O0 codegen for nontemporal stores. Differential Revision: http://reviews.llvm.org/D13698 llvm-svn: 250285
2015-10-14 18:03:13 +08:00
!1 = !{i32 1}