maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/haddsub.ll

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2118 lines
75 KiB
LLVM
Raw Normal View History

[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+sse3 | FileCheck %s --check-prefixes=SSE3,SSE3-SLOW
; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+sse3,fast-hops | FileCheck %s --check-prefixes=SSE3,SSE3-FAST
[X86][SSE] Moved haddps test from phaddsub.ll to haddsub.ll (D61245) Also merged duplicate PR39921 + PR39936 tests llvm-svn: 359437
2019-04-29 19:30:47 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+avx | FileCheck %s --check-prefixes=AVX,AVX-SLOW,AVX1,AVX1-SLOW
; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+avx,fast-hops | FileCheck %s --check-prefixes=AVX,AVX-FAST,AVX1,AVX1-FAST
; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+avx2 | FileCheck %s --check-prefixes=AVX,AVX-SLOW,AVX2,AVX2-SLOW
; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+avx2,fast-hops | FileCheck %s --check-prefixes=AVX,AVX-FAST,AVX2,AVX2-FAST
; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+avx512f | FileCheck %s --check-prefixes=AVX,AVX-SLOW,AVX512,AVX512-SLOW
; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+avx512f,fast-hops | FileCheck %s --check-prefixes=AVX,AVX-FAST,AVX512,AVX512-FAST
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <2 x double> @haddpd1(<2 x double> %x, <2 x double> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: haddpd1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: haddpd %xmm1, %xmm0
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: haddpd1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhaddpd %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <2 x double> %x, <2 x double> %y, <2 x i32> <i32 0, i32 2>
%b = shufflevector <2 x double> %x, <2 x double> %y, <2 x i32> <i32 1, i32 3>
%r = fadd <2 x double> %a, %b
ret <2 x double> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <2 x double> @haddpd2(<2 x double> %x, <2 x double> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: haddpd2:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: haddpd %xmm1, %xmm0
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: haddpd2:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhaddpd %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <2 x double> %x, <2 x double> %y, <2 x i32> <i32 1, i32 2>
%b = shufflevector <2 x double> %y, <2 x double> %x, <2 x i32> <i32 2, i32 1>
%r = fadd <2 x double> %a, %b
ret <2 x double> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <2 x double> @haddpd3(<2 x double> %x) {
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
; SSE3-SLOW-LABEL: haddpd3:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addpd %xmm0, %xmm1
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
;
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
; SSE3-FAST-LABEL: haddpd3:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: haddpd3:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddpd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: haddpd3:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <2 x double> %x, <2 x double> undef, <2 x i32> <i32 0, i32 undef>
%b = shufflevector <2 x double> %x, <2 x double> undef, <2 x i32> <i32 1, i32 undef>
%r = fadd <2 x double> %a, %b
ret <2 x double> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @haddps1(<4 x float> %x, <4 x float> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: haddps1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: haddps %xmm1, %xmm0
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: haddps1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhaddps %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> %y, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%b = shufflevector <4 x float> %x, <4 x float> %y, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%r = fadd <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @haddps2(<4 x float> %x, <4 x float> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: haddps2:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: haddps %xmm1, %xmm0
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: haddps2:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhaddps %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> %y, <4 x i32> <i32 1, i32 2, i32 5, i32 6>
%b = shufflevector <4 x float> %y, <4 x float> %x, <4 x i32> <i32 4, i32 7, i32 0, i32 3>
%r = fadd <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @haddps3(<4 x float> %x) {
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-LABEL: haddps3:
; SSE3: # %bb.0:
; SSE3-NEXT: haddps %xmm0, %xmm0
; SSE3-NEXT: retq
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
;
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-LABEL: haddps3:
; AVX: # %bb.0:
; AVX-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 undef, i32 2, i32 4, i32 6>
%b = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 undef, i32 3, i32 5, i32 7>
%r = fadd <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @haddps4(<4 x float> %x) {
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-LABEL: haddps4:
; SSE3: # %bb.0:
; SSE3-NEXT: haddps %xmm0, %xmm0
; SSE3-NEXT: retq
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
;
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-LABEL: haddps4:
; AVX: # %bb.0:
; AVX-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 0, i32 2, i32 undef, i32 undef>
%b = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 1, i32 3, i32 undef, i32 undef>
%r = fadd <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @haddps5(<4 x float> %x) {
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-LABEL: haddps5:
; SSE3: # %bb.0:
; SSE3-NEXT: haddps %xmm0, %xmm0
; SSE3-NEXT: retq
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
;
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-LABEL: haddps5:
; AVX: # %bb.0:
; AVX-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 0, i32 3, i32 undef, i32 undef>
%b = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 1, i32 2, i32 undef, i32 undef>
%r = fadd <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @haddps6(<4 x float> %x) {
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
; SSE3-SLOW-LABEL: haddps6:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
;
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
; SSE3-FAST-LABEL: haddps6:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: haddps6:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: haddps6:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 0, i32 undef, i32 undef, i32 undef>
%b = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
%r = fadd <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @haddps7(<4 x float> %x) {
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-LABEL: haddps7:
; SSE3: # %bb.0:
; SSE3-NEXT: haddps %xmm0, %xmm0
; SSE3-NEXT: retq
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
;
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-LABEL: haddps7:
; AVX: # %bb.0:
; AVX-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 undef, i32 3, i32 undef, i32 undef>
%b = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 undef, i32 2, i32 undef, i32 undef>
%r = fadd <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <2 x double> @hsubpd1(<2 x double> %x, <2 x double> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: hsubpd1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: hsubpd %xmm1, %xmm0
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: hsubpd1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhsubpd %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <2 x double> %x, <2 x double> %y, <2 x i32> <i32 0, i32 2>
%b = shufflevector <2 x double> %x, <2 x double> %y, <2 x i32> <i32 1, i32 3>
%r = fsub <2 x double> %a, %b
ret <2 x double> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <2 x double> @hsubpd2(<2 x double> %x) {
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
; SSE3-SLOW-LABEL: hsubpd2:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: subpd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
;
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
; SSE3-FAST-LABEL: hsubpd2:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: hsubpd2:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vsubpd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: hsubpd2:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <2 x double> %x, <2 x double> undef, <2 x i32> <i32 0, i32 undef>
%b = shufflevector <2 x double> %x, <2 x double> undef, <2 x i32> <i32 1, i32 undef>
%r = fsub <2 x double> %a, %b
ret <2 x double> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @hsubps1(<4 x float> %x, <4 x float> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: hsubps1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: hsubps %xmm1, %xmm0
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: hsubps1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhsubps %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> %y, <4 x i32> <i32 0, i32 2, i32 4, i32 6>
%b = shufflevector <4 x float> %x, <4 x float> %y, <4 x i32> <i32 1, i32 3, i32 5, i32 7>
%r = fsub <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @hsubps2(<4 x float> %x) {
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-LABEL: hsubps2:
; SSE3: # %bb.0:
; SSE3-NEXT: hsubps %xmm0, %xmm0
; SSE3-NEXT: retq
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
;
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-LABEL: hsubps2:
; AVX: # %bb.0:
; AVX-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 undef, i32 2, i32 4, i32 6>
%b = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 undef, i32 3, i32 5, i32 7>
%r = fsub <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @hsubps3(<4 x float> %x) {
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-LABEL: hsubps3:
; SSE3: # %bb.0:
; SSE3-NEXT: hsubps %xmm0, %xmm0
; SSE3-NEXT: retq
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
;
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-LABEL: hsubps3:
; AVX: # %bb.0:
; AVX-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 0, i32 2, i32 undef, i32 undef>
%b = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 1, i32 3, i32 undef, i32 undef>
%r = fsub <4 x float> %a, %b
ret <4 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x float> @hsubps4(<4 x float> %x) {
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
; SSE3-SLOW-LABEL: hsubps4:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: subps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
;
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
; SSE3-FAST-LABEL: hsubps4:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: hsubps4:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vsubps %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: hsubps4:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
Synthesize SSE3/AVX 128 bit horizontal add/sub instructions from floating point add/sub of appropriate shuffle vectors. Does not synthesize the 256 bit AVX versions because they work differently. llvm-svn: 140332
2011-09-23 04:15:48 +08:00
%a = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 0, i32 undef, i32 undef, i32 undef>
%b = shufflevector <4 x float> %x, <4 x float> undef, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
%r = fsub <4 x float> %a, %b
ret <4 x float> %r
}
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors. llvm-svn: 145680
2011-12-02 15:16:01 +08:00
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <8 x float> @vhaddps1(<8 x float> %x, <8 x float> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: vhaddps1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: haddps %xmm2, %xmm0
; SSE3-NEXT: haddps %xmm3, %xmm1
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: vhaddps1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhaddps %ymm1, %ymm0, %ymm0
; AVX-NEXT: retq
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors. llvm-svn: 145680
2011-12-02 15:16:01 +08:00
%a = shufflevector <8 x float> %x, <8 x float> %y, <8 x i32> <i32 0, i32 2, i32 8, i32 10, i32 4, i32 6, i32 12, i32 14>
%b = shufflevector <8 x float> %x, <8 x float> %y, <8 x i32> <i32 1, i32 3, i32 9, i32 11, i32 5, i32 7, i32 13, i32 15>
%r = fadd <8 x float> %a, %b
ret <8 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <8 x float> @vhaddps2(<8 x float> %x, <8 x float> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: vhaddps2:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: haddps %xmm2, %xmm0
; SSE3-NEXT: haddps %xmm3, %xmm1
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: vhaddps2:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhaddps %ymm1, %ymm0, %ymm0
; AVX-NEXT: retq
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors. llvm-svn: 145680
2011-12-02 15:16:01 +08:00
%a = shufflevector <8 x float> %x, <8 x float> %y, <8 x i32> <i32 1, i32 2, i32 9, i32 10, i32 5, i32 6, i32 13, i32 14>
%b = shufflevector <8 x float> %y, <8 x float> %x, <8 x i32> <i32 8, i32 11, i32 0, i32 3, i32 12, i32 15, i32 4, i32 7>
%r = fadd <8 x float> %a, %b
ret <8 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <8 x float> @vhaddps3(<8 x float> %x) {
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-LABEL: vhaddps3:
; SSE3: # %bb.0:
; SSE3-NEXT: haddps %xmm0, %xmm0
; SSE3-NEXT: haddps %xmm1, %xmm1
; SSE3-NEXT: retq
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
;
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-LABEL: vhaddps3:
; AVX: # %bb.0:
; AVX-NEXT: vhaddps %ymm0, %ymm0, %ymm0
; AVX-NEXT: retq
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors. llvm-svn: 145680
2011-12-02 15:16:01 +08:00
%a = shufflevector <8 x float> %x, <8 x float> undef, <8 x i32> <i32 undef, i32 2, i32 8, i32 10, i32 4, i32 6, i32 undef, i32 14>
%b = shufflevector <8 x float> %x, <8 x float> undef, <8 x i32> <i32 1, i32 3, i32 9, i32 undef, i32 5, i32 7, i32 13, i32 15>
%r = fadd <8 x float> %a, %b
ret <8 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <8 x float> @vhsubps1(<8 x float> %x, <8 x float> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: vhsubps1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: hsubps %xmm2, %xmm0
; SSE3-NEXT: hsubps %xmm3, %xmm1
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: vhsubps1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhsubps %ymm1, %ymm0, %ymm0
; AVX-NEXT: retq
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors. llvm-svn: 145680
2011-12-02 15:16:01 +08:00
%a = shufflevector <8 x float> %x, <8 x float> %y, <8 x i32> <i32 0, i32 2, i32 8, i32 10, i32 4, i32 6, i32 12, i32 14>
%b = shufflevector <8 x float> %x, <8 x float> %y, <8 x i32> <i32 1, i32 3, i32 9, i32 11, i32 5, i32 7, i32 13, i32 15>
%r = fsub <8 x float> %a, %b
ret <8 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <8 x float> @vhsubps3(<8 x float> %x) {
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-LABEL: vhsubps3:
; SSE3: # %bb.0:
; SSE3-NEXT: hsubps %xmm0, %xmm0
; SSE3-NEXT: hsubps %xmm1, %xmm1
; SSE3-NEXT: retq
[x86] add and use fast horizontal vector math subtarget feature This is the planned follow-up to D52997. Here we are reducing horizontal vector math codegen by default. AMD Jaguar (btver2) should have no difference with this patch because it has fast-hops. (If we want to set that bit for other CPUs, let me know.) The code changes are small, but there are many test diffs. For files that are specifically testing for hops, I added RUNs to distinguish fast/slow, so we can see the consequences side-by-side. For files that are primarily concerned with codegen other than hops, I just updated the CHECK lines to reflect the new default codegen. To recap the recent horizontal op story: 1. Before rL343727, we were producing hops for all subtargets for a variety of patterns. Hops were likely not optimal for all targets though. 2. The IR improvement in r343727 exposed a hole in the backend hop pattern matching, so we reduced hop codegen for all subtargets. That was bad for Jaguar (PR39195). 3. We restored the hop codegen for all targets with rL344141. Good for Jaguar, but probably bad for other CPUs. 4. This patch allows us to distinguish when we want to produce hops, so everyone can be happy. I'm not sure if we have the best predicate here, but the intent is to undo the extra hop-iness that was enabled by r344141. Differential Revision: https://reviews.llvm.org/D53095 llvm-svn: 344361
2018-10-13 00:41:02 +08:00
;
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-LABEL: vhsubps3:
; AVX: # %bb.0:
; AVX-NEXT: vhsubps %ymm0, %ymm0, %ymm0
; AVX-NEXT: retq
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors. llvm-svn: 145680
2011-12-02 15:16:01 +08:00
%a = shufflevector <8 x float> %x, <8 x float> undef, <8 x i32> <i32 undef, i32 2, i32 8, i32 10, i32 4, i32 6, i32 undef, i32 14>
%b = shufflevector <8 x float> %x, <8 x float> undef, <8 x i32> <i32 1, i32 3, i32 9, i32 undef, i32 5, i32 7, i32 13, i32 15>
%r = fsub <8 x float> %a, %b
ret <8 x float> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x double> @vhaddpd1(<4 x double> %x, <4 x double> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: vhaddpd1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: haddpd %xmm2, %xmm0
; SSE3-NEXT: haddpd %xmm3, %xmm1
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: vhaddpd1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhaddpd %ymm1, %ymm0, %ymm0
; AVX-NEXT: retq
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors. llvm-svn: 145680
2011-12-02 15:16:01 +08:00
%a = shufflevector <4 x double> %x, <4 x double> %y, <4 x i32> <i32 0, i32 4, i32 2, i32 6>
%b = shufflevector <4 x double> %x, <4 x double> %y, <4 x i32> <i32 1, i32 5, i32 3, i32 7>
%r = fadd <4 x double> %a, %b
ret <4 x double> %r
}
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
define <4 x double> @vhsubpd1(<4 x double> %x, <4 x double> %y) {
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; SSE3-LABEL: vhsubpd1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: hsubpd %xmm2, %xmm0
; SSE3-NEXT: hsubpd %xmm3, %xmm1
; SSE3-NEXT: retq
;
Mass update to CodeGen tests to use CHECK-LABEL for labels corresponding to function definitions for more informative error messages. No functionality change and all updated tests passed locally. This update was done with the following bash script: find test/CodeGen -name "*.ll" | \ while read NAME; do echo "$NAME" if ! grep -q "^; *RUN: *llc.*debug" $NAME; then TEMP=`mktemp -t temp` cp $NAME $TEMP sed -n "s/^define [^@]*@\([A-Za-z0-9_]*\)(.*$/\1/p" < $NAME | \ while read FUNC; do sed -i '' "s/;\(.*\)\([A-Za-z0-9_-]*\):\( *\)$FUNC: *\$/;\1\2-LABEL:\3$FUNC:/g" $TEMP done sed -i '' "s/;\(.*\)-LABEL-LABEL:/;\1-LABEL:/" $TEMP sed -i '' "s/;\(.*\)-NEXT-LABEL:/;\1-NEXT:/" $TEMP sed -i '' "s/;\(.*\)-NOT-LABEL:/;\1-NOT:/" $TEMP sed -i '' "s/;\(.*\)-DAG-LABEL:/;\1-DAG:/" $TEMP mv $TEMP $NAME fi done llvm-svn: 186280
2013-07-14 14:24:09 +08:00
; AVX-LABEL: vhsubpd1:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhsubpd %ymm1, %ymm0, %ymm0
; AVX-NEXT: retq
Add instruction selection support for horizontal add/sub of 256-bit floating point vectors. Also add the test case for 256-bit integer vectors. llvm-svn: 145680
2011-12-02 15:16:01 +08:00
%a = shufflevector <4 x double> %x, <4 x double> %y, <4 x i32> <i32 0, i32 4, i32 2, i32 6>
%b = shufflevector <4 x double> %x, <4 x double> %y, <4 x i32> <i32 1, i32 5, i32 3, i32 7>
%r = fsub <4 x double> %a, %b
ret <4 x double> %r
}
X86: Match for X86ISD nodes in LowerBUILD_VECTOR instead of BUILD_VECTORCombine There doesn't seem to be a reason to perform this target ISD node matching in an DAGCombine, moving it to lowering fixes PR23296. Differential Revision: http://reviews.llvm.org/D9137 llvm-svn: 235394
2015-04-22 01:21:36 +08:00
define <2 x float> @haddps_v2f32(<4 x float> %v0) {
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-LABEL: haddps_v2f32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; SSE3: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; SSE3-NEXT: haddps %xmm0, %xmm0
; SSE3-NEXT: retq
;
; AVX-LABEL: haddps_v2f32:
[CodeGen] Unify MBB reference format in both MIR and debug output As part of the unification of the debug format and the MIR format, print MBB references as '%bb.5'. The MIR printer prints the IR name of a MBB only for block definitions. * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)->getNumber\(\)/" << printMBBReference(*\1)/g' * find . \( -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#" << ([a-zA-Z0-9_]+)\.getNumber\(\)/" << printMBBReference(\1)/g' * find . \( -name "*.txt" -o -name "*.s" -o -name "*.mir" -o -name "*.cpp" -o -name "*.h" -o -name "*.ll" \) -type f -print0 | xargs -0 sed -i '' -E 's/BB#([0-9]+)/%bb.\1/g' * grep -nr 'BB#' and fix Differential Revision: https://reviews.llvm.org/D40422 llvm-svn: 319665
2017-12-05 01:18:51 +08:00
; AVX: # %bb.0:
[X86][SSE] Regenerated HADD/HSUB tests llvm-svn: 257992
2016-01-16 22:03:40 +08:00
; AVX-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-NEXT: retq
X86: Match for X86ISD nodes in LowerBUILD_VECTOR instead of BUILD_VECTORCombine There doesn't seem to be a reason to perform this target ISD node matching in an DAGCombine, moving it to lowering fixes PR23296. Differential Revision: http://reviews.llvm.org/D9137 llvm-svn: 235394
2015-04-22 01:21:36 +08:00
%v0.0 = extractelement <4 x float> %v0, i32 0
%v0.1 = extractelement <4 x float> %v0, i32 1
%v0.2 = extractelement <4 x float> %v0, i32 2
%v0.3 = extractelement <4 x float> %v0, i32 3
%op0 = fadd float %v0.0, %v0.1
%op1 = fadd float %v0.2, %v0.3
%res0 = insertelement <2 x float> undef, float %op0, i32 0
%res1 = insertelement <2 x float> %res0, float %op1, i32 1
ret <2 x float> %res1
}
[x86] add haddps test for PR34111; NFC llvm-svn: 311852
2017-08-28 01:15:49 +08:00
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; 128-bit vectors, float/double, fadd/fsub
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v4f32_fadd_f32(<4 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v4f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v4f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v4f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v4f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x0 = extractelement <4 x float> %x, i32 0
%x1 = extractelement <4 x float> %x, i32 1
%x01 = fadd float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract23_v4f32_fadd_f32(<4 x float> %x) {
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-SLOW-LABEL: extract_extract23_v4f32_fadd_f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
;
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-FAST-LABEL: extract_extract23_v4f32_fadd_f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: extract_extract23_v4f32_fadd_f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-SLOW-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract23_v4f32_fadd_f32:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
%x0 = extractelement <4 x float> %x, i32 2
%x1 = extractelement <4 x float> %x, i32 3
%x01 = fadd float %x0, %x1
ret float %x01
}
define float @extract_extract01_v4f32_fadd_f32_commute(<4 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v4f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v4f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v4f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v4f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x0 = extractelement <4 x float> %x, i32 0
%x1 = extractelement <4 x float> %x, i32 1
%x01 = fadd float %x1, %x0
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract23_v4f32_fadd_f32_commute(<4 x float> %x) {
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-SLOW-LABEL: extract_extract23_v4f32_fadd_f32_commute:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
;
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-FAST-LABEL: extract_extract23_v4f32_fadd_f32_commute:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: extract_extract23_v4f32_fadd_f32_commute:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract23_v4f32_fadd_f32_commute:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
%x0 = extractelement <4 x float> %x, i32 2
%x1 = extractelement <4 x float> %x, i32 3
%x01 = fadd float %x1, %x0
ret float %x01
}
define double @extract_extract01_v2f64_fadd_f64(<2 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v2f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addsd %xmm0, %xmm1
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v2f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v2f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v2f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <2 x double> %x, i32 0
%x1 = extractelement <2 x double> %x, i32 1
%x01 = fadd double %x0, %x1
ret double %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v2f64_fadd_f64_commute(<2 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v2f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addsd %xmm0, %xmm1
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v2f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v2f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v2f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <2 x double> %x, i32 0
%x1 = extractelement <2 x double> %x, i32 1
%x01 = fadd double %x1, %x0
ret double %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v4f32_fsub_f32(<4 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v4f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: subss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v4f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v4f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vsubss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v4f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <4 x float> %x, i32 0
%x1 = extractelement <4 x float> %x, i32 1
%x01 = fsub float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract23_v4f32_fsub_f32(<4 x float> %x) {
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-SLOW-LABEL: extract_extract23_v4f32_fsub_f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-SLOW-NEXT: subss %xmm0, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
;
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-FAST-LABEL: extract_extract23_v4f32_fsub_f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubps %xmm0, %xmm0
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: extract_extract23_v4f32_fsub_f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-SLOW-NEXT: vsubss %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract23_v4f32_fsub_f32:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
%x0 = extractelement <4 x float> %x, i32 2
%x1 = extractelement <4 x float> %x, i32 3
%x01 = fsub float %x0, %x1
ret float %x01
}
define float @extract_extract01_v4f32_fsub_f32_commute(<4 x float> %x) {
; SSE3-LABEL: extract_extract01_v4f32_fsub_f32_commute:
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; SSE3: # %bb.0:
; SSE3-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-NEXT: subss %xmm0, %xmm1
; SSE3-NEXT: movaps %xmm1, %xmm0
; SSE3-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-LABEL: extract_extract01_v4f32_fsub_f32_commute:
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; AVX: # %bb.0:
; AVX-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-NEXT: vsubss %xmm0, %xmm1, %xmm0
; AVX-NEXT: retq
%x0 = extractelement <4 x float> %x, i32 0
%x1 = extractelement <4 x float> %x, i32 1
%x01 = fsub float %x1, %x0
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract23_v4f32_fsub_f32_commute(<4 x float> %x) {
; SSE3-LABEL: extract_extract23_v4f32_fsub_f32_commute:
; SSE3: # %bb.0:
; SSE3-NEXT: movaps %xmm0, %xmm1
; SSE3-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-NEXT: subss %xmm1, %xmm0
; SSE3-NEXT: retq
;
; AVX-LABEL: extract_extract23_v4f32_fsub_f32_commute:
; AVX: # %bb.0:
; AVX-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-NEXT: vsubss %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
%x0 = extractelement <4 x float> %x, i32 2
%x1 = extractelement <4 x float> %x, i32 3
%x01 = fsub float %x1, %x0
ret float %x01
}
define double @extract_extract01_v2f64_fsub_f64(<2 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v2f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: subsd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v2f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v2f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vsubsd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v2f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <2 x double> %x, i32 0
%x1 = extractelement <2 x double> %x, i32 1
%x01 = fsub double %x0, %x1
ret double %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v2f64_fsub_f64_commute(<2 x double> %x) {
; SSE3-LABEL: extract_extract01_v2f64_fsub_f64_commute:
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; SSE3: # %bb.0:
; SSE3-NEXT: movapd %xmm0, %xmm1
; SSE3-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-NEXT: subsd %xmm0, %xmm1
; SSE3-NEXT: movapd %xmm1, %xmm0
; SSE3-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-LABEL: extract_extract01_v2f64_fsub_f64_commute:
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; AVX: # %bb.0:
; AVX-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-NEXT: vsubsd %xmm0, %xmm1, %xmm0
; AVX-NEXT: retq
%x0 = extractelement <2 x double> %x, i32 0
%x1 = extractelement <2 x double> %x, i32 1
%x01 = fsub double %x1, %x0
ret double %x01
}
; 256-bit vectors, float/double, fadd/fsub
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v8f32_fadd_f32(<8 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v8f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v8f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v8f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v8f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x0 = extractelement <8 x float> %x, i32 0
%x1 = extractelement <8 x float> %x, i32 1
%x01 = fadd float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract23_v8f32_fadd_f32(<8 x float> %x) {
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-SLOW-LABEL: extract_extract23_v8f32_fadd_f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
;
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-FAST-LABEL: extract_extract23_v8f32_fadd_f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: extract_extract23_v8f32_fadd_f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-SLOW-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract23_v8f32_fadd_f32:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
%x0 = extractelement <8 x float> %x, i32 2
%x1 = extractelement <8 x float> %x, i32 3
%x01 = fadd float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
define float @extract_extract67_v8f32_fadd_f32(<8 x float> %x) {
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-SLOW-LABEL: extract_extract67_v8f32_fadd_f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: movhlps {{.*#+}} xmm1 = xmm1[1,1]
; SSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: extract_extract67_v8f32_fadd_f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm1, %xmm1
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; SSE3-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
;
[X86][SSE] lowerAddSubToHorizontalOp - enable ymm extraction+fold Limiting scalar hadd/hsub generation to the lowest xmm looks to be unnecessary - we will be extracting one upper xmm whatever, and we can remove a shuffle by using the hop which is inline with what shouldUseHorizontalOp expects to happen anyway. Testing on btver2 (the main target for fast-hops) shows this is beneficial even for float ops where we have a 'shuffle' to extract the float result: https://godbolt.org/z/0R-U-K Differential Revision: https://reviews.llvm.org/D61426 llvm-svn: 359786
2019-05-02 22:00:55 +08:00
; AVX-SLOW-LABEL: extract_extract67_v8f32_fadd_f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-SLOW-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract67_v8f32_fadd_f32:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
%x0 = extractelement <8 x float> %x, i32 6
%x1 = extractelement <8 x float> %x, i32 7
%x01 = fadd float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v8f32_fadd_f32_commute(<8 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v8f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v8f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v8f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v8f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x0 = extractelement <8 x float> %x, i32 0
%x1 = extractelement <8 x float> %x, i32 1
%x01 = fadd float %x1, %x0
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract23_v8f32_fadd_f32_commute(<8 x float> %x) {
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-SLOW-LABEL: extract_extract23_v8f32_fadd_f32_commute:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
;
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-FAST-LABEL: extract_extract23_v8f32_fadd_f32_commute:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: extract_extract23_v8f32_fadd_f32_commute:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract23_v8f32_fadd_f32_commute:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
%x0 = extractelement <8 x float> %x, i32 2
%x1 = extractelement <8 x float> %x, i32 3
%x01 = fadd float %x1, %x0
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
define float @extract_extract67_v8f32_fadd_f32_commute(<8 x float> %x) {
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-SLOW-LABEL: extract_extract67_v8f32_fadd_f32_commute:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: movhlps {{.*#+}} xmm1 = xmm1[1,1]
; SSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: extract_extract67_v8f32_fadd_f32_commute:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm1, %xmm1
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; SSE3-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
;
[X86][SSE] lowerAddSubToHorizontalOp - enable ymm extraction+fold Limiting scalar hadd/hsub generation to the lowest xmm looks to be unnecessary - we will be extracting one upper xmm whatever, and we can remove a shuffle by using the hop which is inline with what shouldUseHorizontalOp expects to happen anyway. Testing on btver2 (the main target for fast-hops) shows this is beneficial even for float ops where we have a 'shuffle' to extract the float result: https://godbolt.org/z/0R-U-K Differential Revision: https://reviews.llvm.org/D61426 llvm-svn: 359786
2019-05-02 22:00:55 +08:00
; AVX-SLOW-LABEL: extract_extract67_v8f32_fadd_f32_commute:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract67_v8f32_fadd_f32_commute:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
%x0 = extractelement <8 x float> %x, i32 6
%x1 = extractelement <8 x float> %x, i32 7
%x01 = fadd float %x1, %x0
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v4f64_fadd_f64(<4 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v4f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addsd %xmm0, %xmm1
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v4f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v4f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v4f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <4 x double> %x, i32 0
%x1 = extractelement <4 x double> %x, i32 1
%x01 = fadd double %x0, %x1
ret double %x01
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
}
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
define double @extract_extract23_v4f64_fadd_f64(<4 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract23_v4f64_fadd_f64:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1],xmm1[1]
; SSE3-SLOW-NEXT: addsd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: extract_extract23_v4f64_fadd_f64:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movapd %xmm1, %xmm0
; SSE3-FAST-NEXT: haddpd %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] lowerAddSubToHorizontalOp - enable ymm extraction+fold Limiting scalar hadd/hsub generation to the lowest xmm looks to be unnecessary - we will be extracting one upper xmm whatever, and we can remove a shuffle by using the hop which is inline with what shouldUseHorizontalOp expects to happen anyway. Testing on btver2 (the main target for fast-hops) shows this is beneficial even for float ops where we have a 'shuffle' to extract the float result: https://godbolt.org/z/0R-U-K Differential Revision: https://reviews.llvm.org/D61426 llvm-svn: 359786
2019-05-02 22:00:55 +08:00
; AVX-SLOW-LABEL: extract_extract23_v4f64_fadd_f64:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract23_v4f64_fadd_f64:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
%x0 = extractelement <4 x double> %x, i32 2
%x1 = extractelement <4 x double> %x, i32 3
%x01 = fadd double %x0, %x1
ret double %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v4f64_fadd_f64_commute(<4 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v4f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addsd %xmm0, %xmm1
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v4f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v4f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v4f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <4 x double> %x, i32 0
%x1 = extractelement <4 x double> %x, i32 1
%x01 = fadd double %x1, %x0
ret double %x01
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
}
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
define double @extract_extract23_v4f64_fadd_f64_commute(<4 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract23_v4f64_fadd_f64_commute:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm0 = xmm0[1],xmm1[1]
; SSE3-SLOW-NEXT: addsd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: extract_extract23_v4f64_fadd_f64_commute:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movapd %xmm1, %xmm0
; SSE3-FAST-NEXT: haddpd %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] lowerAddSubToHorizontalOp - enable ymm extraction+fold Limiting scalar hadd/hsub generation to the lowest xmm looks to be unnecessary - we will be extracting one upper xmm whatever, and we can remove a shuffle by using the hop which is inline with what shouldUseHorizontalOp expects to happen anyway. Testing on btver2 (the main target for fast-hops) shows this is beneficial even for float ops where we have a 'shuffle' to extract the float result: https://godbolt.org/z/0R-U-K Differential Revision: https://reviews.llvm.org/D61426 llvm-svn: 359786
2019-05-02 22:00:55 +08:00
; AVX-SLOW-LABEL: extract_extract23_v4f64_fadd_f64_commute:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract23_v4f64_fadd_f64_commute:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
%x0 = extractelement <4 x double> %x, i32 2
%x1 = extractelement <4 x double> %x, i32 3
%x01 = fadd double %x1, %x0
ret double %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v8f32_fsub_f32(<8 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v8f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: subss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v8f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v8f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vsubss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v8f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x0 = extractelement <8 x float> %x, i32 0
%x1 = extractelement <8 x float> %x, i32 1
%x01 = fsub float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract23_v8f32_fsub_f32(<8 x float> %x) {
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-SLOW-LABEL: extract_extract23_v8f32_fsub_f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; SSE3-SLOW-NEXT: subss %xmm0, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
;
[X86][SSE] Fold scalar horizontal add/sub for non-0/1 element extractions We already perform horizontal add/sub if we extract from elements 0 and 1, this patch extends it to non-0/1 element extraction indices (as long as they are from the lowest 128-bit vector). Differential Revision: https://reviews.llvm.org/D61263 llvm-svn: 359707
2019-05-02 01:13:35 +08:00
; SSE3-FAST-LABEL: extract_extract23_v8f32_fsub_f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubps %xmm0, %xmm0
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: extract_extract23_v8f32_fsub_f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vpermilps {{.*#+}} xmm0 = xmm0[3,1,2,3]
; AVX-SLOW-NEXT: vsubss %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract23_v8f32_fsub_f32:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vmovshdup {{.*#+}} xmm0 = xmm0[1,1,3,3]
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
%x0 = extractelement <8 x float> %x, i32 2
%x1 = extractelement <8 x float> %x, i32 3
%x01 = fsub float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
define float @extract_extract45_v8f32_fsub_f32(<8 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract45_v8f32_fsub_f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm1[1,1,3,3]
; SSE3-SLOW-NEXT: subss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: extract_extract45_v8f32_fsub_f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movaps %xmm1, %xmm0
; SSE3-FAST-NEXT: hsubps %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] lowerAddSubToHorizontalOp - enable ymm extraction+fold Limiting scalar hadd/hsub generation to the lowest xmm looks to be unnecessary - we will be extracting one upper xmm whatever, and we can remove a shuffle by using the hop which is inline with what shouldUseHorizontalOp expects to happen anyway. Testing on btver2 (the main target for fast-hops) shows this is beneficial even for float ops where we have a 'shuffle' to extract the float result: https://godbolt.org/z/0R-U-K Differential Revision: https://reviews.llvm.org/D61426 llvm-svn: 359786
2019-05-02 22:00:55 +08:00
; AVX-SLOW-LABEL: extract_extract45_v8f32_fsub_f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vsubss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: extract_extract45_v8f32_fsub_f32:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractf128 $1, %ymm0, %xmm0
; AVX-FAST-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[X86][SSE] Add scalar horizontal add/sub tests for element extractions from upper lanes As suggested on D61263 llvm-svn: 359671
2019-05-01 19:17:11 +08:00
%x0 = extractelement <8 x float> %x, i32 4
%x1 = extractelement <8 x float> %x, i32 5
%x01 = fsub float %x0, %x1
ret float %x01
}
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; Negative test...or get hoppy and negate?
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v8f32_fsub_f32_commute(<8 x float> %x) {
; SSE3-LABEL: extract_extract01_v8f32_fsub_f32_commute:
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; SSE3: # %bb.0:
; SSE3-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-NEXT: subss %xmm0, %xmm1
; SSE3-NEXT: movaps %xmm1, %xmm0
; SSE3-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-LABEL: extract_extract01_v8f32_fsub_f32_commute:
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; AVX: # %bb.0:
; AVX-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-NEXT: vsubss %xmm0, %xmm1, %xmm0
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
%x0 = extractelement <8 x float> %x, i32 0
%x1 = extractelement <8 x float> %x, i32 1
%x01 = fsub float %x1, %x0
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v4f64_fsub_f64(<4 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v4f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: subsd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v4f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v4f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vsubsd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v4f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <4 x double> %x, i32 0
%x1 = extractelement <4 x double> %x, i32 1
%x01 = fsub double %x0, %x1
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
ret double %x01
}
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; Negative test...or get hoppy and negate?
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v4f64_fsub_f64_commute(<4 x double> %x) {
; SSE3-LABEL: extract_extract01_v4f64_fsub_f64_commute:
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; SSE3: # %bb.0:
; SSE3-NEXT: movapd %xmm0, %xmm1
; SSE3-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; SSE3-NEXT: subsd %xmm0, %xmm1
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; SSE3-NEXT: movapd %xmm1, %xmm0
; SSE3-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-LABEL: extract_extract01_v4f64_fsub_f64_commute:
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; AVX: # %bb.0:
; AVX-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; AVX-NEXT: vsubsd %xmm0, %xmm1, %xmm0
; AVX-NEXT: vzeroupper
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; AVX-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <4 x double> %x, i32 0
%x1 = extractelement <4 x double> %x, i32 1
%x01 = fsub double %x1, %x0
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
ret double %x01
}
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; 512-bit vectors, float/double, fadd/fsub
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v16f32_fadd_f32(<16 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v16f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v16f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v16f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v16f32_fadd_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <16 x float> %x, i32 0
%x1 = extractelement <16 x float> %x, i32 1
%x01 = fadd float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v16f32_fadd_f32_commute(<16 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v16f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v16f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v16f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v16f32_fadd_f32_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <16 x float> %x, i32 0
%x1 = extractelement <16 x float> %x, i32 1
%x01 = fadd float %x1, %x0
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v8f64_fadd_f64(<8 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v8f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addsd %xmm0, %xmm1
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v8f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v8f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v8f64_fadd_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <8 x double> %x, i32 0
%x1 = extractelement <8 x double> %x, i32 1
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x01 = fadd double %x0, %x1
ret double %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v8f64_fadd_f64_commute(<8 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v8f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addsd %xmm0, %xmm1
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v8f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v8f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm0, %xmm1, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v8f64_fadd_f64_commute:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhaddpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <8 x double> %x, i32 0
%x1 = extractelement <8 x double> %x, i32 1
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x01 = fadd double %x1, %x0
ret double %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v16f32_fsub_f32(<16 x float> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v16f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: subss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v16f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v16f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vsubss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v16f32_fsub_f32:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <16 x float> %x, i32 0
%x1 = extractelement <16 x float> %x, i32 1
%x01 = fsub float %x0, %x1
ret float %x01
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v16f32_fsub_f32_commute(<16 x float> %x) {
; SSE3-LABEL: extract_extract01_v16f32_fsub_f32_commute:
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; SSE3: # %bb.0:
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; SSE3-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-NEXT: subss %xmm0, %xmm1
; SSE3-NEXT: movaps %xmm1, %xmm0
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; SSE3-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-LABEL: extract_extract01_v16f32_fsub_f32_commute:
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; AVX: # %bb.0:
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; AVX-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-NEXT: vsubss %xmm0, %xmm1, %xmm0
; AVX-NEXT: vzeroupper
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; AVX-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <16 x float> %x, i32 0
%x1 = extractelement <16 x float> %x, i32 1
%x01 = fsub float %x1, %x0
ret float %x01
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v8f64_fsub_f64(<8 x double> %x) {
; SSE3-SLOW-LABEL: extract_extract01_v8f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movapd %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: subsd %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v8f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: hsubpd %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v8f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vsubsd %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v8f64_fsub_f64:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vhsubpd %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <8 x double> %x, i32 0
%x1 = extractelement <8 x double> %x, i32 1
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x01 = fsub double %x0, %x1
ret double %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define double @extract_extract01_v8f64_fsub_f64_commute(<8 x double> %x) {
; SSE3-LABEL: extract_extract01_v8f64_fsub_f64_commute:
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; SSE3: # %bb.0:
; SSE3-NEXT: movapd %xmm0, %xmm1
; SSE3-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-NEXT: subsd %xmm0, %xmm1
; SSE3-NEXT: movapd %xmm1, %xmm0
; SSE3-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-LABEL: extract_extract01_v8f64_fsub_f64_commute:
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
; AVX: # %bb.0:
; AVX-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-NEXT: vsubsd %xmm0, %xmm1, %xmm0
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <8 x double> %x, i32 0
%x1 = extractelement <8 x double> %x, i32 1
[x86] move misplaced tests; NFC Mixed up integer and FP in rL349923. llvm-svn: 349928
2018-12-22 01:06:43 +08:00
%x01 = fsub double %x1, %x0
ret double %x01
}
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; Check output when 1 or both extracts have extra uses.
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v4f32_fadd_f32_uses1(<4 x float> %x, float* %p) {
; SSE3-SLOW-LABEL: extract_extract01_v4f32_fadd_f32_uses1:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movss %xmm0, (%rdi)
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v4f32_fadd_f32_uses1:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movss %xmm0, (%rdi)
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v4f32_fadd_f32_uses1:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovss %xmm0, (%rdi)
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v4f32_fadd_f32_uses1:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vmovss %xmm0, (%rdi)
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <4 x float> %x, i32 0
store float %x0, float* %p
%x1 = extractelement <4 x float> %x, i32 1
%x01 = fadd float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v4f32_fadd_f32_uses2(<4 x float> %x, float* %p) {
; SSE3-SLOW-LABEL: extract_extract01_v4f32_fadd_f32_uses2:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: movss %xmm1, (%rdi)
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
[x86] lower extracted fadd/fsub to horizontal vector math This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350369
2019-01-04 07:16:19 +08:00
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; SSE3-FAST-LABEL: extract_extract01_v4f32_fadd_f32_uses2:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-FAST-NEXT: movss %xmm1, (%rdi)
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-SLOW-LABEL: extract_extract01_v4f32_fadd_f32_uses2:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vmovss %xmm1, (%rdi)
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-FAST-LABEL: extract_extract01_v4f32_fadd_f32_uses2:
[x86] lower extracted fadd/fsub to horizontal vector math; 2nd try The 1st try for this was at rL350369, but it caused IR-level diffs because our cost models differentiate custom vs. legal/promote lowering. So that was reverted at rL350373. The cost models were fixed independently at rL350403, so this is effectively the same patch as last time. Original commit message: This would show up if we fix horizontal reductions to narrow as they go along, but it's an improvement for size and/or Jaguar (fast-hops) independent of that. We need to do this late to not interfere with other pattern matching of larger horizontal sequences. We can extend this to integer ops in a follow-up patch. Differential Revision: https://reviews.llvm.org/D56011 llvm-svn: 350421
2019-01-05 01:48:13 +08:00
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractps $1, %xmm0, (%rdi)
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
%x0 = extractelement <4 x float> %x, i32 0
%x1 = extractelement <4 x float> %x, i32 1
store float %x1, float* %p
%x01 = fadd float %x0, %x1
ret float %x01
}
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
define float @extract_extract01_v4f32_fadd_f32_uses3(<4 x float> %x, float* %p1, float* %p2) {
; SSE3-LABEL: extract_extract01_v4f32_fadd_f32_uses3:
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; SSE3: # %bb.0:
; SSE3-NEXT: movss %xmm0, (%rdi)
; SSE3-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-NEXT: movss %xmm1, (%rsi)
; SSE3-NEXT: addss %xmm1, %xmm0
; SSE3-NEXT: retq
;
[X86][SSE] Add scalar horizontal add/sub tests for non-0/1 element extractions llvm-svn: 359454
2019-04-29 22:26:27 +08:00
; AVX-LABEL: extract_extract01_v4f32_fadd_f32_uses3:
[x86] add more tests for potential horizontal ops; NFC As discussed in D56011 - add runs for AVX512 and tests with extra uses. llvm-svn: 350221
2019-01-03 00:36:04 +08:00
; AVX: # %bb.0:
; AVX-NEXT: vmovss %xmm0, (%rdi)
; AVX-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-NEXT: vmovss %xmm1, (%rsi)
; AVX-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-NEXT: retq
%x0 = extractelement <4 x float> %x, i32 0
store float %x0, float* %p1
%x1 = extractelement <4 x float> %x, i32 1
store float %x1, float* %p2
%x01 = fadd float %x0, %x1
ret float %x01
}
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; Repeat tests from general reductions to verify output for hoppy targets:
; PR38971: https://bugs.llvm.org/show_bug.cgi?id=38971
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
declare float @llvm.experimental.vector.reduce.v2.fadd.f32.v8f32(float, <8 x float>)
declare double @llvm.experimental.vector.reduce.v2.fadd.f64.v4f64(double, <4 x double>)
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
define float @fadd_reduce_v8f32(float %a0, <8 x float> %a1) {
; SSE3-SLOW-LABEL: fadd_reduce_v8f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: addps %xmm2, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm2
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm1[1]
; SSE3-SLOW-NEXT: addps %xmm1, %xmm2
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm2[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm2, %xmm1
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: fadd_reduce_v8f32:
; SSE3-FAST: # %bb.0:
[SDAG][x86] check for relaxed math when matching an FP reduction If the last step in an FP add reduction allows reassociation and doesn't care about -0.0, then we are free to recognize that computation as a reduction that may reorder the intermediate steps. This is requested directly by PR42705: https://bugs.llvm.org/show_bug.cgi?id=42705 and solves PR42947 (if horizontal math instructions are actually faster than the alternative): https://bugs.llvm.org/show_bug.cgi?id=42947 Differential Revision: https://reviews.llvm.org/D66236 llvm-svn: 368995
2019-08-15 20:43:15 +08:00
; SSE3-FAST-NEXT: haddps %xmm1, %xmm2
; SSE3-FAST-NEXT: haddps %xmm2, %xmm2
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; SSE3-FAST-NEXT: haddps %xmm2, %xmm2
; SSE3-FAST-NEXT: addss %xmm2, %xmm0
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: fadd_reduce_v8f32:
; AVX-SLOW: # %bb.0:
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; AVX-SLOW-NEXT: vextractf128 $1, %ymm1, %xmm2
; AVX-SLOW-NEXT: vaddps %xmm2, %xmm1, %xmm1
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX-SLOW-NEXT: vaddps %xmm2, %xmm1, %xmm1
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm2 = xmm1[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm2, %xmm1, %xmm1
[x86] scalarize extract element 0 of FP math This is another step towards ensuring that we produce the optimal code for reductions, but there are other potential benefits as seen in the tests diffs: 1. Memory loads may get scalarized resulting in more efficient code. 2. Memory stores may get scalarized resulting in more efficient code. 3. Complex ops like fdiv/sqrt get scalarized which may be faster instructions depending on uarch. 4. Even simple ops like addss/subss/mulss/roundss may result in faster operation/less frequency throttling when scalarized depending on uarch. The TODO comment suggests 1 or more follow-ups for opcodes that can currently result in regressions. Differential Revision: https://reviews.llvm.org/D58282 llvm-svn: 355130
2019-03-01 03:47:04 +08:00
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: fadd_reduce_v8f32:
; AVX-FAST: # %bb.0:
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; AVX-FAST-NEXT: vextractf128 $1, %ymm1, %xmm2
[SDAG][x86] check for relaxed math when matching an FP reduction If the last step in an FP add reduction allows reassociation and doesn't care about -0.0, then we are free to recognize that computation as a reduction that may reorder the intermediate steps. This is requested directly by PR42705: https://bugs.llvm.org/show_bug.cgi?id=42705 and solves PR42947 (if horizontal math instructions are actually faster than the alternative): https://bugs.llvm.org/show_bug.cgi?id=42947 Differential Revision: https://reviews.llvm.org/D66236 llvm-svn: 368995
2019-08-15 20:43:15 +08:00
; AVX-FAST-NEXT: vhaddps %xmm1, %xmm2, %xmm1
; AVX-FAST-NEXT: vhaddps %xmm1, %xmm1, %xmm1
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; AVX-FAST-NEXT: vhaddps %xmm1, %xmm1, %xmm1
; AVX-FAST-NEXT: vaddss %xmm1, %xmm0, %xmm0
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
%r = call fast float @llvm.experimental.vector.reduce.v2.fadd.f32.v8f32(float %a0, <8 x float> %a1)
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
ret float %r
}
define double @fadd_reduce_v4f64(double %a0, <4 x double> %a1) {
; SSE3-SLOW-LABEL: fadd_reduce_v4f64:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: addpd %xmm2, %xmm1
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; SSE3-SLOW-NEXT: movapd %xmm1, %xmm2
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm2 = xmm2[1],xmm1[1]
; SSE3-SLOW-NEXT: addsd %xmm1, %xmm2
; SSE3-SLOW-NEXT: addsd %xmm2, %xmm0
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: fadd_reduce_v4f64:
; SSE3-FAST: # %bb.0:
[SDAG][x86] check for relaxed math when matching an FP reduction If the last step in an FP add reduction allows reassociation and doesn't care about -0.0, then we are free to recognize that computation as a reduction that may reorder the intermediate steps. This is requested directly by PR42705: https://bugs.llvm.org/show_bug.cgi?id=42705 and solves PR42947 (if horizontal math instructions are actually faster than the alternative): https://bugs.llvm.org/show_bug.cgi?id=42947 Differential Revision: https://reviews.llvm.org/D66236 llvm-svn: 368995
2019-08-15 20:43:15 +08:00
; SSE3-FAST-NEXT: haddpd %xmm1, %xmm2
; SSE3-FAST-NEXT: haddpd %xmm2, %xmm2
; SSE3-FAST-NEXT: addsd %xmm2, %xmm0
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: fadd_reduce_v4f64:
; AVX-SLOW: # %bb.0:
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; AVX-SLOW-NEXT: vextractf128 $1, %ymm1, %xmm2
; AVX-SLOW-NEXT: vaddpd %xmm2, %xmm1, %xmm1
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm2 = xmm1[1,0]
; AVX-SLOW-NEXT: vaddsd %xmm2, %xmm1, %xmm1
[x86] scalarize extract element 0 of FP math This is another step towards ensuring that we produce the optimal code for reductions, but there are other potential benefits as seen in the tests diffs: 1. Memory loads may get scalarized resulting in more efficient code. 2. Memory stores may get scalarized resulting in more efficient code. 3. Complex ops like fdiv/sqrt get scalarized which may be faster instructions depending on uarch. 4. Even simple ops like addss/subss/mulss/roundss may result in faster operation/less frequency throttling when scalarized depending on uarch. The TODO comment suggests 1 or more follow-ups for opcodes that can currently result in regressions. Differential Revision: https://reviews.llvm.org/D58282 llvm-svn: 355130
2019-03-01 03:47:04 +08:00
; AVX-SLOW-NEXT: vaddsd %xmm1, %xmm0, %xmm0
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: fadd_reduce_v4f64:
; AVX-FAST: # %bb.0:
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; AVX-FAST-NEXT: vextractf128 $1, %ymm1, %xmm2
[SDAG][x86] check for relaxed math when matching an FP reduction If the last step in an FP add reduction allows reassociation and doesn't care about -0.0, then we are free to recognize that computation as a reduction that may reorder the intermediate steps. This is requested directly by PR42705: https://bugs.llvm.org/show_bug.cgi?id=42705 and solves PR42947 (if horizontal math instructions are actually faster than the alternative): https://bugs.llvm.org/show_bug.cgi?id=42947 Differential Revision: https://reviews.llvm.org/D66236 llvm-svn: 368995
2019-08-15 20:43:15 +08:00
; AVX-FAST-NEXT: vhaddpd %xmm1, %xmm2, %xmm1
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
; AVX-FAST-NEXT: vhaddpd %xmm1, %xmm1, %xmm1
; AVX-FAST-NEXT: vaddsd %xmm1, %xmm0, %xmm0
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
Change semantics of fadd/fmul vector reductions. This patch changes how LLVM handles the accumulator/start value in the reduction, by never ignoring it regardless of the presence of fast-math flags on callsites. This change introduces the following new intrinsics to replace the existing ones: llvm.experimental.vector.reduce.fadd -> llvm.experimental.vector.reduce.v2.fadd llvm.experimental.vector.reduce.fmul -> llvm.experimental.vector.reduce.v2.fmul and adds functionality to auto-upgrade existing LLVM IR and bitcode. Reviewers: RKSimon, greened, dmgreen, nikic, simoll, aemerson Reviewed By: nikic Differential Revision: https://reviews.llvm.org/D60261 llvm-svn: 363035
2019-06-11 16:22:10 +08:00
%r = call fast double @llvm.experimental.vector.reduce.v2.fadd.f64.v4f64(double %a0, <4 x double> %a1)
[x86] add tests for horizontal ops (PR38971, PR33758); NFC llvm-svn: 353332
2019-02-07 03:40:11 +08:00
ret double %r
}
[X86][SSE] Moved haddps test from phaddsub.ll to haddsub.ll (D61245) Also merged duplicate PR39921 + PR39936 tests llvm-svn: 359437
2019-04-29 19:30:47 +08:00
define float @PR39936_v8f32(<8 x float>) {
; SSSE3-SLOW-LABEL: PR39936_v8f32:
; SSSE3-SLOW: # %bb.0:
; SSSE3-SLOW-NEXT: haddps %xmm1, %xmm0
; SSSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSSE3-SLOW-NEXT: shufps {{.*#+}} xmm1 = xmm1[0,2],xmm0[2,3]
; SSSE3-SLOW-NEXT: shufps {{.*#+}} xmm0 = xmm0[1,3,2,3]
; SSSE3-SLOW-NEXT: addps %xmm1, %xmm0
; SSSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSSE3-SLOW-NEXT: retq
;
; SSSE3-FAST-LABEL: PR39936_v8f32:
; SSSE3-FAST: # %bb.0:
; SSSE3-FAST-NEXT: haddps %xmm1, %xmm0
; SSSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSSE3-FAST-NEXT: retq
;
; SSE3-SLOW-LABEL: PR39936_v8f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: haddps %xmm1, %xmm0
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; SSE3-SLOW-NEXT: haddps %xmm0, %xmm0
[X86][SSE] Moved haddps test from phaddsub.ll to haddsub.ll (D61245) Also merged duplicate PR39921 + PR39936 tests llvm-svn: 359437
2019-04-29 19:30:47 +08:00
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: PR39936_v8f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: haddps %xmm1, %xmm0
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: haddps %xmm0, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: PR39936_v8f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vextractf128 $1, %ymm0, %xmm1
[X86][SSE] isHorizontalBinOp - add support for target shuffles Add target shuffle decoding to isHorizontalBinOp as well as ISD::VECTOR_SHUFFLE support. This does mean we can go through bitcasts so we need to bitcast the extracted args to ensure they are the correct type Fixes PR39936 and should help with PR39920/PR39921 Differential Revision: https://reviews.llvm.org/D61245 llvm-svn: 359491
2019-04-30 03:52:59 +08:00
; AVX-SLOW-NEXT: vhaddps %xmm1, %xmm0, %xmm0
[X86][SSE] Fold add(shuffle(),shuffle()) to hadd on 'slow' targets (PR39920) As reported on PR39920, "slow horizontal ops" targets tend to internally expand to 2*shuffle+add/sub - so if we can reduce 2*shuffle+add/sub to a hadd/sub then we should do it - similar port usage but reduced instruction count. This works out in most cases, although the "PR22377" regression in vector-shuffle-combining.ll is annoying - going from 2*shuffle+add+shuffle to hadd+2*shuffle - I've opened PR41813 to cover this. Differential Revision: https://reviews.llvm.org/D61308 llvm-svn: 360360
2019-05-10 01:45:01 +08:00
; AVX-SLOW-NEXT: vhaddps %xmm0, %xmm0, %xmm0
[X86][SSE] Moved haddps test from phaddsub.ll to haddsub.ll (D61245) Also merged duplicate PR39921 + PR39936 tests llvm-svn: 359437
2019-04-29 19:30:47 +08:00
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: PR39936_v8f32:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vextractf128 $1, %ymm0, %xmm1
[X86][SSE] isHorizontalBinOp - add support for target shuffles Add target shuffle decoding to isHorizontalBinOp as well as ISD::VECTOR_SHUFFLE support. This does mean we can go through bitcasts so we need to bitcast the extracted args to ensure they are the correct type Fixes PR39936 and should help with PR39920/PR39921 Differential Revision: https://reviews.llvm.org/D61245 llvm-svn: 359491
2019-04-30 03:52:59 +08:00
; AVX-FAST-NEXT: vhaddps %xmm1, %xmm0, %xmm0
[X86][SSE] Moved haddps test from phaddsub.ll to haddsub.ll (D61245) Also merged duplicate PR39921 + PR39936 tests llvm-svn: 359437
2019-04-29 19:30:47 +08:00
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
%2 = shufflevector <8 x float> %0, <8 x float> undef, <8 x i32> <i32 0, i32 2, i32 4, i32 6, i32 undef, i32 undef, i32 undef, i32 undef>
%3 = shufflevector <8 x float> %0, <8 x float> undef, <8 x i32> <i32 1, i32 3, i32 5, i32 7, i32 undef, i32 undef, i32 undef, i32 undef>
%4 = fadd <8 x float> %2, %3
%5 = shufflevector <8 x float> %4, <8 x float> undef, <8 x i32> <i32 0, i32 2, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%6 = shufflevector <8 x float> %4, <8 x float> undef, <8 x i32> <i32 1, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%7 = fadd <8 x float> %5, %6
%8 = shufflevector <8 x float> %7, <8 x float> undef, <8 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%9 = fadd <8 x float> %7, %8
%10 = extractelement <8 x float> %9, i32 0
ret float %10
}
[x86] add tests for reductions that might be better with more horizontal ops; NFC llvm-svn: 366082
2019-07-16 00:59:38 +08:00
define float @hadd32_4(<4 x float> %x225) {
; SSE3-SLOW-LABEL: hadd32_4:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addps %xmm0, %xmm1
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm0, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: hadd32_4:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movaps %xmm0, %xmm1
; SSE3-FAST-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-FAST-NEXT: addps %xmm0, %xmm1
; SSE3-FAST-NEXT: haddps %xmm1, %xmm1
; SSE3-FAST-NEXT: movaps %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: hadd32_4:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: hadd32_4:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-FAST-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: retq
%x226 = shufflevector <4 x float> %x225, <4 x float> undef, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
%x227 = fadd <4 x float> %x225, %x226
%x228 = shufflevector <4 x float> %x227, <4 x float> undef, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
%x229 = fadd <4 x float> %x227, %x228
%x230 = extractelement <4 x float> %x229, i32 0
ret float %x230
}
define float @hadd32_8(<8 x float> %x225) {
; SSE3-SLOW-LABEL: hadd32_8:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addps %xmm0, %xmm1
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm0, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: hadd32_8:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movaps %xmm0, %xmm1
; SSE3-FAST-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-FAST-NEXT: addps %xmm0, %xmm1
; SSE3-FAST-NEXT: haddps %xmm1, %xmm1
; SSE3-FAST-NEXT: movaps %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: hadd32_8:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: hadd32_8:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-FAST-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
%x226 = shufflevector <8 x float> %x225, <8 x float> undef, <8 x i32> <i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x227 = fadd <8 x float> %x225, %x226
%x228 = shufflevector <8 x float> %x227, <8 x float> undef, <8 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x229 = fadd <8 x float> %x227, %x228
%x230 = extractelement <8 x float> %x229, i32 0
ret float %x230
}
define float @hadd32_16(<16 x float> %x225) {
; SSE3-SLOW-LABEL: hadd32_16:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addps %xmm0, %xmm1
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm0, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: hadd32_16:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movaps %xmm0, %xmm1
; SSE3-FAST-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-FAST-NEXT: addps %xmm0, %xmm1
; SSE3-FAST-NEXT: haddps %xmm1, %xmm1
; SSE3-FAST-NEXT: movaps %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: hadd32_16:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: hadd32_16:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-FAST-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
%x226 = shufflevector <16 x float> %x225, <16 x float> undef, <16 x i32> <i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x227 = fadd <16 x float> %x225, %x226
%x228 = shufflevector <16 x float> %x227, <16 x float> undef, <16 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x229 = fadd <16 x float> %x227, %x228
%x230 = extractelement <16 x float> %x229, i32 0
ret float %x230
}
define float @hadd32_4_optsize(<4 x float> %x225) optsize {
; SSE3-LABEL: hadd32_4_optsize:
; SSE3: # %bb.0:
; SSE3-NEXT: movaps %xmm0, %xmm1
; SSE3-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-NEXT: addps %xmm0, %xmm1
; SSE3-NEXT: haddps %xmm1, %xmm1
; SSE3-NEXT: movaps %xmm1, %xmm0
; SSE3-NEXT: retq
;
; AVX-LABEL: hadd32_4_optsize:
; AVX: # %bb.0:
; AVX-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-NEXT: retq
%x226 = shufflevector <4 x float> %x225, <4 x float> undef, <4 x i32> <i32 2, i32 3, i32 undef, i32 undef>
%x227 = fadd <4 x float> %x225, %x226
%x228 = shufflevector <4 x float> %x227, <4 x float> undef, <4 x i32> <i32 1, i32 undef, i32 undef, i32 undef>
%x229 = fadd <4 x float> %x227, %x228
%x230 = extractelement <4 x float> %x229, i32 0
ret float %x230
}
define float @hadd32_8_optsize(<8 x float> %x225) optsize {
; SSE3-LABEL: hadd32_8_optsize:
; SSE3: # %bb.0:
; SSE3-NEXT: movaps %xmm0, %xmm1
; SSE3-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-NEXT: addps %xmm0, %xmm1
; SSE3-NEXT: haddps %xmm1, %xmm1
; SSE3-NEXT: movaps %xmm1, %xmm0
; SSE3-NEXT: retq
;
; AVX-LABEL: hadd32_8_optsize:
; AVX: # %bb.0:
; AVX-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
%x226 = shufflevector <8 x float> %x225, <8 x float> undef, <8 x i32> <i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x227 = fadd <8 x float> %x225, %x226
%x228 = shufflevector <8 x float> %x227, <8 x float> undef, <8 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x229 = fadd <8 x float> %x227, %x228
%x230 = extractelement <8 x float> %x229, i32 0
ret float %x230
}
define float @hadd32_16_optsize(<16 x float> %x225) optsize {
; SSE3-LABEL: hadd32_16_optsize:
; SSE3: # %bb.0:
; SSE3-NEXT: movaps %xmm0, %xmm1
; SSE3-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-NEXT: addps %xmm0, %xmm1
; SSE3-NEXT: haddps %xmm1, %xmm1
; SSE3-NEXT: movaps %xmm1, %xmm0
; SSE3-NEXT: retq
;
; AVX-LABEL: hadd32_16_optsize:
; AVX: # %bb.0:
; AVX-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-NEXT: vzeroupper
; AVX-NEXT: retq
%x226 = shufflevector <16 x float> %x225, <16 x float> undef, <16 x i32> <i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x227 = fadd <16 x float> %x225, %x226
%x228 = shufflevector <16 x float> %x227, <16 x float> undef, <16 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x229 = fadd <16 x float> %x227, %x228
%x230 = extractelement <16 x float> %x229, i32 0
ret float %x230
}
[x86] add tests for fadd reduction; NFC More coverage for D66236. llvm-svn: 368913
2019-08-15 04:21:30 +08:00
define float @partial_reduction_fadd_v8f32(<8 x float> %x) {
; SSE3-SLOW-LABEL: partial_reduction_fadd_v8f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addps %xmm0, %xmm1
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm0, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: partial_reduction_fadd_v8f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movaps %xmm0, %xmm1
; SSE3-FAST-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-FAST-NEXT: addps %xmm0, %xmm1
; SSE3-FAST-NEXT: haddps %xmm1, %xmm1
; SSE3-FAST-NEXT: movaps %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: partial_reduction_fadd_v8f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: partial_reduction_fadd_v8f32:
; AVX-FAST: # %bb.0:
[SDAG][x86] check for relaxed math when matching an FP reduction If the last step in an FP add reduction allows reassociation and doesn't care about -0.0, then we are free to recognize that computation as a reduction that may reorder the intermediate steps. This is requested directly by PR42705: https://bugs.llvm.org/show_bug.cgi?id=42705 and solves PR42947 (if horizontal math instructions are actually faster than the alternative): https://bugs.llvm.org/show_bug.cgi?id=42947 Differential Revision: https://reviews.llvm.org/D66236 llvm-svn: 368995
2019-08-15 20:43:15 +08:00
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
[x86] add tests for fadd reduction; NFC More coverage for D66236. llvm-svn: 368913
2019-08-15 04:21:30 +08:00
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
%x23 = shufflevector <8 x float> %x, <8 x float> undef, <8 x i32> <i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x0213 = fadd <8 x float> %x, %x23
%x13 = shufflevector <8 x float> %x0213, <8 x float> undef, <8 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x0123 = fadd nsz reassoc <8 x float> %x0213, %x13
%r = extractelement <8 x float> %x0123, i32 0
ret float %r
}
[SDAG][x86] check for relaxed math when matching an FP reduction If the last step in an FP add reduction allows reassociation and doesn't care about -0.0, then we are free to recognize that computation as a reduction that may reorder the intermediate steps. This is requested directly by PR42705: https://bugs.llvm.org/show_bug.cgi?id=42705 and solves PR42947 (if horizontal math instructions are actually faster than the alternative): https://bugs.llvm.org/show_bug.cgi?id=42947 Differential Revision: https://reviews.llvm.org/D66236 llvm-svn: 368995
2019-08-15 20:43:15 +08:00
; Negative test - only the flags on the final math op in the
; sequence determine whether we can transform to horizontal ops.
[x86] add tests for fadd reduction; NFC More coverage for D66236. llvm-svn: 368913
2019-08-15 04:21:30 +08:00
define float @partial_reduction_fadd_v8f32_wrong_flags(<8 x float> %x) {
; SSE3-SLOW-LABEL: partial_reduction_fadd_v8f32_wrong_flags:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addps %xmm0, %xmm1
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm0, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: partial_reduction_fadd_v8f32_wrong_flags:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movaps %xmm0, %xmm1
; SSE3-FAST-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-FAST-NEXT: addps %xmm0, %xmm1
; SSE3-FAST-NEXT: haddps %xmm1, %xmm1
; SSE3-FAST-NEXT: movaps %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: partial_reduction_fadd_v8f32_wrong_flags:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: partial_reduction_fadd_v8f32_wrong_flags:
; AVX-FAST: # %bb.0:
; AVX-FAST-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-FAST-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
%x23 = shufflevector <8 x float> %x, <8 x float> undef, <8 x i32> <i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x0213 = fadd fast <8 x float> %x, %x23
%x13 = shufflevector <8 x float> %x0213, <8 x float> undef, <8 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x0123 = fadd ninf nnan <8 x float> %x0213, %x13
%r = extractelement <8 x float> %x0123, i32 0
ret float %r
}
define float @partial_reduction_fadd_v16f32(<16 x float> %x) {
; SSE3-SLOW-LABEL: partial_reduction_fadd_v16f32:
; SSE3-SLOW: # %bb.0:
; SSE3-SLOW-NEXT: movaps %xmm0, %xmm1
; SSE3-SLOW-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-SLOW-NEXT: addps %xmm0, %xmm1
; SSE3-SLOW-NEXT: movshdup {{.*#+}} xmm0 = xmm1[1,1,3,3]
; SSE3-SLOW-NEXT: addss %xmm0, %xmm1
; SSE3-SLOW-NEXT: movaps %xmm1, %xmm0
; SSE3-SLOW-NEXT: retq
;
; SSE3-FAST-LABEL: partial_reduction_fadd_v16f32:
; SSE3-FAST: # %bb.0:
; SSE3-FAST-NEXT: movaps %xmm0, %xmm1
; SSE3-FAST-NEXT: unpckhpd {{.*#+}} xmm1 = xmm1[1],xmm0[1]
; SSE3-FAST-NEXT: addps %xmm0, %xmm1
; SSE3-FAST-NEXT: haddps %xmm1, %xmm1
; SSE3-FAST-NEXT: movaps %xmm1, %xmm0
; SSE3-FAST-NEXT: retq
;
; AVX-SLOW-LABEL: partial_reduction_fadd_v16f32:
; AVX-SLOW: # %bb.0:
; AVX-SLOW-NEXT: vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
; AVX-SLOW-NEXT: vaddps %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
; AVX-SLOW-NEXT: vaddss %xmm1, %xmm0, %xmm0
; AVX-SLOW-NEXT: vzeroupper
; AVX-SLOW-NEXT: retq
;
; AVX-FAST-LABEL: partial_reduction_fadd_v16f32:
; AVX-FAST: # %bb.0:
[SDAG][x86] check for relaxed math when matching an FP reduction If the last step in an FP add reduction allows reassociation and doesn't care about -0.0, then we are free to recognize that computation as a reduction that may reorder the intermediate steps. This is requested directly by PR42705: https://bugs.llvm.org/show_bug.cgi?id=42705 and solves PR42947 (if horizontal math instructions are actually faster than the alternative): https://bugs.llvm.org/show_bug.cgi?id=42947 Differential Revision: https://reviews.llvm.org/D66236 llvm-svn: 368995
2019-08-15 20:43:15 +08:00
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
[x86] add tests for fadd reduction; NFC More coverage for D66236. llvm-svn: 368913
2019-08-15 04:21:30 +08:00
; AVX-FAST-NEXT: vhaddps %xmm0, %xmm0, %xmm0
; AVX-FAST-NEXT: vzeroupper
; AVX-FAST-NEXT: retq
%x23 = shufflevector <16 x float> %x, <16 x float> undef, <16 x i32> <i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x0213 = fadd <16 x float> %x, %x23
%x13 = shufflevector <16 x float> %x0213, <16 x float> undef, <16 x i32> <i32 1, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef, i32 undef>
%x0123 = fadd reassoc nsz <16 x float> %x0213, %x13
%r = extractelement <16 x float> %x0123, i32 0
ret float %r
}