maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/cmovcmov.ll

283 lines
7.8 KiB
LLVM
Raw Normal View History

[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; RUN: llc < %s -asm-verbose=false -mtriple=x86_64-unknown-linux | FileCheck %s --check-prefix=CHECK --check-prefix=CMOV
; RUN: llc < %s -asm-verbose=false -mtriple=i686-unknown-linux | FileCheck %s --check-prefix=CHECK --check-prefix=NOCMOV
target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
; Test 2xCMOV patterns exposed after legalization.
; One way to do that is with (select (fcmp une/oeq)), which gets
; legalized to setp/setne.
; CHECK-LABEL: test_select_fcmp_oeq_i32:
; CMOV-NEXT: ucomiss %xmm1, %xmm0
; CMOV-NEXT: cmovnel %esi, %edi
; CMOV-NEXT: cmovpl %esi, %edi
; CMOV-NEXT: movl %edi, %eax
; CMOV-NEXT: retq
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: flds 8(%esp)
; NOCMOV-NEXT: flds 4(%esp)
; NOCMOV-NEXT: fucompp
; NOCMOV-NEXT: fnstsw %ax
; NOCMOV-NEXT: sahf
; NOCMOV-NEXT: leal 16(%esp), %eax
; NOCMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; NOCMOV-NEXT: jp [[TBB]]
; NOCMOV-NEXT: leal 12(%esp), %eax
; NOCMOV-NEXT:[[TBB]]:
; NOCMOV-NEXT: movl (%eax), %eax
; NOCMOV-NEXT: retl
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
define i32 @test_select_fcmp_oeq_i32(float %a, float %b, i32 %c, i32 %d) #0 {
entry:
%cmp = fcmp oeq float %a, %b
%r = select i1 %cmp, i32 %c, i32 %d
ret i32 %r
}
; CHECK-LABEL: test_select_fcmp_oeq_i64:
; CMOV-NEXT: ucomiss %xmm1, %xmm0
; CMOV-NEXT: cmovneq %rsi, %rdi
; CMOV-NEXT: cmovpq %rsi, %rdi
; CMOV-NEXT: movq %rdi, %rax
; CMOV-NEXT: retq
; NOCMOV-NEXT: flds 8(%esp)
; NOCMOV-NEXT: flds 4(%esp)
; NOCMOV-NEXT: fucompp
; NOCMOV-NEXT: fnstsw %ax
; NOCMOV-NEXT: sahf
; NOCMOV-NEXT: leal 20(%esp), %ecx
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; NOCMOV-NEXT: jp [[TBB]]
; NOCMOV-NEXT: leal 12(%esp), %ecx
; NOCMOV-NEXT: [[TBB]]:
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; NOCMOV-NEXT: movl (%ecx), %eax
; NOCMOV-NEXT: orl $4, %ecx
; NOCMOV-NEXT: movl (%ecx), %edx
; NOCMOV-NEXT: retl
define i64 @test_select_fcmp_oeq_i64(float %a, float %b, i64 %c, i64 %d) #0 {
entry:
%cmp = fcmp oeq float %a, %b
%r = select i1 %cmp, i64 %c, i64 %d
ret i64 %r
}
; CHECK-LABEL: test_select_fcmp_une_i64:
; CMOV-NEXT: ucomiss %xmm1, %xmm0
; CMOV-NEXT: cmovneq %rdi, %rsi
; CMOV-NEXT: cmovpq %rdi, %rsi
; CMOV-NEXT: movq %rsi, %rax
; CMOV-NEXT: retq
; NOCMOV-NEXT: flds 8(%esp)
; NOCMOV-NEXT: flds 4(%esp)
; NOCMOV-NEXT: fucompp
; NOCMOV-NEXT: fnstsw %ax
; NOCMOV-NEXT: sahf
; NOCMOV-NEXT: leal 12(%esp), %ecx
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; NOCMOV-NEXT: jp [[TBB]]
; NOCMOV-NEXT: leal 20(%esp), %ecx
; NOCMOV-NEXT: [[TBB]]:
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; NOCMOV-NEXT: movl (%ecx), %eax
; NOCMOV-NEXT: orl $4, %ecx
; NOCMOV-NEXT: movl (%ecx), %edx
; NOCMOV-NEXT: retl
define i64 @test_select_fcmp_une_i64(float %a, float %b, i64 %c, i64 %d) #0 {
entry:
%cmp = fcmp une float %a, %b
%r = select i1 %cmp, i64 %c, i64 %d
ret i64 %r
}
; CHECK-LABEL: test_select_fcmp_oeq_f64:
; CMOV-NEXT: ucomiss %xmm1, %xmm0
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; CMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; CMOV-NEXT: jp [[TBB]]
; CMOV-NEXT: movaps %xmm2, %xmm3
; CMOV-NEXT: [[TBB]]:
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; CMOV-NEXT: movaps %xmm3, %xmm0
; CMOV-NEXT: retq
; NOCMOV-NEXT: flds 8(%esp)
; NOCMOV-NEXT: flds 4(%esp)
; NOCMOV-NEXT: fucompp
; NOCMOV-NEXT: fnstsw %ax
; NOCMOV-NEXT: sahf
; NOCMOV-NEXT: leal 20(%esp), %eax
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; NOCMOV-NEXT: jp [[TBB]]
; NOCMOV-NEXT: leal 12(%esp), %eax
; NOCMOV-NEXT: [[TBB]]:
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; NOCMOV-NEXT: fldl (%eax)
; NOCMOV-NEXT: retl
define double @test_select_fcmp_oeq_f64(float %a, float %b, double %c, double %d) #0 {
entry:
%cmp = fcmp oeq float %a, %b
%r = select i1 %cmp, double %c, double %d
ret double %r
}
; CHECK-LABEL: test_select_fcmp_oeq_v4i32:
; CMOV-NEXT: ucomiss %xmm1, %xmm0
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; CMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; CMOV-NEXT: jp [[TBB]]
; CMOV-NEXT: movaps %xmm2, %xmm3
; CMOV-NEXT: [[TBB]]:
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; CMOV-NEXT: movaps %xmm3, %xmm0
; CMOV-NEXT: retq
; NOCMOV-NEXT: pushl %edi
; NOCMOV-NEXT: pushl %esi
; NOCMOV-NEXT: flds 20(%esp)
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: flds 16(%esp)
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; NOCMOV-NEXT: fucompp
; NOCMOV-NEXT: fnstsw %ax
; NOCMOV-NEXT: sahf
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: leal 40(%esp), %eax
; NOCMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; NOCMOV-NEXT: jp [[TBB]]
; NOCMOV-NEXT: leal 24(%esp), %eax
; NOCMOV-NEXT: [[TBB]]:
Re-land r237175: [X86] Always return the sret parameter in eax/rax ... This reverts commit r237210. Also fix X86/complex-fca.ll to match the code that we used to generate on win32 and now generate everwhere to conform to SysV. llvm-svn: 237639
2015-05-19 07:35:09 +08:00
; NOCMOV-NEXT: movl (%eax), %ecx
; NOCMOV-NEXT: leal 44(%esp), %edx
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; NOCMOV-NEXT: jp [[TBB]]
Re-land r237175: [X86] Always return the sret parameter in eax/rax ... This reverts commit r237210. Also fix X86/complex-fca.ll to match the code that we used to generate on win32 and now generate everwhere to conform to SysV. llvm-svn: 237639
2015-05-19 07:35:09 +08:00
; NOCMOV-NEXT: leal 28(%esp), %edx
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: [[TBB]]:
Re-land r237175: [X86] Always return the sret parameter in eax/rax ... This reverts commit r237210. Also fix X86/complex-fca.ll to match the code that we used to generate on win32 and now generate everwhere to conform to SysV. llvm-svn: 237639
2015-05-19 07:35:09 +08:00
; NOCMOV-NEXT: movl 12(%esp), %eax
; NOCMOV-NEXT: movl (%edx), %edx
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: leal 48(%esp), %esi
; NOCMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; NOCMOV-NEXT: jp [[TBB]]
; NOCMOV-NEXT: leal 32(%esp), %esi
; NOCMOV-NEXT: [[TBB]]:
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; NOCMOV-NEXT: movl (%esi), %esi
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV-NEXT: leal 52(%esp), %edi
; NOCMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; NOCMOV-NEXT: jp [[TBB]]
; NOCMOV-NEXT: leal 36(%esp), %edi
; NOCMOV-NEXT: [[TBB]]:
; NOCMOV-NEXT: movl (%edi), %edi
Re-land r237175: [X86] Always return the sret parameter in eax/rax ... This reverts commit r237210. Also fix X86/complex-fca.ll to match the code that we used to generate on win32 and now generate everwhere to conform to SysV. llvm-svn: 237639
2015-05-19 07:35:09 +08:00
; NOCMOV-NEXT: movl %edi, 12(%eax)
; NOCMOV-NEXT: movl %esi, 8(%eax)
; NOCMOV-NEXT: movl %edx, 4(%eax)
; NOCMOV-NEXT: movl %ecx, (%eax)
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; NOCMOV-NEXT: popl %esi
; NOCMOV-NEXT: popl %edi
; NOCMOV-NEXT: retl $4
define <4 x i32> @test_select_fcmp_oeq_v4i32(float %a, float %b, <4 x i32> %c, <4 x i32> %d) #0 {
entry:
%cmp = fcmp oeq float %a, %b
%r = select i1 %cmp, <4 x i32> %c, <4 x i32> %d
ret <4 x i32> %r
}
; Also make sure we catch the original code-sequence of interest:
; CMOV: [[ONE_F32_LCPI:.LCPI.*]]:
; CMOV-NEXT: .long 1065353216
; CHECK-LABEL: test_zext_fcmp_une:
; CMOV-NEXT: ucomiss %xmm1, %xmm0
; CMOV-NEXT: movss [[ONE_F32_LCPI]](%rip), %xmm0
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; CMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; CMOV-NEXT: jp [[TBB]]
; CMOV-NEXT: xorps %xmm0, %xmm0
; CMOV-NEXT: [[TBB]]:
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; CMOV-NEXT: retq
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV: jne
; NOCMOV-NEXT: jp
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
define float @test_zext_fcmp_une(float %a, float %b) #0 {
entry:
%cmp = fcmp une float %a, %b
%conv1 = zext i1 %cmp to i32
%conv2 = sitofp i32 %conv1 to float
ret float %conv2
}
; CMOV: [[ONE_F32_LCPI:.LCPI.*]]:
; CMOV-NEXT: .long 1065353216
; CHECK-LABEL: test_zext_fcmp_oeq:
; CMOV-NEXT: ucomiss %xmm1, %xmm0
; CMOV-NEXT: xorps %xmm0, %xmm0
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; CMOV-NEXT: jne [[TBB:.LBB[0-9_]+]]
; CMOV-NEXT: jp [[TBB]]
; CMOV-NEXT: movss [[ONE_F32_LCPI]](%rip), %xmm0
; CMOV-NEXT: [[TBB]]:
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
; CMOV-NEXT: retq
[X86] Special-case 2x CMOV when custom-inserting. This lets us avoid a few copies that are otherwise hard to get rid of. The way this is done is, the custom-inserter looks at the following instruction for another CMOV, and replaces both at the same time. A previous version used a new CMOV2 opcode, but the custom inserter is expected to be able to return a different basic block anyway, which means it's OK - though far from ideal - to alter that block's contents. Explicitly document that, in case it ever makes a difference. Alternatives welcome! Follow-up to r231045. rdar://19767934 Closes http://reviews.llvm.org/D8019 llvm-svn: 231046
2015-03-03 09:21:16 +08:00
; NOCMOV: jne
; NOCMOV-NEXT: jp
[X86] Combine (cmov (and/or (setcc) (setcc))) into (cmov (cmov)). Fold and/or of setcc's to double CMOV: (CMOV F, T, ((cc1 | cc2) != 0)) -> (CMOV (CMOV F, T, cc1), T, cc2) (CMOV F, T, ((cc1 & cc2) != 0)) -> (CMOV (CMOV T, F, !cc1), F, !cc2) When we can't use the CMOV instruction, it might increase branch mispredicts. When we can, or when there is no mispredict, this improves throughput and reduces register pressure. These can't be catched by generic combines, because the pattern can appear when legalizing some instructions (such as fcmp une). rdar://19767934 http://reviews.llvm.org/D7634 llvm-svn: 231045
2015-03-03 09:09:14 +08:00
define float @test_zext_fcmp_oeq(float %a, float %b) #0 {
entry:
%cmp = fcmp oeq float %a, %b
%conv1 = zext i1 %cmp to i32
%conv2 = sitofp i32 %conv1 to float
ret float %conv2
}
attributes #0 = { nounwind }
X86ISelLowering: Fix cmov(cmov) special lowering bug There's a special case in EmitLoweredSelect() that produces an improved lowering for cmov(cmov) patterns. However this special lowering is currently broken if the inner cmov has multiple users so this patch stops using it in this case. If you wonder why this wasn't fixed by continuing to use the special lowering and inserting a 2nd PHI for the inner cmov: I believe this would incur additional copies/register pressure so the special lowering does not improve upon the normal one anymore in this case. This fixes http://llvm.org/PR26256 (= rdar://24329747) llvm-svn: 258729
2016-01-26 06:08:25 +08:00
@g8 = global i8 0
; The following test failed because llvm had a bug where a structure like:
;
; %vreg12<def> = CMOV_GR8 %vreg7, %vreg11 ... (lt)
; %vreg13<def> = CMOV_GR8 %vreg12, %vreg11 ... (gt)
;
; was lowered to:
;
; The first two cmovs got expanded to:
; BB#0:
; JL_1 BB#9
; BB#7:
; JG_1 BB#9
; BB#8:
; BB#9:
; vreg12 = phi(vreg7, BB#8, vreg11, BB#0, vreg12, BB#7)
; vreg13 = COPY vreg12
; Which was invalid as %vreg12 is not the same value as %vreg13
; CHECK-LABEL: no_cascade_opt:
; CMOV-DAG: cmpl %edx, %esi
; CMOV-DAG: movb $20, %al
; CMOV-DAG: movb $20, %dl
CodeGen: BlockPlacement: Precompute layout for chains of triangles. For chains of triangles with small join blocks that can be tail duplicated, a simple calculation of probabilities is insufficient. Tail duplication can be profitable in 3 different ways for these cases: 1) The post-dominators marked 50% are actually taken 56% (This shrinks with longer chains) 2) The chains are statically correlated. Branch probabilities have a very U-shaped distribution. [http://nrs.harvard.edu/urn-3:HUL.InstRepos:24015805] If the branches in a chain are likely to be from the same side of the distribution as their predecessor, but are independent at runtime, this transformation is profitable. (Because the cost of being wrong is a small fixed cost, unlike the standard triangle layout where the cost of being wrong scales with the # of triangles.) 3) The chains are dynamically correlated. If the probability that a previous branch was taken positively influences whether the next branch will be taken We believe that 2 and 3 are common enough to justify the small margin in 1. The code pre-scans a function's CFG to identify this pattern and marks the edges so that the standard layout algorithm can use the computed results. llvm-svn: 296845
2017-03-03 09:00:22 +08:00
; CMOV: jge [[BB2:.LBB[0-9_]+]]
; CMOV: jle [[BB3:.LBB[0-9_]+]]
; CMOV: [[BB0:.LBB[0-9_]+]]
; CMOV: testl %edi, %edi
; CMOV: jne [[BB4:.LBB[0-9_]+]]
; CMOV: [[BB1:.LBB[0-9_]+]]
; CMOV: movb %al, g8(%rip)
; CMOV: retq
; CMOV: [[BB2]]:
[X86] Teach X86FixupBWInsts to promote MOV8rr/MOV16rr to MOV32rr. This re-applies r268760, reverted in r268794. Fixes http://llvm.org/PR27670 The original imp-defs assertion was way overzealous: forward all implicit operands, except imp-defs of the new super-reg def (r268787 for GR64, but also possible for GR16->GR32), or imp-uses of the new super-reg use. While there, mark the source use as Undef, and add an imp-use of the old source reg: that should cover any case of dead super-regs. At the stage the pass runs, flags are unlikely to matter anyway; still, let's be as correct as possible. Also add MIR tests for the various interesting cases. Original commit message: Codesize is less (16) or equal (8), and we avoid partial dependencies. Differential Revision: http://reviews.llvm.org/D19999 llvm-svn: 268831
2016-05-07 09:11:17 +08:00
; CMOV: movl %ecx, %edx
CodeGen: BlockPlacement: Precompute layout for chains of triangles. For chains of triangles with small join blocks that can be tail duplicated, a simple calculation of probabilities is insufficient. Tail duplication can be profitable in 3 different ways for these cases: 1) The post-dominators marked 50% are actually taken 56% (This shrinks with longer chains) 2) The chains are statically correlated. Branch probabilities have a very U-shaped distribution. [http://nrs.harvard.edu/urn-3:HUL.InstRepos:24015805] If the branches in a chain are likely to be from the same side of the distribution as their predecessor, but are independent at runtime, this transformation is profitable. (Because the cost of being wrong is a small fixed cost, unlike the standard triangle layout where the cost of being wrong scales with the # of triangles.) 3) The chains are dynamically correlated. If the probability that a previous branch was taken positively influences whether the next branch will be taken We believe that 2 and 3 are common enough to justify the small margin in 1. The code pre-scans a function's CFG to identify this pattern and marks the edges so that the standard layout algorithm can use the computed results. llvm-svn: 296845
2017-03-03 09:00:22 +08:00
; CMOV: jg [[BB0]]
; CMOV: [[BB3]]:
[X86] Teach X86FixupBWInsts to promote MOV8rr/MOV16rr to MOV32rr. This re-applies r268760, reverted in r268794. Fixes http://llvm.org/PR27670 The original imp-defs assertion was way overzealous: forward all implicit operands, except imp-defs of the new super-reg def (r268787 for GR64, but also possible for GR16->GR32), or imp-uses of the new super-reg use. While there, mark the source use as Undef, and add an imp-use of the old source reg: that should cover any case of dead super-regs. At the stage the pass runs, flags are unlikely to matter anyway; still, let's be as correct as possible. Also add MIR tests for the various interesting cases. Original commit message: Codesize is less (16) or equal (8), and we avoid partial dependencies. Differential Revision: http://reviews.llvm.org/D19999 llvm-svn: 268831
2016-05-07 09:11:17 +08:00
; CMOV: movl %edx, %eax
X86ISelLowering: Fix cmov(cmov) special lowering bug There's a special case in EmitLoweredSelect() that produces an improved lowering for cmov(cmov) patterns. However this special lowering is currently broken if the inner cmov has multiple users so this patch stops using it in this case. If you wonder why this wasn't fixed by continuing to use the special lowering and inserting a 2nd PHI for the inner cmov: I believe this would incur additional copies/register pressure so the special lowering does not improve upon the normal one anymore in this case. This fixes http://llvm.org/PR26256 (= rdar://24329747) llvm-svn: 258729
2016-01-26 06:08:25 +08:00
; CMOV: testl %edi, %edi
CodeGen: BlockPlacement: Precompute layout for chains of triangles. For chains of triangles with small join blocks that can be tail duplicated, a simple calculation of probabilities is insufficient. Tail duplication can be profitable in 3 different ways for these cases: 1) The post-dominators marked 50% are actually taken 56% (This shrinks with longer chains) 2) The chains are statically correlated. Branch probabilities have a very U-shaped distribution. [http://nrs.harvard.edu/urn-3:HUL.InstRepos:24015805] If the branches in a chain are likely to be from the same side of the distribution as their predecessor, but are independent at runtime, this transformation is profitable. (Because the cost of being wrong is a small fixed cost, unlike the standard triangle layout where the cost of being wrong scales with the # of triangles.) 3) The chains are dynamically correlated. If the probability that a previous branch was taken positively influences whether the next branch will be taken We believe that 2 and 3 are common enough to justify the small margin in 1. The code pre-scans a function's CFG to identify this pattern and marks the edges so that the standard layout algorithm can use the computed results. llvm-svn: 296845
2017-03-03 09:00:22 +08:00
; CMOV: je [[BB1]]
; CMOV: [[BB4]]:
[X86] Teach X86FixupBWInsts to promote MOV8rr/MOV16rr to MOV32rr. This re-applies r268760, reverted in r268794. Fixes http://llvm.org/PR27670 The original imp-defs assertion was way overzealous: forward all implicit operands, except imp-defs of the new super-reg def (r268787 for GR64, but also possible for GR16->GR32), or imp-uses of the new super-reg use. While there, mark the source use as Undef, and add an imp-use of the old source reg: that should cover any case of dead super-regs. At the stage the pass runs, flags are unlikely to matter anyway; still, let's be as correct as possible. Also add MIR tests for the various interesting cases. Original commit message: Codesize is less (16) or equal (8), and we avoid partial dependencies. Differential Revision: http://reviews.llvm.org/D19999 llvm-svn: 268831
2016-05-07 09:11:17 +08:00
; CMOV: movl %edx, %eax
X86ISelLowering: Fix cmov(cmov) special lowering bug There's a special case in EmitLoweredSelect() that produces an improved lowering for cmov(cmov) patterns. However this special lowering is currently broken if the inner cmov has multiple users so this patch stops using it in this case. If you wonder why this wasn't fixed by continuing to use the special lowering and inserting a 2nd PHI for the inner cmov: I believe this would incur additional copies/register pressure so the special lowering does not improve upon the normal one anymore in this case. This fixes http://llvm.org/PR26256 (= rdar://24329747) llvm-svn: 258729
2016-01-26 06:08:25 +08:00
; CMOV: movb %al, g8(%rip)
; CMOV: retq
define void @no_cascade_opt(i32 %v0, i32 %v1, i32 %v2, i32 %v3) {
entry:
%c0 = icmp eq i32 %v0, 0
%c1 = icmp slt i32 %v1, %v2
%c2 = icmp sgt i32 %v1, %v2
%trunc = trunc i32 %v3 to i8
%sel0 = select i1 %c1, i8 20, i8 %trunc
%sel1 = select i1 %c2, i8 20, i8 %sel0
%sel2 = select i1 %c0, i8 %sel1, i8 %sel0
store volatile i8 %sel2, i8* @g8
ret void
}