maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/memcmp.ll

910 lines
26 KiB
LLVM
Raw Normal View History

[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[X86] Added cmov target to memcmp test As discussed by @spatel on D35067: "I added the cmov attribute to the 32-bit codegen test because it removes some noise for that file. I think the intent for the SSE vs no-SSE runs is to show the potential difference for the 16 and 32 byte cases rather than the lack of cmov (which has been available for all CPUs since SSE1, so that's why it shows up automatically with -mattr=sse2)." llvm-svn: 308309
2017-07-18 22:19:34 +08:00
; RUN: llc < %s -mtriple=i686-unknown-unknown -mattr=cmov | FileCheck %s --check-prefix=X86 --check-prefix=X86-NOSSE
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; RUN: llc < %s -mtriple=i686-unknown-unknown -mattr=+sse2 | FileCheck %s --check-prefix=X86 --check-prefix=X86-SSE2
; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s --check-prefix=X64 --check-prefix=X64-SSE2
[X86] Added cmov target to memcmp test As discussed by @spatel on D35067: "I added the cmov attribute to the 32-bit codegen test because it removes some noise for that file. I think the intent for the SSE vs no-SSE runs is to show the potential difference for the 16 and 32 byte cases rather than the lack of cmov (which has been available for all CPUs since SSE1, so that's why it shows up automatically with -mattr=sse2)." llvm-svn: 308309
2017-07-18 22:19:34 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=avx2 | FileCheck %s --check-prefix=X64 --check-prefix=X64-AVX2
move an optimization for memcmp out of simplifylibcalls and into SDISel. This optimization was causing simplifylibcalls to introduce type-unsafe nastiness. This is the first step, I'll be expanding the memcmp optimizations shortly, covering things that we really really wouldn't want simplifylibcalls to do. llvm-svn: 92098
2009-12-24 08:37:38 +08:00
; This tests codegen time inlining/optimization of memcmp
; rdar://6480398
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
@.str = private constant [65 x i8] c"0123456789012345678901234567890123456789012345678901234567890123\00", align 1
move an optimization for memcmp out of simplifylibcalls and into SDISel. This optimization was causing simplifylibcalls to introduce type-unsafe nastiness. This is the first step, I'll be expanding the memcmp optimizations shortly, covering things that we really really wouldn't want simplifylibcalls to do. llvm-svn: 92098
2009-12-24 08:37:38 +08:00
[TLI] Robustize SDAG LibFunc proto checking by merging it into TLI. This re-applies commit r292189, reverted in r292191. SelectionDAGBuilder recognizes libfuncs using some homegrown parameter type-checking. Use TLI instead, removing another heap of redundant code. This isn't strictly NFC, as the SDAG code was too lax. Concretely, this means changes are required to a few tests: - calling a non-variadic function via a variadic prototype isn't OK; it just happens to work on x86_64 (but not on, e.g., aarch64). - mempcpy has a size_t parameter; the SDAG code accepts any integer type, which meant using i32 on x86_64 worked. - a handful of SystemZ tests check the SDAG support for lax prototype checking: Ulrich agrees on removing them. I don't think it's worth supporting any of these (IMO) invalid testcases. Instead, fix them to be more meaningful. llvm-svn: 294028
2017-02-04 03:11:19 +08:00
declare i32 @memcmp(i8*, i8*, i64)
move an optimization for memcmp out of simplifylibcalls and into SDISel. This optimization was causing simplifylibcalls to introduce type-unsafe nastiness. This is the first step, I'll be expanding the memcmp optimizations shortly, covering things that we really really wouldn't want simplifylibcalls to do. llvm-svn: 92098
2009-12-24 08:37:38 +08:00
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
define i32 @length2(i8* %X, i8* %Y) nounwind {
[X86] Added cmov target to memcmp test As discussed by @spatel on D35067: "I added the cmov attribute to the 32-bit codegen test because it removes some noise for that file. I think the intent for the SSE vs no-SSE runs is to show the potential difference for the 16 and 32 byte cases rather than the lack of cmov (which has been available for all CPUs since SSE1, so that's why it shows up automatically with -mattr=sse2)." llvm-svn: 308309
2017-07-18 22:19:34 +08:00
; X86-LABEL: length2:
; X86: # BB#0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movzwl (%ecx), %ecx
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X86-NEXT: movzwl (%eax), %edx
[X86] Added cmov target to memcmp test As discussed by @spatel on D35067: "I added the cmov attribute to the 32-bit codegen test because it removes some noise for that file. I think the intent for the SSE vs no-SSE runs is to show the potential difference for the 16 and 32 byte cases rather than the lack of cmov (which has been available for all CPUs since SSE1, so that's why it shows up automatically with -mattr=sse2)." llvm-svn: 308309
2017-07-18 22:19:34 +08:00
; X86-NEXT: rolw $8, %cx
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X86-NEXT: rolw $8, %dx
; X86-NEXT: movzwl %cx, %eax
; X86-NEXT: movzwl %dx, %ecx
; X86-NEXT: subl %ecx, %eax
[X86] Added cmov target to memcmp test As discussed by @spatel on D35067: "I added the cmov attribute to the 32-bit codegen test because it removes some noise for that file. I think the intent for the SSE vs no-SSE runs is to show the potential difference for the 16 and 32 byte cases rather than the lack of cmov (which has been available for all CPUs since SSE1, so that's why it shows up automatically with -mattr=sse2)." llvm-svn: 308309
2017-07-18 22:19:34 +08:00
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length2:
[CGP] add specialization for memcmp expansion with only one basic block llvm-svn: 306485
2017-06-28 07:15:01 +08:00
; X64: # BB#0:
[x86] enable CGP memcmp() expansion for 2/4/8 byte sizes There are a couple of potential improvements as seen in the IR and asm: 1. We're unnecessarily extending to a larger type to compare values. 2. The codegen for (select cond, 1, -1) could avoid a cmov. (or we could change the order of the compares, so we have a select with 0 operand) llvm-svn: 305802
2017-06-20 23:58:30 +08:00
; X64-NEXT: movzwl (%rdi), %eax
; X64-NEXT: movzwl (%rsi), %ecx
; X64-NEXT: rolw $8, %ax
; X64-NEXT: rolw $8, %cx
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X64-NEXT: movzwl %ax, %eax
; X64-NEXT: movzwl %cx, %ecx
; X64-NEXT: subl %ecx, %eax
[x86] enable CGP memcmp() expansion for 2/4/8 byte sizes There are a couple of potential improvements as seen in the IR and asm: 1. We're unnecessarily extending to a larger type to compare values. 2. The codegen for (select cond, 1, -1) could avoid a cmov. (or we could change the order of the compares, so we have a select with 0 operand) llvm-svn: 305802
2017-06-20 23:58:30 +08:00
; X64-NEXT: retq
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 2) nounwind
ret i32 %m
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length2_eq(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length2_eq:
; X86: # BB#0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movzwl (%ecx), %ecx
; X86-NEXT: cmpw (%eax), %cx
; X86-NEXT: sete %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length2_eq:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: movzwl (%rdi), %eax
; X64-NEXT: cmpw (%rsi), %ax
; X64-NEXT: sete %al
; X64-NEXT: retq
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 2) nounwind
%c = icmp eq i32 %m, 0
ret i1 %c
move an optimization for memcmp out of simplifylibcalls and into SDISel. This optimization was causing simplifylibcalls to introduce type-unsafe nastiness. This is the first step, I'll be expanding the memcmp optimizations shortly, covering things that we really really wouldn't want simplifylibcalls to do. llvm-svn: 92098
2009-12-24 08:37:38 +08:00
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length2_eq_const(i8* %X) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length2_eq_const:
; X86: # BB#0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movzwl (%eax), %eax
; X86-NEXT: cmpl $12849, %eax # imm = 0x3231
; X86-NEXT: setne %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length2_eq_const:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: movzwl (%rdi), %eax
; X64-NEXT: cmpl $12849, %eax # imm = 0x3231
; X64-NEXT: setne %al
; X64-NEXT: retq
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* getelementptr inbounds ([65 x i8], [65 x i8]* @.str, i32 0, i32 1), i64 2) nounwind
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%c = icmp ne i32 %m, 0
ret i1 %c
move an optimization for memcmp out of simplifylibcalls and into SDISel. This optimization was causing simplifylibcalls to introduce type-unsafe nastiness. This is the first step, I'll be expanding the memcmp optimizations shortly, covering things that we really really wouldn't want simplifylibcalls to do. llvm-svn: 92098
2009-12-24 08:37:38 +08:00
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length2_eq_nobuiltin_attr(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length2_eq_nobuiltin_attr:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $2
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: testl %eax, %eax
; X86-NEXT: sete %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length2_eq_nobuiltin_attr:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: pushq %rax
; X64-NEXT: movl $2, %edx
; X64-NEXT: callq memcmp
; X64-NEXT: testl %eax, %eax
; X64-NEXT: sete %al
; X64-NEXT: popq %rcx
; X64-NEXT: retq
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 2) nounwind nobuiltin
%c = icmp eq i32 %m, 0
ret i1 %c
[SelectionDAG] Don't treat library calls specially if marked with nobuiltin. To be used by D19781. Differential Revision: http://reviews.llvm.org/D19801 llvm-svn: 273039
2016-06-18 04:24:07 +08:00
}
move an optimization for memcmp out of simplifylibcalls and into SDISel. This optimization was causing simplifylibcalls to introduce type-unsafe nastiness. This is the first step, I'll be expanding the memcmp optimizations shortly, covering things that we really really wouldn't want simplifylibcalls to do. llvm-svn: 92098
2009-12-24 08:37:38 +08:00
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
define i32 @length3(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length3:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X86: # BB#0: # %loadbb
; X86-NEXT: pushl %esi
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movzwl (%eax), %edx
; X86-NEXT: movzwl (%ecx), %esi
; X86-NEXT: rolw $8, %dx
; X86-NEXT: rolw $8, %si
[CGP] use narrower types in memcmp expansion when possible This only affects very small memcmp on x86 for now, but it will become more important if we allow vector-sized load and compares. llvm-svn: 309711
2017-08-02 01:24:54 +08:00
; X86-NEXT: cmpw %si, %dx
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X86-NEXT: jne .LBB4_1
; X86-NEXT: # BB#2: # %loadbb1
; X86-NEXT: movzbl 2(%eax), %eax
; X86-NEXT: movzbl 2(%ecx), %ecx
; X86-NEXT: subl %ecx, %eax
; X86-NEXT: popl %esi
; X86-NEXT: retl
; X86-NEXT: .LBB4_1: # %res_block
; X86-NEXT: movl $-1, %ecx
; X86-NEXT: movl $1, %eax
; X86-NEXT: cmovbl %ecx, %eax
; X86-NEXT: popl %esi
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length3:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64: # BB#0: # %loadbb
; X64-NEXT: movzwl (%rdi), %eax
; X64-NEXT: movzwl (%rsi), %ecx
; X64-NEXT: rolw $8, %ax
; X64-NEXT: rolw $8, %cx
[CGP] use narrower types in memcmp expansion when possible This only affects very small memcmp on x86 for now, but it will become more important if we allow vector-sized load and compares. llvm-svn: 309711
2017-08-02 01:24:54 +08:00
; X64-NEXT: cmpw %cx, %ax
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64-NEXT: jne .LBB4_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: movzbl 2(%rdi), %eax
; X64-NEXT: movzbl 2(%rsi), %ecx
; X64-NEXT: subl %ecx, %eax
; X64-NEXT: retq
; X64-NEXT: .LBB4_1: # %res_block
; X64-NEXT: movl $-1, %ecx
; X64-NEXT: movl $1, %eax
; X64-NEXT: cmovbl %ecx, %eax
; X64-NEXT: retq
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 3) nounwind
ret i32 %m
}
define i1 @length3_eq(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length3_eq:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X86: # BB#0: # %loadbb
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movzwl (%eax), %edx
; X86-NEXT: cmpw (%ecx), %dx
; X86-NEXT: jne .LBB5_1
; X86-NEXT: # BB#2: # %loadbb1
; X86-NEXT: movb 2(%eax), %dl
; X86-NEXT: xorl %eax, %eax
; X86-NEXT: cmpb 2(%ecx), %dl
; X86-NEXT: je .LBB5_3
; X86-NEXT: .LBB5_1: # %res_block
; X86-NEXT: movl $1, %eax
; X86-NEXT: .LBB5_3: # %endblock
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: testl %eax, %eax
; X86-NEXT: setne %al
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length3_eq:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64: # BB#0: # %loadbb
; X64-NEXT: movzwl (%rdi), %eax
; X64-NEXT: cmpw (%rsi), %ax
; X64-NEXT: jne .LBB5_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: movb 2(%rdi), %cl
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: cmpb 2(%rsi), %cl
; X64-NEXT: je .LBB5_3
; X64-NEXT: .LBB5_1: # %res_block
; X64-NEXT: movl $1, %eax
; X64-NEXT: .LBB5_3: # %endblock
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-NEXT: testl %eax, %eax
; X64-NEXT: setne %al
; X64-NEXT: retq
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 3) nounwind
%c = icmp ne i32 %m, 0
ret i1 %c
}
define i32 @length4(i8* %X, i8* %Y) nounwind {
[X86] Added cmov target to memcmp test As discussed by @spatel on D35067: "I added the cmov attribute to the 32-bit codegen test because it removes some noise for that file. I think the intent for the SSE vs no-SSE runs is to show the potential difference for the 16 and 32 byte cases rather than the lack of cmov (which has been available for all CPUs since SSE1, so that's why it shows up automatically with -mattr=sse2)." llvm-svn: 308309
2017-07-18 22:19:34 +08:00
; X86-LABEL: length4:
; X86: # BB#0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl (%ecx), %ecx
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X86-NEXT: movl (%eax), %edx
[X86] Added cmov target to memcmp test As discussed by @spatel on D35067: "I added the cmov attribute to the 32-bit codegen test because it removes some noise for that file. I think the intent for the SSE vs no-SSE runs is to show the potential difference for the 16 and 32 byte cases rather than the lack of cmov (which has been available for all CPUs since SSE1, so that's why it shows up automatically with -mattr=sse2)." llvm-svn: 308309
2017-07-18 22:19:34 +08:00
; X86-NEXT: bswapl %ecx
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X86-NEXT: bswapl %edx
; X86-NEXT: xorl %eax, %eax
; X86-NEXT: cmpl %edx, %ecx
; X86-NEXT: seta %al
; X86-NEXT: sbbl $0, %eax
[X86] Added cmov target to memcmp test As discussed by @spatel on D35067: "I added the cmov attribute to the 32-bit codegen test because it removes some noise for that file. I think the intent for the SSE vs no-SSE runs is to show the potential difference for the 16 and 32 byte cases rather than the lack of cmov (which has been available for all CPUs since SSE1, so that's why it shows up automatically with -mattr=sse2)." llvm-svn: 308309
2017-07-18 22:19:34 +08:00
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length4:
[CGP] add specialization for memcmp expansion with only one basic block llvm-svn: 306485
2017-06-28 07:15:01 +08:00
; X64: # BB#0:
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X64-NEXT: movl (%rdi), %ecx
; X64-NEXT: movl (%rsi), %edx
[x86] enable CGP memcmp() expansion for 2/4/8 byte sizes There are a couple of potential improvements as seen in the IR and asm: 1. We're unnecessarily extending to a larger type to compare values. 2. The codegen for (select cond, 1, -1) could avoid a cmov. (or we could change the order of the compares, so we have a select with 0 operand) llvm-svn: 305802
2017-06-20 23:58:30 +08:00
; X64-NEXT: bswapl %ecx
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X64-NEXT: bswapl %edx
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: cmpl %edx, %ecx
; X64-NEXT: seta %al
; X64-NEXT: sbbl $0, %eax
[x86] enable CGP memcmp() expansion for 2/4/8 byte sizes There are a couple of potential improvements as seen in the IR and asm: 1. We're unnecessarily extending to a larger type to compare values. 2. The codegen for (select cond, 1, -1) could avoid a cmov. (or we could change the order of the compares, so we have a select with 0 operand) llvm-svn: 305802
2017-06-20 23:58:30 +08:00
; X64-NEXT: retq
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 4) nounwind
ret i32 %m
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length4_eq(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length4_eq:
; X86: # BB#0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl (%ecx), %ecx
; X86-NEXT: cmpl (%eax), %ecx
; X86-NEXT: setne %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length4_eq:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: movl (%rdi), %eax
; X64-NEXT: cmpl (%rsi), %eax
; X64-NEXT: setne %al
; X64-NEXT: retq
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 4) nounwind
%c = icmp ne i32 %m, 0
ret i1 %c
move an optimization for memcmp out of simplifylibcalls and into SDISel. This optimization was causing simplifylibcalls to introduce type-unsafe nastiness. This is the first step, I'll be expanding the memcmp optimizations shortly, covering things that we really really wouldn't want simplifylibcalls to do. llvm-svn: 92098
2009-12-24 08:37:38 +08:00
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length4_eq_const(i8* %X) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length4_eq_const:
; X86: # BB#0:
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: cmpl $875770417, (%eax) # imm = 0x34333231
; X86-NEXT: sete %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length4_eq_const:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: cmpl $875770417, (%rdi) # imm = 0x34333231
; X64-NEXT: sete %al
; X64-NEXT: retq
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* getelementptr inbounds ([65 x i8], [65 x i8]* @.str, i32 0, i32 1), i64 4) nounwind
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%c = icmp eq i32 %m, 0
ret i1 %c
handle equality memcmp of 8 bytes on x86-64 with two unaligned loads and a compare. On other targets we end up with a call to memcmp because we don't want 16 individual byte loads. We should be able to use movups as well, but we're failing to select the generated icmp. llvm-svn: 92107
2009-12-24 09:07:17 +08:00
}
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
define i32 @length5(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length5:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X86: # BB#0: # %loadbb
; X86-NEXT: pushl %esi
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movl (%eax), %edx
; X86-NEXT: movl (%ecx), %esi
; X86-NEXT: bswapl %edx
; X86-NEXT: bswapl %esi
; X86-NEXT: cmpl %esi, %edx
; X86-NEXT: jne .LBB9_1
; X86-NEXT: # BB#2: # %loadbb1
; X86-NEXT: movzbl 4(%eax), %eax
; X86-NEXT: movzbl 4(%ecx), %ecx
; X86-NEXT: subl %ecx, %eax
; X86-NEXT: popl %esi
; X86-NEXT: retl
; X86-NEXT: .LBB9_1: # %res_block
; X86-NEXT: movl $-1, %ecx
; X86-NEXT: movl $1, %eax
; X86-NEXT: cmovbl %ecx, %eax
; X86-NEXT: popl %esi
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length5:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64: # BB#0: # %loadbb
; X64-NEXT: movl (%rdi), %eax
; X64-NEXT: movl (%rsi), %ecx
; X64-NEXT: bswapl %eax
; X64-NEXT: bswapl %ecx
[CGP] use narrower types in memcmp expansion when possible This only affects very small memcmp on x86 for now, but it will become more important if we allow vector-sized load and compares. llvm-svn: 309711
2017-08-02 01:24:54 +08:00
; X64-NEXT: cmpl %ecx, %eax
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64-NEXT: jne .LBB9_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: movzbl 4(%rdi), %eax
; X64-NEXT: movzbl 4(%rsi), %ecx
; X64-NEXT: subl %ecx, %eax
; X64-NEXT: retq
; X64-NEXT: .LBB9_1: # %res_block
; X64-NEXT: movl $-1, %ecx
; X64-NEXT: movl $1, %eax
; X64-NEXT: cmovbl %ecx, %eax
; X64-NEXT: retq
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 5) nounwind
ret i32 %m
}
define i1 @length5_eq(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length5_eq:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X86: # BB#0: # %loadbb
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movl (%eax), %edx
; X86-NEXT: cmpl (%ecx), %edx
; X86-NEXT: jne .LBB10_1
; X86-NEXT: # BB#2: # %loadbb1
; X86-NEXT: movb 4(%eax), %dl
; X86-NEXT: xorl %eax, %eax
; X86-NEXT: cmpb 4(%ecx), %dl
; X86-NEXT: je .LBB10_3
; X86-NEXT: .LBB10_1: # %res_block
; X86-NEXT: movl $1, %eax
; X86-NEXT: .LBB10_3: # %endblock
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: testl %eax, %eax
; X86-NEXT: setne %al
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length5_eq:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64: # BB#0: # %loadbb
; X64-NEXT: movl (%rdi), %eax
; X64-NEXT: cmpl (%rsi), %eax
; X64-NEXT: jne .LBB10_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: movb 4(%rdi), %cl
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: cmpb 4(%rsi), %cl
; X64-NEXT: je .LBB10_3
; X64-NEXT: .LBB10_1: # %res_block
; X64-NEXT: movl $1, %eax
; X64-NEXT: .LBB10_3: # %endblock
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-NEXT: testl %eax, %eax
; X64-NEXT: setne %al
; X64-NEXT: retq
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 5) nounwind
%c = icmp ne i32 %m, 0
ret i1 %c
}
define i32 @length8(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length8:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X86: # BB#0: # %loadbb
; X86-NEXT: pushl %esi
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movl {{[0-9]+}}(%esp), %esi
; X86-NEXT: movl (%esi), %ecx
; X86-NEXT: movl (%eax), %edx
; X86-NEXT: bswapl %ecx
; X86-NEXT: bswapl %edx
; X86-NEXT: cmpl %edx, %ecx
; X86-NEXT: jne .LBB11_1
; X86-NEXT: # BB#2: # %loadbb1
; X86-NEXT: movl 4(%esi), %ecx
; X86-NEXT: movl 4(%eax), %edx
; X86-NEXT: bswapl %ecx
; X86-NEXT: bswapl %edx
; X86-NEXT: xorl %eax, %eax
; X86-NEXT: cmpl %edx, %ecx
; X86-NEXT: jne .LBB11_1
; X86-NEXT: # BB#3: # %endblock
; X86-NEXT: popl %esi
; X86-NEXT: retl
; X86-NEXT: .LBB11_1: # %res_block
; X86-NEXT: cmpl %edx, %ecx
; X86-NEXT: movl $-1, %ecx
; X86-NEXT: movl $1, %eax
; X86-NEXT: cmovbl %ecx, %eax
; X86-NEXT: popl %esi
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length8:
[CGP] add specialization for memcmp expansion with only one basic block llvm-svn: 306485
2017-06-28 07:15:01 +08:00
; X64: # BB#0:
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X64-NEXT: movq (%rdi), %rcx
; X64-NEXT: movq (%rsi), %rdx
[x86] enable CGP memcmp() expansion for 2/4/8 byte sizes There are a couple of potential improvements as seen in the IR and asm: 1. We're unnecessarily extending to a larger type to compare values. 2. The codegen for (select cond, 1, -1) could avoid a cmov. (or we could change the order of the compares, so we have a select with 0 operand) llvm-svn: 305802
2017-06-20 23:58:30 +08:00
; X64-NEXT: bswapq %rcx
[CGP] use subtract or subtract-of-cmps for result of memcmp expansion As noted in the code comment, transforming this in the other direction might require a separate transform here in CGP given the block-at-a-time DAG constraint. Besides that theoretical motivation, there are 2 practical motivations for the subtract-of-cmps form: 1. The codegen for both x86 and PPC is better for this IR (though PPC could be better still). There is discussion about canonicalizing IR to the select form ( http://lists.llvm.org/pipermail/llvm-dev/2017-July/114885.html ), so we probably need to add DAG transforms for those patterns anyway, but this improves the memcmp output without waiting for that step. 2. If we allow vector-sized chunks for the load and compare, x86 is better prepared to convert that to optimal code when using subtract-of-cmps, so another prerequisite patch is avoided if we choose to enable that. Differential Revision: https://reviews.llvm.org/D34904 llvm-svn: 309597
2017-08-01 02:08:24 +08:00
; X64-NEXT: bswapq %rdx
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: cmpq %rdx, %rcx
; X64-NEXT: seta %al
; X64-NEXT: sbbl $0, %eax
[x86] enable CGP memcmp() expansion for 2/4/8 byte sizes There are a couple of potential improvements as seen in the IR and asm: 1. We're unnecessarily extending to a larger type to compare values. 2. The codegen for (select cond, 1, -1) could avoid a cmov. (or we could change the order of the compares, so we have a select with 0 operand) llvm-svn: 305802
2017-06-20 23:58:30 +08:00
; X64-NEXT: retq
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 8) nounwind
ret i32 %m
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length8_eq(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length8_eq:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X86: # BB#0: # %loadbb
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NEXT: movl (%eax), %edx
; X86-NEXT: cmpl (%ecx), %edx
; X86-NEXT: jne .LBB12_1
; X86-NEXT: # BB#2: # %loadbb1
; X86-NEXT: movl 4(%eax), %edx
; X86-NEXT: xorl %eax, %eax
; X86-NEXT: cmpl 4(%ecx), %edx
; X86-NEXT: je .LBB12_3
; X86-NEXT: .LBB12_1: # %res_block
; X86-NEXT: movl $1, %eax
; X86-NEXT: .LBB12_3: # %endblock
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: testl %eax, %eax
; X86-NEXT: sete %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length8_eq:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: movq (%rdi), %rax
; X64-NEXT: cmpq (%rsi), %rax
; X64-NEXT: sete %al
; X64-NEXT: retq
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 8) nounwind
%c = icmp eq i32 %m, 0
ret i1 %c
handle equality memcmp of 8 bytes on x86-64 with two unaligned loads and a compare. On other targets we end up with a call to memcmp because we don't want 16 individual byte loads. We should be able to use movups as well, but we're failing to select the generated icmp. llvm-svn: 92107
2009-12-24 09:07:17 +08:00
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length8_eq_const(i8* %X) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length8_eq_const:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X86: # BB#0: # %loadbb
; X86-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-NEXT: cmpl $858927408, (%ecx) # imm = 0x33323130
; X86-NEXT: jne .LBB13_1
; X86-NEXT: # BB#2: # %loadbb1
; X86-NEXT: xorl %eax, %eax
; X86-NEXT: cmpl $926299444, 4(%ecx) # imm = 0x37363534
; X86-NEXT: je .LBB13_3
; X86-NEXT: .LBB13_1: # %res_block
; X86-NEXT: movl $1, %eax
; X86-NEXT: .LBB13_3: # %endblock
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: testl %eax, %eax
; X86-NEXT: setne %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length8_eq_const:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: movabsq $3978425819141910832, %rax # imm = 0x3736353433323130
; X64-NEXT: cmpq %rax, (%rdi)
; X64-NEXT: setne %al
; X64-NEXT: retq
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* getelementptr inbounds ([65 x i8], [65 x i8]* @.str, i32 0, i32 0), i64 8) nounwind
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%c = icmp ne i32 %m, 0
ret i1 %c
}
handle equality memcmp of 8 bytes on x86-64 with two unaligned loads and a compare. On other targets we end up with a call to memcmp because we don't want 16 individual byte loads. We should be able to use movups as well, but we're failing to select the generated icmp. llvm-svn: 92107
2009-12-24 09:07:17 +08:00
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
define i1 @length12_eq(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length12_eq:
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $12
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: testl %eax, %eax
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: setne %al
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length12_eq:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64: # BB#0: # %loadbb
; X64-NEXT: movq (%rdi), %rax
; X64-NEXT: cmpq (%rsi), %rax
; X64-NEXT: jne .LBB14_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: movl 8(%rdi), %ecx
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: cmpl 8(%rsi), %ecx
; X64-NEXT: je .LBB14_3
; X64-NEXT: .LBB14_1: # %res_block
; X64-NEXT: movl $1, %eax
; X64-NEXT: .LBB14_3: # %endblock
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-NEXT: testl %eax, %eax
; X64-NEXT: setne %al
; X64-NEXT: retq
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 12) nounwind
%c = icmp ne i32 %m, 0
ret i1 %c
}
define i32 @length12(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length12:
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $12
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length12:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64: # BB#0: # %loadbb
; X64-NEXT: movq (%rdi), %rcx
; X64-NEXT: movq (%rsi), %rdx
; X64-NEXT: bswapq %rcx
; X64-NEXT: bswapq %rdx
; X64-NEXT: cmpq %rdx, %rcx
; X64-NEXT: jne .LBB15_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: movl 8(%rdi), %ecx
; X64-NEXT: movl 8(%rsi), %edx
; X64-NEXT: bswapl %ecx
; X64-NEXT: bswapl %edx
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: cmpq %rdx, %rcx
; X64-NEXT: jne .LBB15_1
; X64-NEXT: # BB#3: # %endblock
; X64-NEXT: retq
; X64-NEXT: .LBB15_1: # %res_block
; X64-NEXT: cmpq %rdx, %rcx
; X64-NEXT: movl $-1, %ecx
; X64-NEXT: movl $1, %eax
; X64-NEXT: cmovbl %ecx, %eax
; X64-NEXT: retq
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 12) nounwind
ret i32 %m
}
; PR33329 - https://bugs.llvm.org/show_bug.cgi?id=33329
define i32 @length16(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length16:
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $16
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length16:
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64: # BB#0: # %loadbb
; X64-NEXT: movq (%rdi), %rcx
; X64-NEXT: movq (%rsi), %rdx
; X64-NEXT: bswapq %rcx
; X64-NEXT: bswapq %rdx
; X64-NEXT: cmpq %rdx, %rcx
; X64-NEXT: jne .LBB16_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: movq 8(%rdi), %rcx
; X64-NEXT: movq 8(%rsi), %rdx
; X64-NEXT: bswapq %rcx
; X64-NEXT: bswapq %rdx
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: cmpq %rdx, %rcx
; X64-NEXT: jne .LBB16_1
; X64-NEXT: # BB#3: # %endblock
; X64-NEXT: retq
; X64-NEXT: .LBB16_1: # %res_block
; X64-NEXT: cmpq %rdx, %rcx
; X64-NEXT: movl $-1, %ecx
; X64-NEXT: movl $1, %eax
; X64-NEXT: cmovbl %ecx, %eax
; X64-NEXT: retq
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 16) nounwind
ret i32 %m
}
define i1 @length16_eq(i8* %x, i8* %y) nounwind {
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X86-NOSSE-LABEL: length16_eq:
; X86-NOSSE: # BB#0:
; X86-NOSSE-NEXT: pushl $0
; X86-NOSSE-NEXT: pushl $16
; X86-NOSSE-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NOSSE-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NOSSE-NEXT: calll memcmp
; X86-NOSSE-NEXT: addl $16, %esp
; X86-NOSSE-NEXT: testl %eax, %eax
; X86-NOSSE-NEXT: setne %al
; X86-NOSSE-NEXT: retl
;
; X86-SSE2-LABEL: length16_eq:
; X86-SSE2: # BB#0:
; X86-SSE2-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-SSE2-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-SSE2-NEXT: movdqu (%ecx), %xmm0
; X86-SSE2-NEXT: movdqu (%eax), %xmm1
; X86-SSE2-NEXT: pcmpeqb %xmm0, %xmm1
; X86-SSE2-NEXT: pmovmskb %xmm1, %eax
; X86-SSE2-NEXT: cmpl $65535, %eax # imm = 0xFFFF
; X86-SSE2-NEXT: setne %al
; X86-SSE2-NEXT: retl
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
;
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64-LABEL: length16_eq:
; X64: # BB#0: # %loadbb
; X64-NEXT: movq (%rdi), %rax
; X64-NEXT: cmpq (%rsi), %rax
; X64-NEXT: jne .LBB17_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: movq 8(%rdi), %rcx
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: cmpq 8(%rsi), %rcx
; X64-NEXT: je .LBB17_3
; X64-NEXT: .LBB17_1: # %res_block
; X64-NEXT: movl $1, %eax
; X64-NEXT: .LBB17_3: # %endblock
; X64-NEXT: testl %eax, %eax
; X64-NEXT: setne %al
; X64-NEXT: retq
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%call = tail call i32 @memcmp(i8* %x, i8* %y, i64 16) nounwind
%cmp = icmp ne i32 %call, 0
ret i1 %cmp
}
handle equality memcmp of 8 bytes on x86-64 with two unaligned loads and a compare. On other targets we end up with a call to memcmp because we don't want 16 individual byte loads. We should be able to use movups as well, but we're failing to select the generated icmp. llvm-svn: 92107
2009-12-24 09:07:17 +08:00
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length16_eq_const(i8* %X) nounwind {
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X86-NOSSE-LABEL: length16_eq_const:
; X86-NOSSE: # BB#0:
; X86-NOSSE-NEXT: pushl $0
; X86-NOSSE-NEXT: pushl $16
; X86-NOSSE-NEXT: pushl $.L.str
; X86-NOSSE-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NOSSE-NEXT: calll memcmp
; X86-NOSSE-NEXT: addl $16, %esp
; X86-NOSSE-NEXT: testl %eax, %eax
; X86-NOSSE-NEXT: sete %al
; X86-NOSSE-NEXT: retl
;
; X86-SSE2-LABEL: length16_eq_const:
; X86-SSE2: # BB#0:
; X86-SSE2-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-SSE2-NEXT: movdqu (%eax), %xmm0
; X86-SSE2-NEXT: pcmpeqb {{\.LCPI.*}}, %xmm0
; X86-SSE2-NEXT: pmovmskb %xmm0, %eax
; X86-SSE2-NEXT: cmpl $65535, %eax # imm = 0xFFFF
; X86-SSE2-NEXT: sete %al
; X86-SSE2-NEXT: retl
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
;
[x86, CGP] increase memcmp() expansion up to 4 load pairs It should be a win to avoid going out to the system lib for all small memcmp() calls using scalar ops. For x86 32-bit, this means most everything up to 16 bytes. For 64-bit, that doubles because we can do 8-byte loads. Notes: Reduced from 4 to 2 loads for -Os behavior, which might not be optimal in all cases. It's effectively a question of how much do we trust the system implementation. Linux and macOS (and Windows I assume, but did not test) have optimized memcmp() code for x86, so it's probably not bad either way? PPC is using 8/4 for defaults on these. We do not expand at all for -Oz. There are still potential improvements to make for the CGP expansion IR and/or lowering such as avoiding select-of-constants (D34904) and not doing zexts to the max load type before doing a compare. We have special-case SSE/AVX codegen for (memcmp(x, y, 16/32) == 0) that will no longer be produced after this patch. I've shown the experimental justification for that change in PR33329: https://bugs.llvm.org/show_bug.cgi?id=33329#c12 TLDR: While the vector code is a likely winner, we can't guarantee that it's a winner in all cases on all CPUs, so I'm willing to sacrifice it for the greater good of expanding all small memcmp(). If we want to resurrect that codegen, it can be done by adjusting the CGP params or poking a hole to let those fall-through the CGP expansion. Committed on behalf of Sanjay Patel Differential Revision: https://reviews.llvm.org/D35067 llvm-svn: 308322
2017-07-18 23:55:30 +08:00
; X64-LABEL: length16_eq_const:
; X64: # BB#0: # %loadbb
; X64-NEXT: movabsq $3978425819141910832, %rax # imm = 0x3736353433323130
; X64-NEXT: cmpq %rax, (%rdi)
; X64-NEXT: jne .LBB18_1
; X64-NEXT: # BB#2: # %loadbb1
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: movabsq $3833745473465760056, %rcx # imm = 0x3534333231303938
; X64-NEXT: cmpq %rcx, 8(%rdi)
; X64-NEXT: je .LBB18_3
; X64-NEXT: .LBB18_1: # %res_block
; X64-NEXT: movl $1, %eax
; X64-NEXT: .LBB18_3: # %endblock
; X64-NEXT: testl %eax, %eax
; X64-NEXT: sete %al
; X64-NEXT: retq
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* getelementptr inbounds ([65 x i8], [65 x i8]* @.str, i32 0, i32 0), i64 16) nounwind
%c = icmp eq i32 %m, 0
ret i1 %c
}
[X86] Add 24-byte memcmp tests (PR33914) llvm-svn: 308963
2017-07-25 18:33:36 +08:00
; PR33914 - https://bugs.llvm.org/show_bug.cgi?id=33914
define i32 @length24(i8* %X, i8* %Y) nounwind {
; X86-LABEL: length24:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $24
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: retl
;
; X64-LABEL: length24:
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X64: # BB#0:
; X64-NEXT: movl $24, %edx
; X64-NEXT: jmp memcmp # TAILCALL
[X86] Add 24-byte memcmp tests (PR33914) llvm-svn: 308963
2017-07-25 18:33:36 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 24) nounwind
ret i32 %m
}
define i1 @length24_eq(i8* %x, i8* %y) nounwind {
; X86-LABEL: length24_eq:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $24
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: testl %eax, %eax
; X86-NEXT: sete %al
; X86-NEXT: retl
;
; X64-LABEL: length24_eq:
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X64: # BB#0:
; X64-NEXT: pushq %rax
; X64-NEXT: movl $24, %edx
; X64-NEXT: callq memcmp
[X86] Add 24-byte memcmp tests (PR33914) llvm-svn: 308963
2017-07-25 18:33:36 +08:00
; X64-NEXT: testl %eax, %eax
; X64-NEXT: sete %al
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X64-NEXT: popq %rcx
[X86] Add 24-byte memcmp tests (PR33914) llvm-svn: 308963
2017-07-25 18:33:36 +08:00
; X64-NEXT: retq
%call = tail call i32 @memcmp(i8* %x, i8* %y, i64 24) nounwind
%cmp = icmp eq i32 %call, 0
ret i1 %cmp
}
define i1 @length24_eq_const(i8* %X) nounwind {
; X86-LABEL: length24_eq_const:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $24
; X86-NEXT: pushl $.L.str
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: testl %eax, %eax
; X86-NEXT: setne %al
; X86-NEXT: retl
;
; X64-LABEL: length24_eq_const:
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X64: # BB#0:
; X64-NEXT: pushq %rax
; X64-NEXT: movl $.L.str, %esi
; X64-NEXT: movl $24, %edx
; X64-NEXT: callq memcmp
[X86] Add 24-byte memcmp tests (PR33914) llvm-svn: 308963
2017-07-25 18:33:36 +08:00
; X64-NEXT: testl %eax, %eax
; X64-NEXT: setne %al
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X64-NEXT: popq %rcx
[X86] Add 24-byte memcmp tests (PR33914) llvm-svn: 308963
2017-07-25 18:33:36 +08:00
; X64-NEXT: retq
%m = tail call i32 @memcmp(i8* %X, i8* getelementptr inbounds ([65 x i8], [65 x i8]* @.str, i32 0, i32 0), i64 24) nounwind
%c = icmp ne i32 %m, 0
ret i1 %c
}
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
define i32 @length32(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length32:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $32
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length32:
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X64: # BB#0:
; X64-NEXT: movl $32, %edx
; X64-NEXT: jmp memcmp # TAILCALL
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 32) nounwind
ret i32 %m
}
; PR33325 - https://bugs.llvm.org/show_bug.cgi?id=33325
define i1 @length32_eq(i8* %x, i8* %y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length32_eq:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $32
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: testl %eax, %eax
; X86-NEXT: sete %al
; X86-NEXT: retl
;
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X64-SSE2-LABEL: length32_eq:
; X64-SSE2: # BB#0:
; X64-SSE2-NEXT: pushq %rax
; X64-SSE2-NEXT: movl $32, %edx
; X64-SSE2-NEXT: callq memcmp
; X64-SSE2-NEXT: testl %eax, %eax
; X64-SSE2-NEXT: sete %al
; X64-SSE2-NEXT: popq %rcx
; X64-SSE2-NEXT: retq
;
; X64-AVX2-LABEL: length32_eq:
; X64-AVX2: # BB#0:
; X64-AVX2-NEXT: vmovdqu (%rdi), %ymm0
; X64-AVX2-NEXT: vpcmpeqb (%rsi), %ymm0, %ymm0
; X64-AVX2-NEXT: vpmovmskb %ymm0, %eax
; X64-AVX2-NEXT: cmpl $-1, %eax
; X64-AVX2-NEXT: sete %al
; X64-AVX2-NEXT: vzeroupper
; X64-AVX2-NEXT: retq
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
%call = tail call i32 @memcmp(i8* %x, i8* %y, i64 32) nounwind
%cmp = icmp eq i32 %call, 0
ret i1 %cmp
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length32_eq_const(i8* %X) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length32_eq_const:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $32
; X86-NEXT: pushl $.L.str
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: testl %eax, %eax
; X86-NEXT: setne %al
; X86-NEXT: retl
;
[X86][CGP] Reduce memcmp() expansion to 2 load pairs (PR33914) D35067/rL308322 attempted to support up to 4 load pairs for memcmp inlining which resulted in regressions for some optimized libc memcmp implementations (PR33914). Until we can match these more optimal cases, this patch reduces the memcmp expansion to a maximum of 2 load pairs (which matches what we do for -Os). This patch should be considered for the 5.0.0 release branch as well Differential Revision: https://reviews.llvm.org/D35830 llvm-svn: 308986
2017-07-26 01:04:37 +08:00
; X64-SSE2-LABEL: length32_eq_const:
; X64-SSE2: # BB#0:
; X64-SSE2-NEXT: pushq %rax
; X64-SSE2-NEXT: movl $.L.str, %esi
; X64-SSE2-NEXT: movl $32, %edx
; X64-SSE2-NEXT: callq memcmp
; X64-SSE2-NEXT: testl %eax, %eax
; X64-SSE2-NEXT: setne %al
; X64-SSE2-NEXT: popq %rcx
; X64-SSE2-NEXT: retq
;
; X64-AVX2-LABEL: length32_eq_const:
; X64-AVX2: # BB#0:
; X64-AVX2-NEXT: vmovdqu (%rdi), %ymm0
; X64-AVX2-NEXT: vpcmpeqb {{.*}}(%rip), %ymm0, %ymm0
; X64-AVX2-NEXT: vpmovmskb %ymm0, %eax
; X64-AVX2-NEXT: cmpl $-1, %eax
; X64-AVX2-NEXT: setne %al
; X64-AVX2-NEXT: vzeroupper
; X64-AVX2-NEXT: retq
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* getelementptr inbounds ([65 x i8], [65 x i8]* @.str, i32 0, i32 0), i64 32) nounwind
%c = icmp ne i32 %m, 0
ret i1 %c
}
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
define i32 @length64(i8* %X, i8* %Y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length64:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $64
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: retl
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
;
; X64-LABEL: length64:
; X64: # BB#0:
; X64-NEXT: movl $64, %edx
; X64-NEXT: jmp memcmp # TAILCALL
%m = tail call i32 @memcmp(i8* %X, i8* %Y, i64 64) nounwind
ret i32 %m
}
define i1 @length64_eq(i8* %x, i8* %y) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length64_eq:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $64
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: testl %eax, %eax
; X86-NEXT: setne %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length64_eq:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: pushq %rax
; X64-NEXT: movl $64, %edx
; X64-NEXT: callq memcmp
; X64-NEXT: testl %eax, %eax
; X64-NEXT: setne %al
; X64-NEXT: popq %rcx
; X64-NEXT: retq
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
%call = tail call i32 @memcmp(i8* %x, i8* %y, i64 64) nounwind
%cmp = icmp ne i32 %call, 0
ret i1 %cmp
}
[x86] remove unused param from tests; NFC llvm-svn: 304989
2017-06-09 01:02:39 +08:00
define i1 @length64_eq_const(i8* %X) nounwind {
[X86] Updated 32-bit memcmp tests to run with/without SSE2 llvm-svn: 306816
2017-06-30 19:23:59 +08:00
; X86-LABEL: length64_eq_const:
; X86: # BB#0:
; X86-NEXT: pushl $0
; X86-NEXT: pushl $64
; X86-NEXT: pushl $.L.str
; X86-NEXT: pushl {{[0-9]+}}(%esp)
; X86-NEXT: calll memcmp
; X86-NEXT: addl $16, %esp
; X86-NEXT: testl %eax, %eax
; X86-NEXT: sete %al
; X86-NEXT: retl
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
;
[x86] add tests for memcmp expansion; NFC We already had a test to demonstrate PR33325: https://bugs.llvm.org/show_bug.cgi?id=33325 I'm adding tests for general memcmp expansion (see D34005 / D33963) and: https://bugs.llvm.org/show_bug.cgi?id=33329 ...plus non-power-of-2 sizes, so we can see what that looks like currently or if expanded. llvm-svn: 304979
2017-06-08 23:01:29 +08:00
; X64-LABEL: length64_eq_const:
[x86] add 32-bit RUN for better memcmp coverage; NFC llvm-svn: 298744
2017-03-25 06:09:48 +08:00
; X64: # BB#0:
; X64-NEXT: pushq %rax
; X64-NEXT: movl $.L.str, %esi
; X64-NEXT: movl $64, %edx
; X64-NEXT: callq memcmp
; X64-NEXT: testl %eax, %eax
; X64-NEXT: sete %al
; X64-NEXT: popq %rcx
; X64-NEXT: retq
[x86] add memcmp tests, remove run Add tests for vector lengths that could be handled without a libcall. There's an explicit test for 'nobuiltin', so there's not much value in a separate run that checks that same behavior over and over again. llvm-svn: 298611
2017-03-23 23:38:22 +08:00
%m = tail call i32 @memcmp(i8* %X, i8* getelementptr inbounds ([65 x i8], [65 x i8]* @.str, i32 0, i32 0), i64 64) nounwind
[x86] improve tests, add tests, auto-generate checks; NFC llvm-svn: 298553
2017-03-23 06:39:17 +08:00
%c = icmp eq i32 %m, 0
ret i1 %c
move an optimization for memcmp out of simplifylibcalls and into SDISel. This optimization was causing simplifylibcalls to introduce type-unsafe nastiness. This is the first step, I'll be expanding the memcmp optimizations shortly, covering things that we really really wouldn't want simplifylibcalls to do. llvm-svn: 92098
2009-12-24 08:37:38 +08:00
}