maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/speculative-load-hardening.ll

865 lines
28 KiB
LLVM
Raw Normal View History

[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -x86-speculative-load-hardening | FileCheck %s --check-prefix=X64
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -x86-speculative-load-hardening -x86-speculative-load-hardening-lfence | FileCheck %s --check-prefix=X64-LFENCE
;
; FIXME: Add support for 32-bit and other EH ABIs.
declare void @leak(i32 %v1, i32 %v2)
declare void @sink(i32)
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
define i32 @test_trivial_entry_load(i32* %ptr) nounwind {
; X64-LABEL: test_trivial_entry_load:
; X64: # %bb.0: # %entry
; X64-NEXT: movq %rsp, %rcx
; X64-NEXT: movq $-1, %rax
; X64-NEXT: sarq $63, %rcx
; X64-NEXT: movl (%rdi), %eax
; X64-NEXT: orl %ecx, %eax
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_trivial_entry_load:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: movl (%rdi), %eax
; X64-LFENCE-NEXT: retq
entry:
%v = load i32, i32* %ptr
ret i32 %v
}
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
define void @test_basic_conditions(i32 %a, i32 %b, i32 %c, i32* %ptr1, i32* %ptr2, i32** %ptr3) nounwind {
; X64-LABEL: test_basic_conditions:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %rbx
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq $-1, %rbx
; X64-NEXT: sarq $63, %rax
; X64-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jne .LBB1_1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.2: # %then1
; X64-NEXT: cmovneq %rbx, %rax
; X64-NEXT: testl %esi, %esi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: je .LBB1_4
; X64-NEXT: .LBB1_1:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmoveq %rbx, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB1_8: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
; X64-NEXT: retq
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB1_4: # %then2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %r8, %r15
; X64-NEXT: cmovneq %rbx, %rax
; X64-NEXT: testl %edx, %edx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: je .LBB1_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.5: # %else3
; X64-NEXT: cmoveq %rbx, %rax
; X64-NEXT: movslq (%r9), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: leaq (%r15,%rcx,4), %r14
; X64-NEXT: movl %ecx, (%r15,%rcx,4)
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB1_7
; X64-NEXT: .LBB1_6: # %then3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovneq %rbx, %rax
; X64-NEXT: movl (%rcx), %ecx
; X64-NEXT: addl (%r15), %ecx
; X64-NEXT: orl %eax, %ecx
; X64-NEXT: movslq %ecx, %rdi
; X64-NEXT: movl (%r15,%rdi,4), %esi
; X64-NEXT: orl %eax, %esi
; X64-NEXT: movq (%r9), %r14
; X64-NEXT: orq %rax, %r14
; X64-NEXT: addl (%r14), %esi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: # kill: def $edi killed $edi killed $rdi
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq leak
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB1_7: # %merge
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movslq (%r14), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: movl $0, (%r15,%rcx,4)
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB1_8
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
;
; X64-LFENCE-LABEL: test_basic_conditions:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jne .LBB1_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.1: # %then1
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: testl %esi, %esi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: je .LBB1_2
; X64-LFENCE-NEXT: .LBB1_6: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: retq
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB1_2: # %then2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: movq %r8, %rbx
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: testl %edx, %edx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: je .LBB1_3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.4: # %else3
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq (%r9), %rax
; X64-LFENCE-NEXT: leaq (%rbx,%rax,4), %r14
; X64-LFENCE-NEXT: movl %eax, (%rbx,%rax,4)
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jmp .LBB1_5
; X64-LFENCE-NEXT: .LBB1_3: # %then3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movl (%rcx), %eax
; X64-LFENCE-NEXT: addl (%rbx), %eax
; X64-LFENCE-NEXT: movslq %eax, %rdi
; X64-LFENCE-NEXT: movl (%rbx,%rdi,4), %esi
; X64-LFENCE-NEXT: movq (%r9), %r14
; X64-LFENCE-NEXT: addl (%r14), %esi
; X64-LFENCE-NEXT: # kill: def $edi killed $edi killed $rdi
; X64-LFENCE-NEXT: callq leak
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB1_5: # %merge
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: movslq (%r14), %rax
; X64-LFENCE-NEXT: movl $0, (%rbx,%rax,4)
; X64-LFENCE-NEXT: addq $8, %rsp
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: retq
entry:
%a.cmp = icmp eq i32 %a, 0
br i1 %a.cmp, label %then1, label %exit
then1:
%b.cmp = icmp eq i32 %b, 0
br i1 %b.cmp, label %then2, label %exit
then2:
%c.cmp = icmp eq i32 %c, 0
br i1 %c.cmp, label %then3, label %else3
then3:
%secret1 = load i32, i32* %ptr1
%secret2 = load i32, i32* %ptr2
%secret.sum1 = add i32 %secret1, %secret2
%ptr2.idx = getelementptr i32, i32* %ptr2, i32 %secret.sum1
%secret3 = load i32, i32* %ptr2.idx
%secret4 = load i32*, i32** %ptr3
%secret5 = load i32, i32* %secret4
%secret.sum2 = add i32 %secret3, %secret5
call void @leak(i32 %secret.sum1, i32 %secret.sum2)
br label %merge
else3:
%secret6 = load i32*, i32** %ptr3
%cast = ptrtoint i32* %secret6 to i32
%ptr2.idx2 = getelementptr i32, i32* %ptr2, i32 %cast
store i32 %cast, i32* %ptr2.idx2
br label %merge
merge:
%phi = phi i32* [ %secret4, %then3 ], [ %ptr2.idx2, %else3 ]
%secret7 = load i32, i32* %phi
%ptr2.idx3 = getelementptr i32, i32* %ptr2, i32 %secret7
store i32 0, i32* %ptr2.idx3
br label %exit
exit:
ret void
}
define void @test_basic_loop(i32 %a, i32 %b, i32* %ptr1, i32* %ptr2) nounwind {
; X64-LABEL: test_basic_loop:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %rbp
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %r12
; X64-NEXT: pushq %rbx
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq $-1, %r15
; X64-NEXT: sarq $63, %rax
; X64-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: je .LBB2_2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.1:
; X64-NEXT: cmoveq %r15, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB2_5
; X64-NEXT: .LBB2_2: # %l.header.preheader
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rcx, %r14
; X64-NEXT: movq %rdx, %r12
; X64-NEXT: movl %esi, %ebp
; X64-NEXT: cmovneq %r15, %rax
; X64-NEXT: xorl %ebx, %ebx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB2_3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB2_6: # in Loop: Header=BB2_3 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovgeq %r15, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB2_3: # %l.header
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # =>This Inner Loop Header: Depth=1
; X64-NEXT: movslq (%r12), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: movq %rax, %rdx
; X64-NEXT: orq %r14, %rdx
; X64-NEXT: movl (%rdx,%rcx,4), %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: incl %ebx
; X64-NEXT: cmpl %ebp, %ebx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jl .LBB2_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.4:
; X64-NEXT: cmovlq %r15, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB2_5: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r12
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
; X64-NEXT: popq %rbp
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_basic_loop:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %rbp
; X64-LFENCE-NEXT: pushq %r15
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jne .LBB2_3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.1: # %l.header.preheader
; X64-LFENCE-NEXT: movq %rcx, %r14
; X64-LFENCE-NEXT: movq %rdx, %r15
; X64-LFENCE-NEXT: movl %esi, %ebp
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: xorl %ebx, %ebx
; X64-LFENCE-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB2_2: # %l.header
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # =>This Inner Loop Header: Depth=1
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq (%r15), %rax
; X64-LFENCE-NEXT: movl (%r14,%rax,4), %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: incl %ebx
; X64-LFENCE-NEXT: cmpl %ebp, %ebx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jl .LBB2_2
; X64-LFENCE-NEXT: .LBB2_3: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: addq $8, %rsp
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %r15
; X64-LFENCE-NEXT: popq %rbp
; X64-LFENCE-NEXT: retq
entry:
%a.cmp = icmp eq i32 %a, 0
br i1 %a.cmp, label %l.header, label %exit
l.header:
%i = phi i32 [ 0, %entry ], [ %i.next, %l.header ]
%secret = load i32, i32* %ptr1
%ptr2.idx = getelementptr i32, i32* %ptr2, i32 %secret
%leak = load i32, i32* %ptr2.idx
call void @sink(i32 %leak)
%i.next = add i32 %i, 1
%i.cmp = icmp slt i32 %i.next, %b
br i1 %i.cmp, label %l.header, label %exit
exit:
ret void
}
define void @test_basic_nested_loop(i32 %a, i32 %b, i32 %c, i32* %ptr1, i32* %ptr2) nounwind {
; X64-LABEL: test_basic_nested_loop:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %rbp
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %r13
; X64-NEXT: pushq %r12
; X64-NEXT: pushq %rbx
; X64-NEXT: pushq %rax
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq $-1, %r12
; X64-NEXT: sarq $63, %rax
; X64-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: je .LBB3_2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.1:
; X64-NEXT: cmoveq %r12, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB3_10
; X64-NEXT: .LBB3_2: # %l1.header.preheader
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %r8, %r14
; X64-NEXT: movq %rcx, %rbx
; X64-NEXT: movl %edx, %ebp
; X64-NEXT: movl %esi, %r15d
; X64-NEXT: cmovneq %r12, %rax
; X64-NEXT: xorl %r13d, %r13d
; X64-NEXT: movl %esi, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
; X64-NEXT: testl %r15d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jg .LBB3_5
; X64-NEXT: jmp .LBB3_4
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_12:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovgeq %r12, %rax
; X64-NEXT: testl %r15d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jle .LBB3_4
; X64-NEXT: .LBB3_5: # %l2.header.preheader
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovleq %r12, %rax
; X64-NEXT: xorl %r15d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB3_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_11: # in Loop: Header=BB3_6 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovgeq %r12, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_6: # %l2.header
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # =>This Inner Loop Header: Depth=1
; X64-NEXT: movslq (%rbx), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: movq %rax, %rdx
; X64-NEXT: orq %r14, %rdx
; X64-NEXT: movl (%rdx,%rcx,4), %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: incl %r15d
; X64-NEXT: cmpl %ebp, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jl .LBB3_11
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.7:
; X64-NEXT: cmovlq %r12, %rax
; X64-NEXT: movl {{[-0-9]+}}(%r{{[sb]}}p), %r15d # 4-byte Reload
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB3_8
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_4:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovgq %r12, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_8: # %l1.latch
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movslq (%rbx), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: movq %rax, %rdx
; X64-NEXT: orq %r14, %rdx
; X64-NEXT: movl (%rdx,%rcx,4), %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: incl %r13d
; X64-NEXT: cmpl %r15d, %r13d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jl .LBB3_12
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.9:
; X64-NEXT: cmovlq %r12, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_10: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: addq $8, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r12
; X64-NEXT: popq %r13
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
; X64-NEXT: popq %rbp
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_basic_nested_loop:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %rbp
; X64-LFENCE-NEXT: pushq %r15
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %r13
; X64-LFENCE-NEXT: pushq %r12
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jne .LBB3_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.1: # %l1.header.preheader
; X64-LFENCE-NEXT: movq %r8, %r14
; X64-LFENCE-NEXT: movq %rcx, %rbx
; X64-LFENCE-NEXT: movl %edx, %r13d
; X64-LFENCE-NEXT: movl %esi, %r15d
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: xorl %r12d, %r12d
; X64-LFENCE-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB3_2: # %l1.header
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # =>This Loop Header: Depth=1
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: # Child Loop BB3_4 Depth 2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: testl %r15d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jle .LBB3_5
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.3: # %l2.header.preheader
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: # in Loop: Header=BB3_2 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: xorl %ebp, %ebp
; X64-LFENCE-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB3_4: # %l2.header
; X64-LFENCE-NEXT: # Parent Loop BB3_2 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # => This Inner Loop Header: Depth=2
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq (%rbx), %rax
; X64-LFENCE-NEXT: movl (%r14,%rax,4), %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: incl %ebp
; X64-LFENCE-NEXT: cmpl %r13d, %ebp
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jl .LBB3_4
; X64-LFENCE-NEXT: .LBB3_5: # %l1.latch
; X64-LFENCE-NEXT: # in Loop: Header=BB3_2 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq (%rbx), %rax
; X64-LFENCE-NEXT: movl (%r14,%rax,4), %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: incl %r12d
; X64-LFENCE-NEXT: cmpl %r15d, %r12d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jl .LBB3_2
; X64-LFENCE-NEXT: .LBB3_6: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: addq $8, %rsp
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r12
; X64-LFENCE-NEXT: popq %r13
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %r15
; X64-LFENCE-NEXT: popq %rbp
; X64-LFENCE-NEXT: retq
entry:
%a.cmp = icmp eq i32 %a, 0
br i1 %a.cmp, label %l1.header, label %exit
l1.header:
%i = phi i32 [ 0, %entry ], [ %i.next, %l1.latch ]
%b.cmp = icmp sgt i32 %b, 0
br i1 %b.cmp, label %l2.header, label %l1.latch
l2.header:
%j = phi i32 [ 0, %l1.header ], [ %j.next, %l2.header ]
%secret = load i32, i32* %ptr1
%ptr2.idx = getelementptr i32, i32* %ptr2, i32 %secret
%leak = load i32, i32* %ptr2.idx
call void @sink(i32 %leak)
%j.next = add i32 %j, 1
%j.cmp = icmp slt i32 %j.next, %c
br i1 %j.cmp, label %l2.header, label %l1.latch
l1.latch:
%secret2 = load i32, i32* %ptr1
%ptr2.idx2 = getelementptr i32, i32* %ptr2, i32 %secret2
%leak2 = load i32, i32* %ptr2.idx2
call void @sink(i32 %leak2)
%i.next = add i32 %i, 1
%i.cmp = icmp slt i32 %i.next, %b
br i1 %i.cmp, label %l1.header, label %exit
exit:
ret void
}
declare i32 @__gxx_personality_v0(...)
declare i8* @__cxa_allocate_exception(i64) local_unnamed_addr
declare void @__cxa_throw(i8*, i8*, i8*) local_unnamed_addr
define void @test_basic_eh(i32 %a, i32* %ptr1, i32* %ptr2) nounwind personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
; X64-LABEL: test_basic_eh:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %rbp
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %rbx
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq $-1, %rcx
; X64-NEXT: sarq $63, %rax
; X64-NEXT: cmpl $41, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jg .LBB4_1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.2: # %thrower
; X64-NEXT: movq %rdx, %r14
; X64-NEXT: movq %rsi, %rbx
; X64-NEXT: cmovgq %rcx, %rax
; X64-NEXT: movslq %edi, %rcx
; X64-NEXT: movl (%rsi,%rcx,4), %ebp
; X64-NEXT: orl %eax, %ebp
; X64-NEXT: movl $4, %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq __cxa_allocate_exception
; X64-NEXT: movq %rsp, %rcx
; X64-NEXT: sarq $63, %rcx
; X64-NEXT: movl %ebp, (%rax)
; X64-NEXT: .Ltmp0:
; X64-NEXT: xorl %esi, %esi
; X64-NEXT: xorl %edx, %edx
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: movq %rax, %rdi
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: callq __cxa_throw
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: .Ltmp1:
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB4_3
; X64-NEXT: .LBB4_1:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovleq %rcx, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB4_3: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r14
; X64-NEXT: popq %rbp
; X64-NEXT: retq
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB4_4: # %lpad
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .Ltmp2:
; X64-NEXT: movq %rsp, %rcx
; X64-NEXT: sarq $63, %rcx
; X64-NEXT: movl (%rax), %eax
; X64-NEXT: addl (%rbx), %eax
; X64-NEXT: orl %ecx, %eax
; X64-NEXT: cltq
; X64-NEXT: movl (%r14,%rax,4), %edi
; X64-NEXT: orl %ecx, %edi
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: callq sink
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
;
; X64-LFENCE-LABEL: test_basic_eh:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %rbp
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: cmpl $41, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jg .LBB4_2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.1: # %thrower
; X64-LFENCE-NEXT: movq %rdx, %r14
; X64-LFENCE-NEXT: movq %rsi, %rbx
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq %edi, %rax
; X64-LFENCE-NEXT: movl (%rsi,%rax,4), %ebp
; X64-LFENCE-NEXT: movl $4, %edi
; X64-LFENCE-NEXT: callq __cxa_allocate_exception
; X64-LFENCE-NEXT: movl %ebp, (%rax)
; X64-LFENCE-NEXT: .Ltmp0:
; X64-LFENCE-NEXT: xorl %esi, %esi
; X64-LFENCE-NEXT: xorl %edx, %edx
; X64-LFENCE-NEXT: movq %rax, %rdi
; X64-LFENCE-NEXT: callq __cxa_throw
; X64-LFENCE-NEXT: .Ltmp1:
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB4_2: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %rbp
; X64-LFENCE-NEXT: retq
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB4_3: # %lpad
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: .Ltmp2:
; X64-LFENCE-NEXT: movl (%rax), %eax
; X64-LFENCE-NEXT: addl (%rbx), %eax
; X64-LFENCE-NEXT: cltq
; X64-LFENCE-NEXT: movl (%r14,%rax,4), %edi
; X64-LFENCE-NEXT: callq sink
entry:
%a.cmp = icmp slt i32 %a, 42
br i1 %a.cmp, label %thrower, label %exit
thrower:
%badidx = getelementptr i32, i32* %ptr1, i32 %a
%secret1 = load i32, i32* %badidx
%e.ptr = call i8* @__cxa_allocate_exception(i64 4)
%e.ptr.cast = bitcast i8* %e.ptr to i32*
store i32 %secret1, i32* %e.ptr.cast
invoke void @__cxa_throw(i8* %e.ptr, i8* null, i8* null)
to label %exit unwind label %lpad
exit:
ret void
lpad:
%e = landingpad { i8*, i32 }
catch i8* null
%e.catch.ptr = extractvalue { i8*, i32 } %e, 0
%e.catch.ptr.cast = bitcast i8* %e.catch.ptr to i32*
%secret1.catch = load i32, i32* %e.catch.ptr.cast
%secret2 = load i32, i32* %ptr1
%secret.sum = add i32 %secret1.catch, %secret2
%ptr2.idx = getelementptr i32, i32* %ptr2, i32 %secret.sum
%leak = load i32, i32* %ptr2.idx
call void @sink(i32 %leak)
unreachable
}
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
declare void @sink_float(float)
declare void @sink_double(double)
; Test direct and converting loads of floating point values.
define void @test_fp_loads(float* %fptr, double* %dptr, i32* %i32ptr, i64* %i64ptr) nounwind {
; X64-LABEL: test_fp_loads:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %r12
; X64-NEXT: pushq %rbx
; X64-NEXT: pushq %rax
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq %rcx, %r15
; X64-NEXT: movq %rdx, %r14
; X64-NEXT: movq %rsi, %rbx
; X64-NEXT: movq %rdi, %r12
; X64-NEXT: movq $-1, %rcx
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %r12
; X64-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_float
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %rbx
; X64-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_double
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; X64-NEXT: cvtsd2ss %xmm0, %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_float
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; X64-NEXT: cvtss2sd %xmm0, %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_double
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %r14
; X64-NEXT: xorps %xmm0, %xmm0
; X64-NEXT: cvtsi2ssl (%r14), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_float
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %r15
; X64-NEXT: xorps %xmm0, %xmm0
; X64-NEXT: cvtsi2sdq (%r15), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_double
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: xorps %xmm0, %xmm0
; X64-NEXT: cvtsi2ssq (%r15), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_float
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: xorps %xmm0, %xmm0
; X64-NEXT: cvtsi2sdl (%r14), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_double
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: addq $8, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r12
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_fp_loads:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %r15
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %r12
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: movq %rcx, %r15
; X64-LFENCE-NEXT: movq %rdx, %r14
; X64-LFENCE-NEXT: movq %rsi, %rbx
; X64-LFENCE-NEXT: movq %rdi, %r12
; X64-LFENCE-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; X64-LFENCE-NEXT: callq sink_float
; X64-LFENCE-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; X64-LFENCE-NEXT: callq sink_double
; X64-LFENCE-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; X64-LFENCE-NEXT: cvtsd2ss %xmm0, %xmm0
; X64-LFENCE-NEXT: callq sink_float
; X64-LFENCE-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; X64-LFENCE-NEXT: cvtss2sd %xmm0, %xmm0
; X64-LFENCE-NEXT: callq sink_double
; X64-LFENCE-NEXT: xorps %xmm0, %xmm0
; X64-LFENCE-NEXT: cvtsi2ssl (%r14), %xmm0
; X64-LFENCE-NEXT: callq sink_float
; X64-LFENCE-NEXT: xorps %xmm0, %xmm0
; X64-LFENCE-NEXT: cvtsi2sdq (%r15), %xmm0
; X64-LFENCE-NEXT: callq sink_double
; X64-LFENCE-NEXT: xorps %xmm0, %xmm0
; X64-LFENCE-NEXT: cvtsi2ssq (%r15), %xmm0
; X64-LFENCE-NEXT: callq sink_float
; X64-LFENCE-NEXT: xorps %xmm0, %xmm0
; X64-LFENCE-NEXT: cvtsi2sdl (%r14), %xmm0
; X64-LFENCE-NEXT: callq sink_double
; X64-LFENCE-NEXT: addq $8, %rsp
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r12
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %r15
; X64-LFENCE-NEXT: retq
entry:
%f1 = load float, float* %fptr
call void @sink_float(float %f1)
%d1 = load double, double* %dptr
call void @sink_double(double %d1)
%f2.d = load double, double* %dptr
%f2 = fptrunc double %f2.d to float
call void @sink_float(float %f2)
%d2.f = load float, float* %fptr
%d2 = fpext float %d2.f to double
call void @sink_double(double %d2)
%f3.i = load i32, i32* %i32ptr
%f3 = sitofp i32 %f3.i to float
call void @sink_float(float %f3)
%d3.i = load i64, i64* %i64ptr
%d3 = sitofp i64 %d3.i to double
call void @sink_double(double %d3)
%f4.i = load i64, i64* %i64ptr
%f4 = sitofp i64 %f4.i to float
call void @sink_float(float %f4)
%d4.i = load i32, i32* %i32ptr
%d4 = sitofp i32 %d4.i to double
call void @sink_double(double %d4)
ret void
}
declare void @sink_v4f32(<4 x float>)
declare void @sink_v2f64(<2 x double>)
declare void @sink_v16i8(<16 x i8>)
declare void @sink_v8i16(<8 x i16>)
declare void @sink_v4i32(<4 x i32>)
declare void @sink_v2i64(<2 x i64>)
; Test loads of vectors.
define void @test_vec_loads(<4 x float>* %v4f32ptr, <2 x double>* %v2f64ptr, <16 x i8>* %v16i8ptr, <8 x i16>* %v8i16ptr, <4 x i32>* %v4i32ptr, <2 x i64>* %v2i64ptr) nounwind {
; X64-LABEL: test_vec_loads:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %r13
; X64-NEXT: pushq %r12
; X64-NEXT: pushq %rbx
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq %r9, %r14
; X64-NEXT: movq %r8, %r15
; X64-NEXT: movq %rcx, %r12
; X64-NEXT: movq %rdx, %r13
; X64-NEXT: movq %rsi, %rbx
; X64-NEXT: movq $-1, %rcx
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %rdi
; X64-NEXT: movaps (%rdi), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v4f32
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %rbx
; X64-NEXT: movaps (%rbx), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v2f64
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %r13
; X64-NEXT: movaps (%r13), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v16i8
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %r12
; X64-NEXT: movaps (%r12), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v8i16
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %r15
; X64-NEXT: movaps (%r15), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v4i32
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %r14
; X64-NEXT: movaps (%r14), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v2i64
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: sarq $63, %rax
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r12
; X64-NEXT: popq %r13
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_vec_loads:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %r15
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %r13
; X64-LFENCE-NEXT: pushq %r12
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: movq %r9, %r14
; X64-LFENCE-NEXT: movq %r8, %r15
; X64-LFENCE-NEXT: movq %rcx, %r12
; X64-LFENCE-NEXT: movq %rdx, %r13
; X64-LFENCE-NEXT: movq %rsi, %rbx
; X64-LFENCE-NEXT: movaps (%rdi), %xmm0
; X64-LFENCE-NEXT: callq sink_v4f32
; X64-LFENCE-NEXT: movaps (%rbx), %xmm0
; X64-LFENCE-NEXT: callq sink_v2f64
; X64-LFENCE-NEXT: movaps (%r13), %xmm0
; X64-LFENCE-NEXT: callq sink_v16i8
; X64-LFENCE-NEXT: movaps (%r12), %xmm0
; X64-LFENCE-NEXT: callq sink_v8i16
; X64-LFENCE-NEXT: movaps (%r15), %xmm0
; X64-LFENCE-NEXT: callq sink_v4i32
; X64-LFENCE-NEXT: movaps (%r14), %xmm0
; X64-LFENCE-NEXT: callq sink_v2i64
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r12
; X64-LFENCE-NEXT: popq %r13
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %r15
; X64-LFENCE-NEXT: retq
entry:
%x1 = load <4 x float>, <4 x float>* %v4f32ptr
call void @sink_v4f32(<4 x float> %x1)
%x2 = load <2 x double>, <2 x double>* %v2f64ptr
call void @sink_v2f64(<2 x double> %x2)
%x3 = load <16 x i8>, <16 x i8>* %v16i8ptr
call void @sink_v16i8(<16 x i8> %x3)
%x4 = load <8 x i16>, <8 x i16>* %v8i16ptr
call void @sink_v8i16(<8 x i16> %x4)
%x5 = load <4 x i32>, <4 x i32>* %v4i32ptr
call void @sink_v4i32(<4 x i32> %x5)
%x6 = load <2 x i64>, <2 x i64>* %v2i64ptr
call void @sink_v2i64(<2 x i64> %x6)
ret void
}