maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/speculative-load-hardening.ll

1145 lines
37 KiB
LLVM
Raw Normal View History

[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --check-prefix=X64
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -x86-slh-lfence | FileCheck %s --check-prefix=X64-LFENCE
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
;
; FIXME: Add support for 32-bit and other EH ABIs.
declare void @leak(i32 %v1, i32 %v2)
declare void @sink(i32)
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
define i32 @test_trivial_entry_load(i32* %ptr) speculative_load_hardening {
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LABEL: test_trivial_entry_load:
; X64: # %bb.0: # %entry
; X64-NEXT: movq %rsp, %rcx
; X64-NEXT: movq $-1, %rax
; X64-NEXT: sarq $63, %rcx
; X64-NEXT: movl (%rdi), %eax
; X64-NEXT: orl %ecx, %eax
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_trivial_entry_load:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: movl (%rdi), %eax
; X64-LFENCE-NEXT: retq
entry:
%v = load i32, i32* %ptr
ret i32 %v
}
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
define void @test_basic_conditions(i32 %a, i32 %b, i32 %c, i32* %ptr1, i32* %ptr2, i32** %ptr3) speculative_load_hardening {
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LABEL: test_basic_conditions:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %r15
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: pushq %r14
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 24
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: pushq %rbx
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 32
; X64-NEXT: .cfi_offset %rbx, -32
; X64-NEXT: .cfi_offset %r14, -24
; X64-NEXT: .cfi_offset %r15, -16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq $-1, %rbx
; X64-NEXT: sarq $63, %rax
; X64-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jne .LBB1_1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.2: # %then1
; X64-NEXT: cmovneq %rbx, %rax
; X64-NEXT: testl %esi, %esi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: je .LBB1_4
; X64-NEXT: .LBB1_1:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmoveq %rbx, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB1_8: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: popq %rbx
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 24
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: popq %r14
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: popq %r15
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 8
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: retq
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB1_4: # %then2
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 32
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq %r8, %r14
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovneq %rbx, %rax
; X64-NEXT: testl %edx, %edx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: je .LBB1_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.5: # %else3
; X64-NEXT: cmoveq %rbx, %rax
; X64-NEXT: movslq (%r9), %rcx
; X64-NEXT: orq %rax, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: leaq (%r14,%rcx,4), %r15
; X64-NEXT: movl %ecx, (%r14,%rcx,4)
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB1_7
; X64-NEXT: .LBB1_6: # %then3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovneq %rbx, %rax
; X64-NEXT: movl (%rcx), %ecx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: addl (%r14), %ecx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movslq %ecx, %rdi
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: orq %rax, %rdi
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movl (%r14,%rdi,4), %esi
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: orl %eax, %esi
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq (%r9), %r15
; X64-NEXT: orq %rax, %r15
; X64-NEXT: addl (%r15), %esi
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: # kill: def $edi killed $edi killed $rdi
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq leak
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr0:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr0, %rcx
; X64-NEXT: cmovneq %rbx, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB1_7: # %merge
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movslq (%r15), %rcx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: orq %rax, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movl $0, (%r14,%rcx,4)
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB1_8
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
;
; X64-LFENCE-LABEL: test_basic_conditions:
; X64-LFENCE: # %bb.0: # %entry
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 16
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 24
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 32
; X64-LFENCE-NEXT: .cfi_offset %rbx, -24
; X64-LFENCE-NEXT: .cfi_offset %r14, -16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jne .LBB1_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.1: # %then1
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: testl %esi, %esi
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: jne .LBB1_6
; X64-LFENCE-NEXT: # %bb.2: # %then2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: movq %r8, %rbx
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: testl %edx, %edx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: je .LBB1_3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.4: # %else3
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq (%r9), %rax
; X64-LFENCE-NEXT: leaq (%rbx,%rax,4), %r14
; X64-LFENCE-NEXT: movl %eax, (%rbx,%rax,4)
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jmp .LBB1_5
; X64-LFENCE-NEXT: .LBB1_3: # %then3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movl (%rcx), %eax
; X64-LFENCE-NEXT: addl (%rbx), %eax
; X64-LFENCE-NEXT: movslq %eax, %rdi
; X64-LFENCE-NEXT: movl (%rbx,%rdi,4), %esi
; X64-LFENCE-NEXT: movq (%r9), %r14
; X64-LFENCE-NEXT: addl (%r14), %esi
; X64-LFENCE-NEXT: # kill: def $edi killed $edi killed $rdi
; X64-LFENCE-NEXT: callq leak
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB1_5: # %merge
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: movslq (%r14), %rax
; X64-LFENCE-NEXT: movl $0, (%rbx,%rax,4)
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .LBB1_6: # %exit
; X64-LFENCE-NEXT: lfence
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: addq $8, %rsp
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 24
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: popq %rbx
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: popq %r14
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 8
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: retq
entry:
%a.cmp = icmp eq i32 %a, 0
br i1 %a.cmp, label %then1, label %exit
then1:
%b.cmp = icmp eq i32 %b, 0
br i1 %b.cmp, label %then2, label %exit
then2:
%c.cmp = icmp eq i32 %c, 0
br i1 %c.cmp, label %then3, label %else3
then3:
%secret1 = load i32, i32* %ptr1
%secret2 = load i32, i32* %ptr2
%secret.sum1 = add i32 %secret1, %secret2
%ptr2.idx = getelementptr i32, i32* %ptr2, i32 %secret.sum1
%secret3 = load i32, i32* %ptr2.idx
%secret4 = load i32*, i32** %ptr3
%secret5 = load i32, i32* %secret4
%secret.sum2 = add i32 %secret3, %secret5
call void @leak(i32 %secret.sum1, i32 %secret.sum2)
br label %merge
else3:
%secret6 = load i32*, i32** %ptr3
%cast = ptrtoint i32* %secret6 to i32
%ptr2.idx2 = getelementptr i32, i32* %ptr2, i32 %cast
store i32 %cast, i32* %ptr2.idx2
br label %merge
merge:
%phi = phi i32* [ %secret4, %then3 ], [ %ptr2.idx2, %else3 ]
%secret7 = load i32, i32* %phi
%ptr2.idx3 = getelementptr i32, i32* %ptr2, i32 %secret7
store i32 0, i32* %ptr2.idx3
br label %exit
exit:
ret void
}
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
define void @test_basic_loop(i32 %a, i32 %b, i32* %ptr1, i32* %ptr2) nounwind speculative_load_hardening {
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LABEL: test_basic_loop:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %rbp
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %r12
; X64-NEXT: pushq %rbx
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq $-1, %r15
; X64-NEXT: sarq $63, %rax
; X64-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: je .LBB2_2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.1:
; X64-NEXT: cmoveq %r15, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB2_5
; X64-NEXT: .LBB2_2: # %l.header.preheader
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rcx, %r14
; X64-NEXT: movq %rdx, %r12
; X64-NEXT: movl %esi, %ebp
; X64-NEXT: cmovneq %r15, %rax
; X64-NEXT: xorl %ebx, %ebx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB2_3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB2_6: # in Loop: Header=BB2_3 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: cmovgeq %r15, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB2_3: # %l.header
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # =>This Inner Loop Header: Depth=1
; X64-NEXT: movslq (%r12), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: movq %rax, %rdx
; X64-NEXT: orq %r14, %rdx
; X64-NEXT: movl (%rdx,%rcx,4), %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr1:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr1, %rcx
; X64-NEXT: cmovneq %r15, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: incl %ebx
; X64-NEXT: cmpl %ebp, %ebx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jl .LBB2_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.4:
; X64-NEXT: cmovlq %r15, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB2_5: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r12
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
; X64-NEXT: popq %rbp
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_basic_loop:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %rbp
; X64-LFENCE-NEXT: pushq %r15
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jne .LBB2_3
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.1: # %l.header.preheader
; X64-LFENCE-NEXT: movq %rcx, %r14
; X64-LFENCE-NEXT: movq %rdx, %r15
; X64-LFENCE-NEXT: movl %esi, %ebp
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: xorl %ebx, %ebx
; X64-LFENCE-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB2_2: # %l.header
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # =>This Inner Loop Header: Depth=1
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq (%r15), %rax
; X64-LFENCE-NEXT: movl (%r14,%rax,4), %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: incl %ebx
; X64-LFENCE-NEXT: cmpl %ebp, %ebx
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jl .LBB2_2
; X64-LFENCE-NEXT: .LBB2_3: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: addq $8, %rsp
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %r15
; X64-LFENCE-NEXT: popq %rbp
; X64-LFENCE-NEXT: retq
entry:
%a.cmp = icmp eq i32 %a, 0
br i1 %a.cmp, label %l.header, label %exit
l.header:
%i = phi i32 [ 0, %entry ], [ %i.next, %l.header ]
%secret = load i32, i32* %ptr1
%ptr2.idx = getelementptr i32, i32* %ptr2, i32 %secret
%leak = load i32, i32* %ptr2.idx
call void @sink(i32 %leak)
%i.next = add i32 %i, 1
%i.cmp = icmp slt i32 %i.next, %b
br i1 %i.cmp, label %l.header, label %exit
exit:
ret void
}
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
define void @test_basic_nested_loop(i32 %a, i32 %b, i32 %c, i32* %ptr1, i32* %ptr2) nounwind speculative_load_hardening {
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LABEL: test_basic_nested_loop:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %rbp
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %r13
; X64-NEXT: pushq %r12
; X64-NEXT: pushq %rbx
; X64-NEXT: pushq %rax
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq $-1, %rbp
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rax
; X64-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: je .LBB3_2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.1:
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmoveq %rbp, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB3_10
; X64-NEXT: .LBB3_2: # %l1.header.preheader
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %r8, %r14
; X64-NEXT: movq %rcx, %rbx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movl %edx, %r12d
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movl %esi, %r15d
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovneq %rbp, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: xorl %r13d, %r13d
; X64-NEXT: movl %esi, {{[-0-9]+}}(%r{{[sb]}}p) # 4-byte Spill
; X64-NEXT: testl %r15d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jg .LBB3_5
; X64-NEXT: jmp .LBB3_4
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_12:
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovgeq %rbp, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: testl %r15d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jle .LBB3_4
; X64-NEXT: .LBB3_5: # %l2.header.preheader
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovleq %rbp, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: xorl %r15d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB3_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_11: # in Loop: Header=BB3_6 Depth=1
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovgeq %rbp, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_6: # %l2.header
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # =>This Inner Loop Header: Depth=1
; X64-NEXT: movslq (%rbx), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: movq %rax, %rdx
; X64-NEXT: orq %r14, %rdx
; X64-NEXT: movl (%rdx,%rcx,4), %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr2:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr2, %rcx
; X64-NEXT: cmovneq %rbp, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: incl %r15d
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpl %r12d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jl .LBB3_11
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.7:
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovlq %rbp, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movl {{[-0-9]+}}(%r{{[sb]}}p), %r15d # 4-byte Reload
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB3_8
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_4:
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovgq %rbp, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_8: # %l1.latch
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movslq (%rbx), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: movq %rax, %rdx
; X64-NEXT: orq %r14, %rdx
; X64-NEXT: movl (%rdx,%rcx,4), %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr3:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr3, %rcx
; X64-NEXT: cmovneq %rbp, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: incl %r13d
; X64-NEXT: cmpl %r15d, %r13d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jl .LBB3_12
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.9:
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovlq %rbp, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB3_10: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: addq $8, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r12
; X64-NEXT: popq %r13
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
; X64-NEXT: popq %rbp
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_basic_nested_loop:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %rbp
; X64-LFENCE-NEXT: pushq %r15
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %r13
; X64-LFENCE-NEXT: pushq %r12
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: testl %edi, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jne .LBB3_6
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.1: # %l1.header.preheader
; X64-LFENCE-NEXT: movq %r8, %r14
; X64-LFENCE-NEXT: movq %rcx, %rbx
; X64-LFENCE-NEXT: movl %edx, %r13d
; X64-LFENCE-NEXT: movl %esi, %r15d
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: xorl %r12d, %r12d
; X64-LFENCE-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB3_2: # %l1.header
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # =>This Loop Header: Depth=1
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: # Child Loop BB3_4 Depth 2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: testl %r15d, %r15d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jle .LBB3_5
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.3: # %l2.header.preheader
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: # in Loop: Header=BB3_2 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: xorl %ebp, %ebp
; X64-LFENCE-NEXT: .p2align 4, 0x90
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB3_4: # %l2.header
; X64-LFENCE-NEXT: # Parent Loop BB3_2 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # => This Inner Loop Header: Depth=2
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq (%rbx), %rax
; X64-LFENCE-NEXT: movl (%r14,%rax,4), %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: incl %ebp
; X64-LFENCE-NEXT: cmpl %r13d, %ebp
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jl .LBB3_4
; X64-LFENCE-NEXT: .LBB3_5: # %l1.latch
; X64-LFENCE-NEXT: # in Loop: Header=BB3_2 Depth=1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq (%rbx), %rax
; X64-LFENCE-NEXT: movl (%r14,%rax,4), %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: incl %r12d
; X64-LFENCE-NEXT: cmpl %r15d, %r12d
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jl .LBB3_2
; X64-LFENCE-NEXT: .LBB3_6: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: addq $8, %rsp
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r12
; X64-LFENCE-NEXT: popq %r13
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %r15
; X64-LFENCE-NEXT: popq %rbp
; X64-LFENCE-NEXT: retq
entry:
%a.cmp = icmp eq i32 %a, 0
br i1 %a.cmp, label %l1.header, label %exit
l1.header:
%i = phi i32 [ 0, %entry ], [ %i.next, %l1.latch ]
%b.cmp = icmp sgt i32 %b, 0
br i1 %b.cmp, label %l2.header, label %l1.latch
l2.header:
%j = phi i32 [ 0, %l1.header ], [ %j.next, %l2.header ]
%secret = load i32, i32* %ptr1
%ptr2.idx = getelementptr i32, i32* %ptr2, i32 %secret
%leak = load i32, i32* %ptr2.idx
call void @sink(i32 %leak)
%j.next = add i32 %j, 1
%j.cmp = icmp slt i32 %j.next, %c
br i1 %j.cmp, label %l2.header, label %l1.latch
l1.latch:
%secret2 = load i32, i32* %ptr1
%ptr2.idx2 = getelementptr i32, i32* %ptr2, i32 %secret2
%leak2 = load i32, i32* %ptr2.idx2
call void @sink(i32 %leak2)
%i.next = add i32 %i, 1
%i.cmp = icmp slt i32 %i.next, %b
br i1 %i.cmp, label %l1.header, label %exit
exit:
ret void
}
declare i32 @__gxx_personality_v0(...)
declare i8* @__cxa_allocate_exception(i64) local_unnamed_addr
declare void @__cxa_throw(i8*, i8*, i8*) local_unnamed_addr
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
define void @test_basic_eh(i32 %a, i32* %ptr1, i32* %ptr2) speculative_load_hardening personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LABEL: test_basic_eh:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %rbp
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 16
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %r15
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 24
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %r14
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 32
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %rbx
; X64-NEXT: .cfi_def_cfa_offset 40
; X64-NEXT: pushq %rax
; X64-NEXT: .cfi_def_cfa_offset 48
; X64-NEXT: .cfi_offset %rbx, -40
; X64-NEXT: .cfi_offset %r14, -32
; X64-NEXT: .cfi_offset %r15, -24
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_offset %rbp, -16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq $-1, %r15
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rax
; X64-NEXT: cmpl $41, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jg .LBB4_1
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: # %bb.2: # %thrower
; X64-NEXT: movq %rdx, %r14
; X64-NEXT: movq %rsi, %rbx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovgq %r15, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movslq %edi, %rcx
; X64-NEXT: movl (%rsi,%rcx,4), %ebp
; X64-NEXT: orl %eax, %ebp
; X64-NEXT: movl $4, %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq __cxa_allocate_exception
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr4:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rdx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr4, %rdx
; X64-NEXT: cmovneq %r15, %rcx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movl %ebp, (%rax)
; X64-NEXT: .Ltmp0:
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: movq %rax, %rdi
Bias physical register immediate assignments The machine scheduler currently biases register copies to/from physical registers to be closer to their point of use / def to minimize their live ranges. This change extends this to also physical register assignments from immediate values. This causes a reduction in reduction in overall register pressure and minor reduction in spills and indirectly fixes an out-of-registers assertion (PR39391). Most test changes are from minor instruction reorderings and register name selection changes and direct consequences of that. Reviewers: MatzeB, qcolombet, myatsina, pcc Subscribers: nemanjai, jvesely, nhaehnle, eraman, hiraditya, javed.absar, arphaman, jfb, jsji, llvm-commits Differential Revision: https://reviews.llvm.org/D54218 llvm-svn: 346894
2018-11-15 05:11:53 +08:00
; X64-NEXT: xorl %esi, %esi
; X64-NEXT: xorl %edx, %edx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: callq __cxa_throw
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr5:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr5, %rcx
; X64-NEXT: cmovneq %r15, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .Ltmp1:
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: jmp .LBB4_3
; X64-NEXT: .LBB4_1:
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmovleq %r15, %rax
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB4_3: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: addq $8, %rsp
; X64-NEXT: .cfi_def_cfa_offset 40
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: popq %rbx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 32
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: popq %r14
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 24
; X64-NEXT: popq %r15
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: popq %rbp
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 8
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: retq
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-NEXT: .LBB4_4: # %lpad
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .cfi_def_cfa_offset 48
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: .Ltmp2:
; X64-NEXT: movq %rsp, %rcx
; X64-NEXT: sarq $63, %rcx
; X64-NEXT: movl (%rax), %eax
; X64-NEXT: addl (%rbx), %eax
; X64-NEXT: cltq
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: orq %rcx, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movl (%r14,%rax,4), %edi
; X64-NEXT: orl %ecx, %edi
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr6:
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr6, %rcx
; X64-NEXT: cmovneq %r15, %rax
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
;
; X64-LFENCE-LABEL: test_basic_eh:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %rbp
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: pushq %r14
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 24
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: pushq %rbx
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 32
; X64-LFENCE-NEXT: .cfi_offset %rbx, -32
; X64-LFENCE-NEXT: .cfi_offset %r14, -24
; X64-LFENCE-NEXT: .cfi_offset %rbp, -16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: cmpl $41, %edi
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: jg .LBB4_2
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: # %bb.1: # %thrower
; X64-LFENCE-NEXT: movq %rdx, %r14
; X64-LFENCE-NEXT: movq %rsi, %rbx
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: movslq %edi, %rax
; X64-LFENCE-NEXT: movl (%rsi,%rax,4), %ebp
; X64-LFENCE-NEXT: movl $4, %edi
; X64-LFENCE-NEXT: callq __cxa_allocate_exception
; X64-LFENCE-NEXT: movl %ebp, (%rax)
; X64-LFENCE-NEXT: .Ltmp0:
Bias physical register immediate assignments The machine scheduler currently biases register copies to/from physical registers to be closer to their point of use / def to minimize their live ranges. This change extends this to also physical register assignments from immediate values. This causes a reduction in reduction in overall register pressure and minor reduction in spills and indirectly fixes an out-of-registers assertion (PR39391). Most test changes are from minor instruction reorderings and register name selection changes and direct consequences of that. Reviewers: MatzeB, qcolombet, myatsina, pcc Subscribers: nemanjai, jvesely, nhaehnle, eraman, hiraditya, javed.absar, arphaman, jfb, jsji, llvm-commits Differential Revision: https://reviews.llvm.org/D54218 llvm-svn: 346894
2018-11-15 05:11:53 +08:00
; X64-LFENCE-NEXT: movq %rax, %rdi
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: xorl %esi, %esi
; X64-LFENCE-NEXT: xorl %edx, %edx
; X64-LFENCE-NEXT: callq __cxa_throw
; X64-LFENCE-NEXT: .Ltmp1:
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB4_2: # %exit
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: lfence
; X64-LFENCE-NEXT: popq %rbx
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 24
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: popq %r14
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 16
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: popq %rbp
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 8
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: retq
[x86/SLH] Fix an issue where we wouldn't harden any loads if we found no conditions. This is only valid to do if we're hardening calls and rets with LFENCE which results in an LFENCE guarding the entire entry block for us. llvm-svn: 337089
2018-07-14 17:32:37 +08:00
; X64-LFENCE-NEXT: .LBB4_3: # %lpad
[x86/slh] Add unwind info to several tests to make it more obvious that we aren't incorrectly generating any of it when doing SLH. There was a bug that only occured with SLH that very much looked like it could be caused by bad unwind info, and so this was a prime suspect. Turns out that everything is fine, but this way we'll *see* if we end up, for example, putting things we shouldn't inside the prolog. llvm-svn: 338480
2018-08-01 11:01:10 +08:00
; X64-LFENCE-NEXT: .cfi_def_cfa_offset 32
[SLH] Introduce a new pass to do Speculative Load Hardening to mitigate Spectre variant #1 for x86. There is a lengthy, detailed RFC thread on llvm-dev which discusses the high level issues. High level discussion is probably best there. I've split the design document out of this patch and will land it separately once I update it to reflect the latest edits and updates to the Google doc used in the RFC thread. This patch is really just an initial step. It isn't quite ready for prime time and is only exposed via debugging flags. It has two major limitations currently: 1) It only supports x86-64, and only certain ABIs. Many assumptions are currently hard-coded and need to be factored out of the code here. 2) It doesn't include any options for more fine-grained control, either of which control flow edges are significant or which loads are important to be hardened. 3) The code is still quite rough and the testing lighter than I'd like. However, this is enough for people to begin using. I have had numerous requests from people to be able to experiment with this patch to understand the trade-offs it presents and how to use it. We would also like to encourage work to similar effect in other toolchains. The ARM folks are actively developing a system based on this for AArch64. We hope to merge this with their efforts when both are far enough along. But we also don't want to block making this available on that effort. Many thanks to the *numerous* people who helped along the way here. For this patch in particular, both Eric and Craig did a ton of review to even have confidence in it as an early, rough cut at this functionality. Differential Revision: https://reviews.llvm.org/D44824 llvm-svn: 336990
2018-07-13 19:13:58 +08:00
; X64-LFENCE-NEXT: .Ltmp2:
; X64-LFENCE-NEXT: movl (%rax), %eax
; X64-LFENCE-NEXT: addl (%rbx), %eax
; X64-LFENCE-NEXT: cltq
; X64-LFENCE-NEXT: movl (%r14,%rax,4), %edi
; X64-LFENCE-NEXT: callq sink
entry:
%a.cmp = icmp slt i32 %a, 42
br i1 %a.cmp, label %thrower, label %exit
thrower:
%badidx = getelementptr i32, i32* %ptr1, i32 %a
%secret1 = load i32, i32* %badidx
%e.ptr = call i8* @__cxa_allocate_exception(i64 4)
%e.ptr.cast = bitcast i8* %e.ptr to i32*
store i32 %secret1, i32* %e.ptr.cast
invoke void @__cxa_throw(i8* %e.ptr, i8* null, i8* null)
to label %exit unwind label %lpad
exit:
ret void
lpad:
%e = landingpad { i8*, i32 }
catch i8* null
%e.catch.ptr = extractvalue { i8*, i32 } %e, 0
%e.catch.ptr.cast = bitcast i8* %e.catch.ptr to i32*
%secret1.catch = load i32, i32* %e.catch.ptr.cast
%secret2 = load i32, i32* %ptr1
%secret.sum = add i32 %secret1.catch, %secret2
%ptr2.idx = getelementptr i32, i32* %ptr2, i32 %secret.sum
%leak = load i32, i32* %ptr2.idx
call void @sink(i32 %leak)
unreachable
}
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
declare void @sink_float(float)
declare void @sink_double(double)
; Test direct and converting loads of floating point values.
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
define void @test_fp_loads(float* %fptr, double* %dptr, i32* %i32ptr, i64* %i64ptr) nounwind speculative_load_hardening {
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-LABEL: test_fp_loads:
; X64: # %bb.0: # %entry
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %r13
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: pushq %r12
; X64-NEXT: pushq %rbx
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq %rcx, %r15
; X64-NEXT: movq %rdx, %r14
; X64-NEXT: movq %rsi, %rbx
; X64-NEXT: movq %rdi, %r12
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq $-1, %r13
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %r12
; X64-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_float
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr7:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr7, %rcx
; X64-NEXT: cmovneq %r13, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: orq %rax, %rbx
; X64-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_double
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr8:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr8, %rcx
; X64-NEXT: cmovneq %r13, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; X64-NEXT: cvtsd2ss %xmm0, %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_float
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr9:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr9, %rcx
; X64-NEXT: cmovneq %r13, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; X64-NEXT: cvtss2sd %xmm0, %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_double
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr10:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr10, %rcx
; X64-NEXT: cmovneq %r13, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: orq %rax, %r14
; X64-NEXT: xorps %xmm0, %xmm0
; X64-NEXT: cvtsi2ssl (%r14), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_float
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr11:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr11, %rcx
; X64-NEXT: cmovneq %r13, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: orq %rax, %r15
; X64-NEXT: xorps %xmm0, %xmm0
; X64-NEXT: cvtsi2sdq (%r15), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_double
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr12:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr12, %rcx
; X64-NEXT: cmovneq %r13, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: xorps %xmm0, %xmm0
; X64-NEXT: cvtsi2ssq (%r15), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_float
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr13:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr13, %rcx
; X64-NEXT: cmovneq %r13, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: xorps %xmm0, %xmm0
; X64-NEXT: cvtsi2sdl (%r14), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_double
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr14:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr14, %rcx
; X64-NEXT: cmovneq %r13, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r12
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: popq %r13
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_fp_loads:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %r15
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %r12
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: movq %rcx, %r15
; X64-LFENCE-NEXT: movq %rdx, %r14
; X64-LFENCE-NEXT: movq %rsi, %rbx
; X64-LFENCE-NEXT: movq %rdi, %r12
; X64-LFENCE-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; X64-LFENCE-NEXT: callq sink_float
; X64-LFENCE-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; X64-LFENCE-NEXT: callq sink_double
; X64-LFENCE-NEXT: movsd {{.*#+}} xmm0 = mem[0],zero
; X64-LFENCE-NEXT: cvtsd2ss %xmm0, %xmm0
; X64-LFENCE-NEXT: callq sink_float
; X64-LFENCE-NEXT: movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
; X64-LFENCE-NEXT: cvtss2sd %xmm0, %xmm0
; X64-LFENCE-NEXT: callq sink_double
; X64-LFENCE-NEXT: xorps %xmm0, %xmm0
; X64-LFENCE-NEXT: cvtsi2ssl (%r14), %xmm0
; X64-LFENCE-NEXT: callq sink_float
; X64-LFENCE-NEXT: xorps %xmm0, %xmm0
; X64-LFENCE-NEXT: cvtsi2sdq (%r15), %xmm0
; X64-LFENCE-NEXT: callq sink_double
; X64-LFENCE-NEXT: xorps %xmm0, %xmm0
; X64-LFENCE-NEXT: cvtsi2ssq (%r15), %xmm0
; X64-LFENCE-NEXT: callq sink_float
; X64-LFENCE-NEXT: xorps %xmm0, %xmm0
; X64-LFENCE-NEXT: cvtsi2sdl (%r14), %xmm0
; X64-LFENCE-NEXT: callq sink_double
; X64-LFENCE-NEXT: addq $8, %rsp
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r12
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %r15
; X64-LFENCE-NEXT: retq
entry:
%f1 = load float, float* %fptr
call void @sink_float(float %f1)
%d1 = load double, double* %dptr
call void @sink_double(double %d1)
%f2.d = load double, double* %dptr
%f2 = fptrunc double %f2.d to float
call void @sink_float(float %f2)
%d2.f = load float, float* %fptr
%d2 = fpext float %d2.f to double
call void @sink_double(double %d2)
%f3.i = load i32, i32* %i32ptr
%f3 = sitofp i32 %f3.i to float
call void @sink_float(float %f3)
%d3.i = load i64, i64* %i64ptr
%d3 = sitofp i64 %d3.i to double
call void @sink_double(double %d3)
%f4.i = load i64, i64* %i64ptr
%f4 = sitofp i64 %f4.i to float
call void @sink_float(float %f4)
%d4.i = load i32, i32* %i32ptr
%d4 = sitofp i32 %d4.i to double
call void @sink_double(double %d4)
ret void
}
declare void @sink_v4f32(<4 x float>)
declare void @sink_v2f64(<2 x double>)
declare void @sink_v16i8(<16 x i8>)
declare void @sink_v8i16(<8 x i16>)
declare void @sink_v4i32(<4 x i32>)
declare void @sink_v2i64(<2 x i64>)
; Test loads of vectors.
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
define void @test_vec_loads(<4 x float>* %v4f32ptr, <2 x double>* %v2f64ptr, <16 x i8>* %v16i8ptr, <8 x i16>* %v8i16ptr, <4 x i32>* %v4i32ptr, <2 x i64>* %v2i64ptr) nounwind speculative_load_hardening {
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-LABEL: test_vec_loads:
; X64: # %bb.0: # %entry
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %rbp
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: pushq %r15
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %r13
; X64-NEXT: pushq %r12
; X64-NEXT: pushq %rbx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq %r9, %r14
; X64-NEXT: movq %r8, %r15
; X64-NEXT: movq %rcx, %r12
; X64-NEXT: movq %rdx, %r13
; X64-NEXT: movq %rsi, %rbx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq $-1, %rbp
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
; X64-NEXT: orq %rax, %rdi
; X64-NEXT: movaps (%rdi), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v4f32
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr15:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr15, %rcx
; X64-NEXT: cmovneq %rbp, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: orq %rax, %rbx
; X64-NEXT: movaps (%rbx), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v2f64
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr16:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr16, %rcx
; X64-NEXT: cmovneq %rbp, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: orq %rax, %r13
; X64-NEXT: movaps (%r13), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v16i8
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr17:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr17, %rcx
; X64-NEXT: cmovneq %rbp, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: orq %rax, %r12
; X64-NEXT: movaps (%r12), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v8i16
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr18:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr18, %rcx
; X64-NEXT: cmovneq %rbp, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: orq %rax, %r15
; X64-NEXT: movaps (%r15), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v4i32
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr19:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr19, %rcx
; X64-NEXT: cmovneq %rbp, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: orq %rax, %r14
; X64-NEXT: movaps (%r14), %xmm0
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink_v2i64
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr20:
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr20, %rcx
; X64-NEXT: cmovneq %rbp, %rax
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: addq $8, %rsp
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r12
; X64-NEXT: popq %r13
; X64-NEXT: popq %r14
; X64-NEXT: popq %r15
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: popq %rbp
[x86/SLH] Fix a bug where we would try to post-load harden non-GPRs. Found cases that hit the assert I added. This patch factors the validity checking into a nice helper routine and calls it when deciding to harden post-load, and asserts it when doing so later. I've added tests for the various ways of loading a floating point type, as well as loading all vector permutations. Even though many of these go to identical instructions, it seems good to somewhat comprehensively test them. I'm confident there will be more fixes needed here, I'll try to add tests each time as I get this predicate adjusted. llvm-svn: 337160
2018-07-16 19:38:48 +08:00
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_vec_loads:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %r15
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %r13
; X64-LFENCE-NEXT: pushq %r12
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: movq %r9, %r14
; X64-LFENCE-NEXT: movq %r8, %r15
; X64-LFENCE-NEXT: movq %rcx, %r12
; X64-LFENCE-NEXT: movq %rdx, %r13
; X64-LFENCE-NEXT: movq %rsi, %rbx
; X64-LFENCE-NEXT: movaps (%rdi), %xmm0
; X64-LFENCE-NEXT: callq sink_v4f32
; X64-LFENCE-NEXT: movaps (%rbx), %xmm0
; X64-LFENCE-NEXT: callq sink_v2f64
; X64-LFENCE-NEXT: movaps (%r13), %xmm0
; X64-LFENCE-NEXT: callq sink_v16i8
; X64-LFENCE-NEXT: movaps (%r12), %xmm0
; X64-LFENCE-NEXT: callq sink_v8i16
; X64-LFENCE-NEXT: movaps (%r15), %xmm0
; X64-LFENCE-NEXT: callq sink_v4i32
; X64-LFENCE-NEXT: movaps (%r14), %xmm0
; X64-LFENCE-NEXT: callq sink_v2i64
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r12
; X64-LFENCE-NEXT: popq %r13
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: popq %r15
; X64-LFENCE-NEXT: retq
entry:
%x1 = load <4 x float>, <4 x float>* %v4f32ptr
call void @sink_v4f32(<4 x float> %x1)
%x2 = load <2 x double>, <2 x double>* %v2f64ptr
call void @sink_v2f64(<2 x double> %x2)
%x3 = load <16 x i8>, <16 x i8>* %v16i8ptr
call void @sink_v16i8(<16 x i8> %x3)
%x4 = load <8 x i16>, <8 x i16>* %v8i16ptr
call void @sink_v8i16(<8 x i16> %x4)
%x5 = load <4 x i32>, <4 x i32>* %v4i32ptr
call void @sink_v4i32(<4 x i32> %x5)
%x6 = load <2 x i64>, <2 x i64>* %v2i64ptr
call void @sink_v2i64(<2 x i64> %x6)
ret void
}
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
[x86/SLH] Add a real Clang flag and LLVM IR attribute for Speculative Load Hardening. Wires up the existing pass to work with a proper IR attribute rather than just a hidden/internal flag. The internal flag continues to work for now, but I'll likely remove it soon. Most of the churn here is adding the IR attribute. I talked about this Kristof Beyls and he seemed at least initially OK with this direction. The idea of using a full attribute here is that we *do* expect at least some forms of this for other architectures. There isn't anything *inherently* x86-specific about this technique, just that we only have an implementation for x86 at the moment. While we could potentially expose this as a Clang-level attribute as well, that seems like a good question to defer for the moment as it isn't 100% clear whether that or some other programmer interface (or both?) would be best. We'll defer the programmer interface side of this for now, but at least get to the point where the feature can be enabled without relying on implementation details. This also allows us to do something that was really hard before: we can enable *just* the indirect call retpolines when using SLH. For x86, we don't have any other way to mitigate indirect calls. Other architectures may take a different approach of course, and none of this is surfaced to user-level flags. Differential Revision: https://reviews.llvm.org/D51157 llvm-svn: 341363
2018-09-04 20:38:00 +08:00
define void @test_deferred_hardening(i32* %ptr1, i32* %ptr2, i32 %x) nounwind speculative_load_hardening {
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-LABEL: test_deferred_hardening:
; X64: # %bb.0: # %entry
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %r15
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: pushq %r14
; X64-NEXT: pushq %rbx
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq %rsi, %r14
; X64-NEXT: movq %rdi, %rbx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq $-1, %r15
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: sarq $63, %rax
; X64-NEXT: movl (%rdi), %edi
; X64-NEXT: incl %edi
; X64-NEXT: imull %edx, %edi
; X64-NEXT: orl %eax, %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr21:
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr21, %rcx
; X64-NEXT: cmovneq %r15, %rax
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: movl (%rbx), %ecx
; X64-NEXT: movl (%r14), %edx
; X64-NEXT: leal 1(%rcx,%rdx), %edi
; X64-NEXT: orl %eax, %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr22:
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr22, %rcx
; X64-NEXT: cmovneq %r15, %rax
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: movl (%rbx), %edi
; X64-NEXT: shll $7, %edi
; X64-NEXT: orl %eax, %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr23:
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr23, %rcx
; X64-NEXT: cmovneq %r15, %rax
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: movzwl (%rbx), %ecx
; X64-NEXT: sarw $7, %cx
; X64-NEXT: movzwl %cx, %edi
; X64-NEXT: notl %edi
; X64-NEXT: orl %eax, %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr24:
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr24, %rcx
; X64-NEXT: cmovneq %r15, %rax
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: movzwl (%rbx), %ecx
; X64-NEXT: rolw $9, %cx
; X64-NEXT: movswl %cx, %edi
; X64-NEXT: negl %edi
; X64-NEXT: orl %eax, %edi
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq sink
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr25:
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rcx
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr25, %rcx
; X64-NEXT: cmovneq %r15, %rax
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: popq %rbx
; X64-NEXT: popq %r14
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: popq %r15
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-NEXT: retq
;
; X64-LFENCE-LABEL: test_deferred_hardening:
; X64-LFENCE: # %bb.0: # %entry
; X64-LFENCE-NEXT: pushq %r14
; X64-LFENCE-NEXT: pushq %rbx
; X64-LFENCE-NEXT: pushq %rax
; X64-LFENCE-NEXT: movq %rsi, %r14
; X64-LFENCE-NEXT: movq %rdi, %rbx
; X64-LFENCE-NEXT: movl (%rdi), %edi
; X64-LFENCE-NEXT: incl %edi
; X64-LFENCE-NEXT: imull %edx, %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: movl (%rbx), %eax
; X64-LFENCE-NEXT: movl (%r14), %ecx
; X64-LFENCE-NEXT: leal 1(%rax,%rcx), %edi
; X64-LFENCE-NEXT: callq sink
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
; X64-LFENCE-NEXT: movl (%rbx), %edi
; X64-LFENCE-NEXT: shll $7, %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: movzwl (%rbx), %eax
; X64-LFENCE-NEXT: sarw $7, %ax
; X64-LFENCE-NEXT: movzwl %ax, %edi
; X64-LFENCE-NEXT: notl %edi
; X64-LFENCE-NEXT: callq sink
; X64-LFENCE-NEXT: movzwl (%rbx), %eax
; X64-LFENCE-NEXT: rolw $9, %ax
; X64-LFENCE-NEXT: movswl %ax, %edi
; X64-LFENCE-NEXT: negl %edi
; X64-LFENCE-NEXT: callq sink
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
; X64-LFENCE-NEXT: addq $8, %rsp
; X64-LFENCE-NEXT: popq %rbx
; X64-LFENCE-NEXT: popq %r14
; X64-LFENCE-NEXT: retq
entry:
%a1 = load i32, i32* %ptr1
%a2 = add i32 %a1, 1
%a3 = mul i32 %a2, %x
call void @sink(i32 %a3)
%b1 = load i32, i32* %ptr1
%b2 = add i32 %b1, 1
%b3 = load i32, i32* %ptr2
%b4 = add i32 %b2, %b3
call void @sink(i32 %b4)
[x86/SLH] Flesh out the data-invariant instruction table a bit based on feedback from Craig. Summary: The only thing he suggested that I've skipped here is the double-wide multiply instructions. Multiply is an area I'm nervous about there being some hidden data-dependent behavior, and it doesn't seem important for any benchmarks I have, so skipping it and sticking with the minimal multiply support that matches what I know is widely used in existing crypto libraries. We can always add double-wide multiply when we have clarity from vendors about its behavior and guarantees. I've tried to at least cover the fundamentals here with tests, although I've not tried to cover every width or permutation. I can add more tests where folks think it would be helpful. Reviewers: craig.topper Subscribers: sanjoy, mcrosier, hiraditya, llvm-commits Differential Revision: https://reviews.llvm.org/D49413 llvm-svn: 337308
2018-07-18 02:07:59 +08:00
%c1 = load i32, i32* %ptr1
%c2 = shl i32 %c1, 7
call void @sink(i32 %c2)
%d1 = load i32, i32* %ptr1
; Check trunc and integer ops narrower than i32.
%d2 = trunc i32 %d1 to i16
%d3 = ashr i16 %d2, 7
%d4 = zext i16 %d3 to i32
%d5 = xor i32 %d4, -1
call void @sink(i32 %d5)
%e1 = load i32, i32* %ptr1
%e2 = trunc i32 %e1 to i16
%e3 = lshr i16 %e2, 7
%e4 = shl i16 %e2, 9
%e5 = or i16 %e3, %e4
%e6 = sext i16 %e5 to i32
%e7 = sub i32 0, %e6
call void @sink(i32 %e7)
[x86/SLH] Completely rework how we sink post-load hardening past data invariant instructions to be both more correct and much more powerful. While testing, I continued to find issues with sinking post-load hardening. Unfortunately, it was amazingly hard to create any useful tests of this because we were mostly sinking across copies and other loading instructions. The fact that we couldn't sink past normal arithmetic was really a big oversight. So first, I've ported roughly the same set of instructions from the data invariant loads to also have their non-loading varieties understood to be data invariant. I've also added a few instructions that came up so often it again made testing complicated: inc, dec, and lea. With this, I was able to shake out a few nasty bugs in the validity checking. We need to restrict to hardening single-def instructions with defined registers that match a particular form: GPRs that don't have a NOREX constraint directly attached to their register class. The (tiny!) test case included catches all of the issues I was seeing (once we can sink the hardening at all) except for the NOREX issue. The only test I have there is horrible. It is large, inexplicable, and doesn't even produce an error unless you try to emit encodings. I can keep looking for a way to test it, but I'm out of ideas really. Thanks to Ben for giving me at least a sanity-check review. I'll follow up with Craig to go over this more thoroughly post-commit, but without it SLH crashes everywhere so landing it for now. Differential Revision: https://reviews.llvm.org/D49378 llvm-svn: 337177
2018-07-16 22:58:32 +08:00
ret void
}