maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/llvm/test/CodeGen/X86/speculative-load-hardening-...

815 lines
26 KiB
LLVM
Raw Normal View History

[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -x86-speculative-load-hardening -data-sections | FileCheck %s --check-prefix=X64
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -x86-speculative-load-hardening -relocation-model pic -data-sections | FileCheck %s --check-prefix=X64-PIC
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu -x86-speculative-load-hardening -data-sections -mattr=+retpoline | FileCheck %s --check-prefix=X64-RETPOLINE
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
;
; FIXME: Add support for 32-bit.
@global_fnptr = external global i32 ()*
@global_blockaddrs = constant [4 x i8*] [
i8* blockaddress(@test_indirectbr_global, %bb0),
i8* blockaddress(@test_indirectbr_global, %bb1),
i8* blockaddress(@test_indirectbr_global, %bb2),
i8* blockaddress(@test_indirectbr_global, %bb3)
]
define i32 @test_indirect_call(i32 ()** %ptr) nounwind {
; X64-LABEL: test_indirect_call:
; X64: # %bb.0: # %entry
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %rbx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq $-1, %rbx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: sarq $63, %rax
; X64-NEXT: movq (%rdi), %rcx
; X64-NEXT: orq %rax, %rcx
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
; X64-NEXT: callq *%rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr0:
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: movq %rsp, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rdx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr0, %rdx
; X64-NEXT: cmovneq %rbx, %rcx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: popq %rbx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: retq
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
;
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-LABEL: test_indirect_call:
; X64-PIC: # %bb.0: # %entry
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: pushq %rbx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: movq $-1, %rbx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: sarq $63, %rax
; X64-PIC-NEXT: movq (%rdi), %rcx
; X64-PIC-NEXT: orq %rax, %rcx
; X64-PIC-NEXT: shlq $47, %rax
; X64-PIC-NEXT: orq %rax, %rsp
; X64-PIC-NEXT: callq *%rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: .Lslh_ret_addr0:
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: movq %rsp, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: movq -{{[0-9]+}}(%rsp), %rdx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: leaq .Lslh_ret_addr0(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rbx, %rcx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: shlq $47, %rcx
; X64-PIC-NEXT: orq %rcx, %rsp
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: popq %rbx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: retq
;
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-LABEL: test_indirect_call:
; X64-RETPOLINE: # %bb.0: # %entry
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: pushq %rbx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: movq $-1, %rbx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: sarq $63, %rax
; X64-RETPOLINE-NEXT: movq (%rdi), %r11
; X64-RETPOLINE-NEXT: orq %rax, %r11
; X64-RETPOLINE-NEXT: shlq $47, %rax
; X64-RETPOLINE-NEXT: orq %rax, %rsp
; X64-RETPOLINE-NEXT: callq __llvm_retpoline_r11
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: .Lslh_ret_addr0:
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: movq %rsp, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: movq -{{[0-9]+}}(%rsp), %rdx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: cmpq $.Lslh_ret_addr0, %rdx
; X64-RETPOLINE-NEXT: cmovneq %rbx, %rcx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: shlq $47, %rcx
; X64-RETPOLINE-NEXT: orq %rcx, %rsp
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: popq %rbx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: retq
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
entry:
%fp = load i32 ()*, i32 ()** %ptr
%v = call i32 %fp()
ret i32 %v
}
define i32 @test_indirect_tail_call(i32 ()** %ptr) nounwind {
; X64-LABEL: test_indirect_tail_call:
; X64: # %bb.0: # %entry
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq $-1, %rcx
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: movq (%rdi), %rcx
; X64-NEXT: orq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: jmpq *%rcx # TAILCALL
;
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-LABEL: test_indirect_tail_call:
; X64-PIC: # %bb.0: # %entry
; X64-PIC-NEXT: movq %rsp, %rax
; X64-PIC-NEXT: movq $-1, %rcx
; X64-PIC-NEXT: sarq $63, %rax
; X64-PIC-NEXT: movq (%rdi), %rcx
; X64-PIC-NEXT: orq %rax, %rcx
; X64-PIC-NEXT: shlq $47, %rax
; X64-PIC-NEXT: orq %rax, %rsp
; X64-PIC-NEXT: jmpq *%rcx # TAILCALL
;
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-LABEL: test_indirect_tail_call:
; X64-RETPOLINE: # %bb.0: # %entry
; X64-RETPOLINE-NEXT: movq %rsp, %rax
; X64-RETPOLINE-NEXT: movq $-1, %rcx
; X64-RETPOLINE-NEXT: sarq $63, %rax
; X64-RETPOLINE-NEXT: movq (%rdi), %r11
; X64-RETPOLINE-NEXT: orq %rax, %r11
; X64-RETPOLINE-NEXT: shlq $47, %rax
; X64-RETPOLINE-NEXT: orq %rax, %rsp
; X64-RETPOLINE-NEXT: jmp __llvm_retpoline_r11 # TAILCALL
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
entry:
%fp = load i32 ()*, i32 ()** %ptr
%v = tail call i32 %fp()
ret i32 %v
}
define i32 @test_indirect_call_global() nounwind {
; X64-LABEL: test_indirect_call_global:
; X64: # %bb.0: # %entry
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: pushq %rbx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq $-1, %rbx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: movq global_fnptr(%rip), %rcx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: orq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: callq *%rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: .Lslh_ret_addr1:
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movq %rsp, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: movq -{{[0-9]+}}(%rsp), %rdx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: cmpq $.Lslh_ret_addr1, %rdx
; X64-NEXT: cmovneq %rbx, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-NEXT: popq %rbx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: retq
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
;
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-LABEL: test_indirect_call_global:
; X64-PIC: # %bb.0: # %entry
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: pushq %rbx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: movq $-1, %rbx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: sarq $63, %rax
; X64-PIC-NEXT: movq global_fnptr@GOTPCREL(%rip), %rcx
; X64-PIC-NEXT: movq (%rcx), %rcx
; X64-PIC-NEXT: orq %rax, %rcx
; X64-PIC-NEXT: shlq $47, %rax
; X64-PIC-NEXT: orq %rax, %rsp
; X64-PIC-NEXT: callq *%rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: .Lslh_ret_addr1:
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: movq %rsp, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: movq -{{[0-9]+}}(%rsp), %rdx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: leaq .Lslh_ret_addr1(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rbx, %rcx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: shlq $47, %rcx
; X64-PIC-NEXT: orq %rcx, %rsp
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-PIC-NEXT: popq %rbx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-NEXT: retq
;
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-LABEL: test_indirect_call_global:
; X64-RETPOLINE: # %bb.0: # %entry
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: pushq %rbx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: movq %rsp, %rax
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: movq $-1, %rbx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-RETPOLINE-NEXT: movq global_fnptr(%rip), %r11
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: shlq $47, %rax
; X64-RETPOLINE-NEXT: orq %rax, %rsp
; X64-RETPOLINE-NEXT: callq __llvm_retpoline_r11
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: .Lslh_ret_addr1:
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: movq %rsp, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: movq -{{[0-9]+}}(%rsp), %rdx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: cmpq $.Lslh_ret_addr1, %rdx
; X64-RETPOLINE-NEXT: cmovneq %rbx, %rcx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: shlq $47, %rcx
; X64-RETPOLINE-NEXT: orq %rcx, %rsp
[x86/SLH] Teach SLH to harden against the "ret2spec" attack by implementing the proposed mitigation technique described in the original design document. The idea is to check after calls that the return address used to arrive at that location is in fact the correct address. In the event of a mis-predicted return which reaches a *valid* return but not the *correct* return, this will detect the mismatch much like it would a mispredicted conditional branch. This is the last published attack vector that I am aware of in the Spectre v1 space which is not mitigated by SLH+retpolines. However, don't read *too* much into that: this is an area of ongoing research where we expect more issues to be discovered in the future, and it also makes no attempt to mitigate Spectre v4. Still, this is an important completeness bar for SLH. The change here is of course delightfully simple. It was predicated on cutting support for post-instruction symbols into LLVM which was not at all simple. Many thanks to Hal Finkel, Reid Kleckner, and Justin Bogner who helped me figure out how to do a bunch of the complex changes involved there. Differential Revision: https://reviews.llvm.org/D50837 llvm-svn: 341358
2018-09-04 18:59:10 +08:00
; X64-RETPOLINE-NEXT: popq %rbx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: retq
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
entry:
%fp = load i32 ()*, i32 ()** @global_fnptr
%v = call i32 %fp()
ret i32 %v
}
define i32 @test_indirect_tail_call_global() nounwind {
; X64-LABEL: test_indirect_tail_call_global:
; X64: # %bb.0: # %entry
; X64-NEXT: movq %rsp, %rax
; X64-NEXT: movq $-1, %rcx
; X64-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: movq global_fnptr(%rip), %rcx
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: orq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: shlq $47, %rax
; X64-NEXT: orq %rax, %rsp
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-NEXT: jmpq *%rcx # TAILCALL
;
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-LABEL: test_indirect_tail_call_global:
; X64-PIC: # %bb.0: # %entry
; X64-PIC-NEXT: movq %rsp, %rax
; X64-PIC-NEXT: movq $-1, %rcx
; X64-PIC-NEXT: sarq $63, %rax
; X64-PIC-NEXT: movq global_fnptr@GOTPCREL(%rip), %rcx
; X64-PIC-NEXT: movq (%rcx), %rcx
; X64-PIC-NEXT: orq %rax, %rcx
; X64-PIC-NEXT: shlq $47, %rax
; X64-PIC-NEXT: orq %rax, %rsp
; X64-PIC-NEXT: jmpq *%rcx # TAILCALL
;
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-LABEL: test_indirect_tail_call_global:
; X64-RETPOLINE: # %bb.0: # %entry
; X64-RETPOLINE-NEXT: movq %rsp, %rax
; X64-RETPOLINE-NEXT: movq $-1, %rcx
; X64-RETPOLINE-NEXT: sarq $63, %rax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-RETPOLINE-NEXT: movq global_fnptr(%rip), %r11
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-NEXT: shlq $47, %rax
; X64-RETPOLINE-NEXT: orq %rax, %rsp
; X64-RETPOLINE-NEXT: jmp __llvm_retpoline_r11 # TAILCALL
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
entry:
%fp = load i32 ()*, i32 ()** @global_fnptr
%v = tail call i32 %fp()
ret i32 %v
}
define i32 @test_indirectbr(i8** %ptr) nounwind {
; X64-LABEL: test_indirectbr:
; X64: # %bb.0: # %entry
; X64-NEXT: movq %rsp, %rcx
; X64-NEXT: movq $-1, %rax
; X64-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: movq (%rdi), %rdx
; X64-NEXT: orq %rcx, %rdx
; X64-NEXT: jmpq *%rdx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: .LBB4_1: # %bb0
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB4_1, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $2, %eax
; X64-NEXT: jmp .LBB4_2
; X64-NEXT: .LBB4_4: # %bb2
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB4_4, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $13, %eax
; X64-NEXT: jmp .LBB4_2
; X64-NEXT: .LBB4_5: # %bb3
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB4_5, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $42, %eax
; X64-NEXT: jmp .LBB4_2
; X64-NEXT: .LBB4_3: # %bb1
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB4_3, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $7, %eax
; X64-NEXT: .LBB4_2: # %bb0
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: retq
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
;
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-LABEL: test_indirectbr:
; X64-PIC: # %bb.0: # %entry
; X64-PIC-NEXT: movq %rsp, %rcx
; X64-PIC-NEXT: movq $-1, %rax
; X64-PIC-NEXT: sarq $63, %rcx
; X64-PIC-NEXT: movq (%rdi), %rdx
; X64-PIC-NEXT: orq %rcx, %rdx
; X64-PIC-NEXT: jmpq *%rdx
; X64-PIC-NEXT: .LBB4_1: # %bb0
; X64-PIC-NEXT: leaq .LBB4_1(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $2, %eax
; X64-PIC-NEXT: jmp .LBB4_2
; X64-PIC-NEXT: .LBB4_4: # %bb2
; X64-PIC-NEXT: leaq .LBB4_4(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $13, %eax
; X64-PIC-NEXT: jmp .LBB4_2
; X64-PIC-NEXT: .LBB4_5: # %bb3
; X64-PIC-NEXT: leaq .LBB4_5(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $42, %eax
; X64-PIC-NEXT: jmp .LBB4_2
; X64-PIC-NEXT: .LBB4_3: # %bb1
; X64-PIC-NEXT: leaq .LBB4_3(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $7, %eax
; X64-PIC-NEXT: .LBB4_2: # %bb0
; X64-PIC-NEXT: shlq $47, %rcx
; X64-PIC-NEXT: orq %rcx, %rsp
; X64-PIC-NEXT: retq
;
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-LABEL: test_indirectbr:
; X64-RETPOLINE: # %bb.0: # %entry
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
entry:
%a = load i8*, i8** %ptr
indirectbr i8* %a, [ label %bb0, label %bb1, label %bb2, label %bb3 ]
bb0:
ret i32 2
bb1:
ret i32 7
bb2:
ret i32 13
bb3:
ret i32 42
}
define i32 @test_indirectbr_global(i32 %idx) nounwind {
; X64-LABEL: test_indirectbr_global:
; X64: # %bb.0: # %entry
; X64-NEXT: movq %rsp, %rcx
; X64-NEXT: movq $-1, %rax
; X64-NEXT: sarq $63, %rcx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: movslq %edi, %rdx
; X64-NEXT: movq global_blockaddrs(,%rdx,8), %rdx
; X64-NEXT: orq %rcx, %rdx
; X64-NEXT: jmpq *%rdx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: .Ltmp0: # Block address taken
; X64-NEXT: .LBB5_1: # %bb0
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB5_1, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $2, %eax
; X64-NEXT: jmp .LBB5_2
; X64-NEXT: .Ltmp1: # Block address taken
; X64-NEXT: .LBB5_4: # %bb2
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB5_4, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $13, %eax
; X64-NEXT: jmp .LBB5_2
; X64-NEXT: .Ltmp2: # Block address taken
; X64-NEXT: .LBB5_5: # %bb3
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB5_5, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $42, %eax
; X64-NEXT: jmp .LBB5_2
; X64-NEXT: .Ltmp3: # Block address taken
; X64-NEXT: .LBB5_3: # %bb1
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB5_3, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $7, %eax
; X64-NEXT: .LBB5_2: # %bb0
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: retq
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
;
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-LABEL: test_indirectbr_global:
; X64-PIC: # %bb.0: # %entry
; X64-PIC-NEXT: movq %rsp, %rcx
; X64-PIC-NEXT: movq $-1, %rax
; X64-PIC-NEXT: sarq $63, %rcx
; X64-PIC-NEXT: movslq %edi, %rdx
; X64-PIC-NEXT: movq global_blockaddrs@GOTPCREL(%rip), %rsi
; X64-PIC-NEXT: movq (%rsi,%rdx,8), %rdx
; X64-PIC-NEXT: orq %rcx, %rdx
; X64-PIC-NEXT: jmpq *%rdx
; X64-PIC-NEXT: .Ltmp0: # Block address taken
; X64-PIC-NEXT: .LBB5_1: # %bb0
; X64-PIC-NEXT: leaq .LBB5_1(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $2, %eax
; X64-PIC-NEXT: jmp .LBB5_2
; X64-PIC-NEXT: .Ltmp1: # Block address taken
; X64-PIC-NEXT: .LBB5_4: # %bb2
; X64-PIC-NEXT: leaq .LBB5_4(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $13, %eax
; X64-PIC-NEXT: jmp .LBB5_2
; X64-PIC-NEXT: .Ltmp2: # Block address taken
; X64-PIC-NEXT: .LBB5_5: # %bb3
; X64-PIC-NEXT: leaq .LBB5_5(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $42, %eax
; X64-PIC-NEXT: jmp .LBB5_2
; X64-PIC-NEXT: .Ltmp3: # Block address taken
; X64-PIC-NEXT: .LBB5_3: # %bb1
; X64-PIC-NEXT: leaq .LBB5_3(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $7, %eax
; X64-PIC-NEXT: .LBB5_2: # %bb0
; X64-PIC-NEXT: shlq $47, %rcx
; X64-PIC-NEXT: orq %rcx, %rsp
; X64-PIC-NEXT: retq
;
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-LABEL: test_indirectbr_global:
; X64-RETPOLINE: # %bb.0: # %entry
; X64-RETPOLINE-NEXT: movq %rsp, %rcx
; X64-RETPOLINE-NEXT: movq $-1, %rax
; X64-RETPOLINE-NEXT: sarq $63, %rcx
; X64-RETPOLINE-NEXT: movslq %edi, %rdx
; X64-RETPOLINE-NEXT: movq global_blockaddrs(,%rdx,8), %rdx
; X64-RETPOLINE-NEXT: orq %rcx, %rdx
; X64-RETPOLINE-NEXT: cmpq $2, %rdx
; X64-RETPOLINE-NEXT: je .LBB6_5
; X64-RETPOLINE-NEXT: # %bb.1: # %entry
; X64-RETPOLINE-NEXT: cmoveq %rax, %rcx
; X64-RETPOLINE-NEXT: cmpq $3, %rdx
; X64-RETPOLINE-NEXT: je .LBB6_6
; X64-RETPOLINE-NEXT: # %bb.2: # %entry
; X64-RETPOLINE-NEXT: cmoveq %rax, %rcx
; X64-RETPOLINE-NEXT: cmpq $4, %rdx
; X64-RETPOLINE-NEXT: jne .LBB6_3
; X64-RETPOLINE-NEXT: .Ltmp0: # Block address taken
; X64-RETPOLINE-NEXT: # %bb.7: # %bb3
; X64-RETPOLINE-NEXT: cmovneq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $42, %eax
; X64-RETPOLINE-NEXT: jmp .LBB6_4
; X64-RETPOLINE-NEXT: .Ltmp1: # Block address taken
; X64-RETPOLINE-NEXT: .LBB6_5: # %bb1
; X64-RETPOLINE-NEXT: cmovneq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $7, %eax
; X64-RETPOLINE-NEXT: jmp .LBB6_4
; X64-RETPOLINE-NEXT: .Ltmp2: # Block address taken
; X64-RETPOLINE-NEXT: .LBB6_6: # %bb2
; X64-RETPOLINE-NEXT: cmovneq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $13, %eax
; X64-RETPOLINE-NEXT: jmp .LBB6_4
; X64-RETPOLINE-NEXT: .Ltmp3: # Block address taken
; X64-RETPOLINE-NEXT: .LBB6_3: # %bb0
; X64-RETPOLINE-NEXT: cmoveq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $2, %eax
; X64-RETPOLINE-NEXT: .LBB6_4: # %bb0
; X64-RETPOLINE-NEXT: shlq $47, %rcx
; X64-RETPOLINE-NEXT: orq %rcx, %rsp
; X64-RETPOLINE-NEXT: retq
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
entry:
%ptr = getelementptr [4 x i8*], [4 x i8*]* @global_blockaddrs, i32 0, i32 %idx
%a = load i8*, i8** %ptr
indirectbr i8* %a, [ label %bb0, label %bb1, label %bb2, label %bb3 ]
bb0:
ret i32 2
bb1:
ret i32 7
bb2:
ret i32 13
bb3:
ret i32 42
}
; This function's switch is crafted to trigger jump-table lowering in the x86
; backend so that we can test how the exact jump table lowering behaves.
define i32 @test_switch_jumptable(i32 %idx) nounwind {
; X64-LABEL: test_switch_jumptable:
; X64: # %bb.0: # %entry
; X64-NEXT: movq %rsp, %rcx
; X64-NEXT: movq $-1, %rax
; X64-NEXT: sarq $63, %rcx
; X64-NEXT: cmpl $3, %edi
; X64-NEXT: ja .LBB6_2
; X64-NEXT: # %bb.1: # %entry
; X64-NEXT: cmovaq %rax, %rcx
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: movl %edi, %edx
; X64-NEXT: movq .LJTI6_0(,%rdx,8), %rdx
; X64-NEXT: orq %rcx, %rdx
; X64-NEXT: jmpq *%rdx
; X64-NEXT: .LBB6_4: # %bb1
; X64-NEXT: cmpq $.LBB6_4, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $7, %eax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: jmp .LBB6_3
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: .LBB6_2: # %bb0
; X64-NEXT: cmovbeq %rax, %rcx
; X64-NEXT: movl $2, %eax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: jmp .LBB6_3
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: .LBB6_5: # %bb2
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB6_5, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $13, %eax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: jmp .LBB6_3
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: .LBB6_6: # %bb3
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB6_6, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $42, %eax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: jmp .LBB6_3
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: .LBB6_7: # %bb5
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: cmpq $.LBB6_7, %rdx
; X64-NEXT: cmovneq %rax, %rcx
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: movl $11, %eax
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-NEXT: .LBB6_3: # %bb0
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
; X64-NEXT: shlq $47, %rcx
; X64-NEXT: orq %rcx, %rsp
; X64-NEXT: retq
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
;
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; X64-PIC-LABEL: test_switch_jumptable:
; X64-PIC: # %bb.0: # %entry
; X64-PIC-NEXT: movq %rsp, %rcx
; X64-PIC-NEXT: movq $-1, %rax
; X64-PIC-NEXT: sarq $63, %rcx
; X64-PIC-NEXT: cmpl $3, %edi
; X64-PIC-NEXT: ja .LBB6_2
; X64-PIC-NEXT: # %bb.1: # %entry
; X64-PIC-NEXT: cmovaq %rax, %rcx
; X64-PIC-NEXT: movl %edi, %edx
; X64-PIC-NEXT: leaq .LJTI6_0(%rip), %rsi
; X64-PIC-NEXT: movslq (%rsi,%rdx,4), %rdx
; X64-PIC-NEXT: addq %rsi, %rdx
; X64-PIC-NEXT: orq %rcx, %rdx
; X64-PIC-NEXT: jmpq *%rdx
; X64-PIC-NEXT: .LBB6_4: # %bb1
; X64-PIC-NEXT: leaq .LBB6_4(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $7, %eax
; X64-PIC-NEXT: jmp .LBB6_3
; X64-PIC-NEXT: .LBB6_2: # %bb0
; X64-PIC-NEXT: cmovbeq %rax, %rcx
; X64-PIC-NEXT: movl $2, %eax
; X64-PIC-NEXT: jmp .LBB6_3
; X64-PIC-NEXT: .LBB6_5: # %bb2
; X64-PIC-NEXT: leaq .LBB6_5(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $13, %eax
; X64-PIC-NEXT: jmp .LBB6_3
; X64-PIC-NEXT: .LBB6_6: # %bb3
; X64-PIC-NEXT: leaq .LBB6_6(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $42, %eax
; X64-PIC-NEXT: jmp .LBB6_3
; X64-PIC-NEXT: .LBB6_7: # %bb5
; X64-PIC-NEXT: leaq .LBB6_7(%rip), %rsi
; X64-PIC-NEXT: cmpq %rsi, %rdx
; X64-PIC-NEXT: cmovneq %rax, %rcx
; X64-PIC-NEXT: movl $11, %eax
; X64-PIC-NEXT: .LBB6_3: # %bb0
; X64-PIC-NEXT: shlq $47, %rcx
; X64-PIC-NEXT: orq %rcx, %rsp
; X64-PIC-NEXT: retq
;
[x86/SLH] Teach the x86 speculative load hardening pass to harden against v1.2 BCBS attacks directly. Attacks using spectre v1.2 (a subset of BCBS) are described in the paper here: https://people.csail.mit.edu/vlk/spectre11.pdf The core idea is to speculatively store over the address in a vtable, jumptable, or other target of indirect control flow that will be subsequently loaded. Speculative execution after such a store can forward the stored value to subsequent loads, and if called or jumped to, the speculative execution will be steered to this potentially attacker controlled address. Up until now, this could be mitigated by enableing retpolines. However, that is a relatively expensive technique to mitigate this particular flavor. Especially because in most cases SLH will have already mitigated this. To fully mitigate this with SLH, we need to do two core things: 1) Unfold loads from calls and jumps, allowing the loads to be post-load hardened. 2) Force hardening of incoming registers even if we didn't end up needing to harden the load itself. The reason we need to do these two things is because hardening calls and jumps from this particular variant is importantly different from hardening against leak of secret data. Because the "bad" data here isn't a secret, but in fact speculatively stored by the attacker, it may be loaded from any address, regardless of whether it is read-only memory, mapped memory, or a "hardened" address. The only 100% effective way to harden these instructions is to harden the their operand itself. But to the extent possible, we'd like to take advantage of all the other hardening going on, we just need a fallback in case none of that happened to cover the particular input to the control transfer instruction. For users of SLH, currently they are paing 2% to 6% performance overhead for retpolines, but this mechanism is expected to be substantially cheaper. However, it is worth reminding folks that this does not mitigate all of the things retpolines do -- most notably, variant #2 is not in *any way* mitigated by this technique. So users of SLH may still want to enable retpolines, and the implementation is carefuly designed to gracefully leverage retpolines to avoid the need for further hardening here when they are enabled. Differential Revision: https://reviews.llvm.org/D49663 llvm-svn: 337878
2018-07-25 09:51:29 +08:00
; X64-RETPOLINE-LABEL: test_switch_jumptable:
; X64-RETPOLINE: # %bb.0: # %entry
; X64-RETPOLINE-NEXT: movq %rsp, %rcx
; X64-RETPOLINE-NEXT: movq $-1, %rax
; X64-RETPOLINE-NEXT: sarq $63, %rcx
; X64-RETPOLINE-NEXT: cmpl $1, %edi
; X64-RETPOLINE-NEXT: jg .LBB7_4
; X64-RETPOLINE-NEXT: # %bb.1: # %entry
; X64-RETPOLINE-NEXT: cmovgq %rax, %rcx
; X64-RETPOLINE-NEXT: testl %edi, %edi
; X64-RETPOLINE-NEXT: je .LBB7_8
; X64-RETPOLINE-NEXT: # %bb.2: # %entry
; X64-RETPOLINE-NEXT: cmoveq %rax, %rcx
; X64-RETPOLINE-NEXT: cmpl $1, %edi
; X64-RETPOLINE-NEXT: jne .LBB7_6
; X64-RETPOLINE-NEXT: # %bb.3: # %bb2
; X64-RETPOLINE-NEXT: cmovneq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $13, %eax
; X64-RETPOLINE-NEXT: jmp .LBB7_7
; X64-RETPOLINE-NEXT: .LBB7_4: # %entry
; X64-RETPOLINE-NEXT: cmovleq %rax, %rcx
; X64-RETPOLINE-NEXT: cmpl $2, %edi
; X64-RETPOLINE-NEXT: je .LBB7_9
; X64-RETPOLINE-NEXT: # %bb.5: # %entry
; X64-RETPOLINE-NEXT: cmoveq %rax, %rcx
; X64-RETPOLINE-NEXT: cmpl $3, %edi
; X64-RETPOLINE-NEXT: jne .LBB7_6
; X64-RETPOLINE-NEXT: # %bb.10: # %bb5
; X64-RETPOLINE-NEXT: cmovneq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $11, %eax
; X64-RETPOLINE-NEXT: jmp .LBB7_7
; X64-RETPOLINE-NEXT: .LBB7_6:
; X64-RETPOLINE-NEXT: cmoveq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $2, %eax
; X64-RETPOLINE-NEXT: jmp .LBB7_7
; X64-RETPOLINE-NEXT: .LBB7_8: # %bb1
; X64-RETPOLINE-NEXT: cmovneq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $7, %eax
; X64-RETPOLINE-NEXT: jmp .LBB7_7
; X64-RETPOLINE-NEXT: .LBB7_9: # %bb3
; X64-RETPOLINE-NEXT: cmovneq %rax, %rcx
; X64-RETPOLINE-NEXT: movl $42, %eax
; X64-RETPOLINE-NEXT: .LBB7_7: # %bb0
; X64-RETPOLINE-NEXT: shlq $47, %rcx
; X64-RETPOLINE-NEXT: orq %rcx, %rsp
; X64-RETPOLINE-NEXT: retq
[x86/SLH] Add a test covering indirect forms of control flow. NFC. This specifically covers different ways of making indirect calls and jumps. There are some bugs in SLH that I will be fixing in subsequent patches where the diff in the generated instructions makes the bug fix much more clear, so just checking in a baseline of this test to start. I'm also going to be adding direct mitigation for variant 1.2 which this file very specifically tests in the various forms it can arise on x86. Again, the diff to the generated instructions should make the change for that much more clear, so having the test as a baseline seems useful. llvm-svn: 337672
2018-07-23 15:51:51 +08:00
entry:
switch i32 %idx, label %bb0 [
i32 0, label %bb1
i32 1, label %bb2
i32 2, label %bb3
i32 3, label %bb5
]
bb0:
ret i32 2
bb1:
ret i32 7
bb2:
ret i32 13
bb3:
ret i32 42
bb5:
ret i32 11
}
[x86/SLH] Teach SLH to harden indirect branches and switches without retpolines. This implements the core design of tracing the intended target into the target, checking it, and using that to update the predicate state. It takes advantage of a few interesting aspects of SLH to make it a bit easier to implement: - We already split critical edges with conditional branches, so we can assume those are gone. - We already unfolded any memory access in the indirect branch instruction itself. I've left hard errors in place to catch if any of these somewhat subtle invariants get violated. There is some code that I can factor out and share with D50837 when it lands, but I didn't want to couple landing the two patches, so I'll do that in a follow-up cleanup commit if alright. Factoring out the code to handle different scenarios of materializing an address remains frustratingly hard. In a bunch of cases you want to fold one of the cases into an immediate operand of some other instruction, and you also have both symbols and basic blocks being used which require different methods on the MI builder (and different operand kinds). Still, I'll take a stab at sharing at least some of this code in a follow-up if I can figure out how. Differential Revision: https://reviews.llvm.org/D51083 llvm-svn: 341356
2018-09-04 18:44:21 +08:00
; This function's switch is crafted to trigger jump-table lowering in the x86
; backend so that we can test how the exact jump table lowering behaves, but
; also arranges for fallthroughs from case to case to ensure that this pattern
; too can be handled.
define i32 @test_switch_jumptable_fallthrough(i32 %idx, i32* %a.ptr, i32* %b.ptr, i32* %c.ptr, i32* %d.ptr) nounwind {
; X64-LABEL: test_switch_jumptable_fallthrough:
; X64: # %bb.0: # %entry
; X64-NEXT: movq %rsp, %r9
; X64-NEXT: movq $-1, %r10
; X64-NEXT: sarq $63, %r9
; X64-NEXT: cmpl $3, %edi
; X64-NEXT: ja .LBB7_2
; X64-NEXT: # %bb.1: # %entry
; X64-NEXT: cmovaq %r10, %r9
; X64-NEXT: xorl %eax, %eax
; X64-NEXT: movl %edi, %esi
; X64-NEXT: movq .LJTI7_0(,%rsi,8), %rsi
; X64-NEXT: orq %r9, %rsi
; X64-NEXT: jmpq *%rsi
; X64-NEXT: .LBB7_2: # %bb0
; X64-NEXT: cmovbeq %r10, %r9
; X64-NEXT: movl (%rsi), %eax
; X64-NEXT: orl %r9d, %eax
; X64-NEXT: movq $.LBB7_3, %rsi
; X64-NEXT: .LBB7_3: # %bb1
; X64-NEXT: cmpq $.LBB7_3, %rsi
; X64-NEXT: cmovneq %r10, %r9
; X64-NEXT: addl (%rdx), %eax
; X64-NEXT: orl %r9d, %eax
; X64-NEXT: movq $.LBB7_4, %rsi
; X64-NEXT: .LBB7_4: # %bb2
; X64-NEXT: cmpq $.LBB7_4, %rsi
; X64-NEXT: cmovneq %r10, %r9
; X64-NEXT: addl (%rcx), %eax
; X64-NEXT: orl %r9d, %eax
; X64-NEXT: movq $.LBB7_5, %rsi
; X64-NEXT: .LBB7_5: # %bb3
; X64-NEXT: cmpq $.LBB7_5, %rsi
; X64-NEXT: cmovneq %r10, %r9
; X64-NEXT: addl (%r8), %eax
; X64-NEXT: orl %r9d, %eax
; X64-NEXT: movq $.LBB7_6, %rsi
; X64-NEXT: .LBB7_6: # %bb4
; X64-NEXT: cmpq $.LBB7_6, %rsi
; X64-NEXT: cmovneq %r10, %r9
; X64-NEXT: shlq $47, %r9
; X64-NEXT: orq %r9, %rsp
; X64-NEXT: retq
;
; X64-PIC-LABEL: test_switch_jumptable_fallthrough:
; X64-PIC: # %bb.0: # %entry
; X64-PIC-NEXT: movq %rsp, %r9
; X64-PIC-NEXT: movq $-1, %r10
; X64-PIC-NEXT: sarq $63, %r9
; X64-PIC-NEXT: cmpl $3, %edi
; X64-PIC-NEXT: ja .LBB7_2
; X64-PIC-NEXT: # %bb.1: # %entry
; X64-PIC-NEXT: cmovaq %r10, %r9
; X64-PIC-NEXT: xorl %eax, %eax
; X64-PIC-NEXT: movl %edi, %esi
; X64-PIC-NEXT: leaq .LJTI7_0(%rip), %rdi
; X64-PIC-NEXT: movslq (%rdi,%rsi,4), %rsi
; X64-PIC-NEXT: addq %rdi, %rsi
; X64-PIC-NEXT: orq %r9, %rsi
; X64-PIC-NEXT: jmpq *%rsi
; X64-PIC-NEXT: .LBB7_2: # %bb0
; X64-PIC-NEXT: cmovbeq %r10, %r9
; X64-PIC-NEXT: movl (%rsi), %eax
; X64-PIC-NEXT: orl %r9d, %eax
; X64-PIC-NEXT: leaq .LBB7_3(%rip), %rsi
; X64-PIC-NEXT: .LBB7_3: # %bb1
; X64-PIC-NEXT: leaq .LBB7_3(%rip), %rdi
; X64-PIC-NEXT: cmpq %rdi, %rsi
; X64-PIC-NEXT: cmovneq %r10, %r9
; X64-PIC-NEXT: addl (%rdx), %eax
; X64-PIC-NEXT: orl %r9d, %eax
; X64-PIC-NEXT: leaq .LBB7_4(%rip), %rsi
; X64-PIC-NEXT: .LBB7_4: # %bb2
; X64-PIC-NEXT: leaq .LBB7_4(%rip), %rdx
; X64-PIC-NEXT: cmpq %rdx, %rsi
; X64-PIC-NEXT: cmovneq %r10, %r9
; X64-PIC-NEXT: addl (%rcx), %eax
; X64-PIC-NEXT: orl %r9d, %eax
; X64-PIC-NEXT: leaq .LBB7_5(%rip), %rsi
; X64-PIC-NEXT: .LBB7_5: # %bb3
; X64-PIC-NEXT: leaq .LBB7_5(%rip), %rcx
; X64-PIC-NEXT: cmpq %rcx, %rsi
; X64-PIC-NEXT: cmovneq %r10, %r9
; X64-PIC-NEXT: addl (%r8), %eax
; X64-PIC-NEXT: orl %r9d, %eax
; X64-PIC-NEXT: leaq .LBB7_6(%rip), %rsi
; X64-PIC-NEXT: .LBB7_6: # %bb4
; X64-PIC-NEXT: leaq .LBB7_6(%rip), %rcx
; X64-PIC-NEXT: cmpq %rcx, %rsi
; X64-PIC-NEXT: cmovneq %r10, %r9
; X64-PIC-NEXT: shlq $47, %r9
; X64-PIC-NEXT: orq %r9, %rsp
; X64-PIC-NEXT: retq
;
; X64-RETPOLINE-LABEL: test_switch_jumptable_fallthrough:
; X64-RETPOLINE: # %bb.0: # %entry
; X64-RETPOLINE-NEXT: movq %rsp, %r9
; X64-RETPOLINE-NEXT: movq $-1, %r10
; X64-RETPOLINE-NEXT: sarq $63, %r9
; X64-RETPOLINE-NEXT: xorl %eax, %eax
; X64-RETPOLINE-NEXT: cmpl $1, %edi
; X64-RETPOLINE-NEXT: jg .LBB8_5
; X64-RETPOLINE-NEXT: # %bb.1: # %entry
; X64-RETPOLINE-NEXT: cmovgq %r10, %r9
; X64-RETPOLINE-NEXT: testl %edi, %edi
; X64-RETPOLINE-NEXT: je .LBB8_2
; X64-RETPOLINE-NEXT: # %bb.3: # %entry
; X64-RETPOLINE-NEXT: cmoveq %r10, %r9
; X64-RETPOLINE-NEXT: cmpl $1, %edi
; X64-RETPOLINE-NEXT: jne .LBB8_8
; X64-RETPOLINE-NEXT: # %bb.4:
; X64-RETPOLINE-NEXT: cmovneq %r10, %r9
; X64-RETPOLINE-NEXT: jmp .LBB8_10
; X64-RETPOLINE-NEXT: .LBB8_5: # %entry
; X64-RETPOLINE-NEXT: cmovleq %r10, %r9
; X64-RETPOLINE-NEXT: cmpl $2, %edi
; X64-RETPOLINE-NEXT: je .LBB8_6
; X64-RETPOLINE-NEXT: # %bb.7: # %entry
; X64-RETPOLINE-NEXT: cmoveq %r10, %r9
; X64-RETPOLINE-NEXT: cmpl $3, %edi
; X64-RETPOLINE-NEXT: jne .LBB8_8
; X64-RETPOLINE-NEXT: # %bb.13:
; X64-RETPOLINE-NEXT: cmovneq %r10, %r9
; X64-RETPOLINE-NEXT: jmp .LBB8_12
; X64-RETPOLINE-NEXT: .LBB8_8:
; X64-RETPOLINE-NEXT: cmoveq %r10, %r9
; X64-RETPOLINE-NEXT: movl (%rsi), %eax
; X64-RETPOLINE-NEXT: orl %r9d, %eax
; X64-RETPOLINE-NEXT: jmp .LBB8_9
; X64-RETPOLINE-NEXT: .LBB8_2:
; X64-RETPOLINE-NEXT: cmovneq %r10, %r9
; X64-RETPOLINE-NEXT: .LBB8_9: # %bb1
; X64-RETPOLINE-NEXT: addl (%rdx), %eax
; X64-RETPOLINE-NEXT: orl %r9d, %eax
; X64-RETPOLINE-NEXT: .LBB8_10: # %bb2
; X64-RETPOLINE-NEXT: addl (%rcx), %eax
; X64-RETPOLINE-NEXT: orl %r9d, %eax
; X64-RETPOLINE-NEXT: jmp .LBB8_11
; X64-RETPOLINE-NEXT: .LBB8_6:
; X64-RETPOLINE-NEXT: cmovneq %r10, %r9
; X64-RETPOLINE-NEXT: .LBB8_11: # %bb3
; X64-RETPOLINE-NEXT: addl (%r8), %eax
; X64-RETPOLINE-NEXT: orl %r9d, %eax
; X64-RETPOLINE-NEXT: .LBB8_12: # %bb4
; X64-RETPOLINE-NEXT: shlq $47, %r9
; X64-RETPOLINE-NEXT: orq %r9, %rsp
; X64-RETPOLINE-NEXT: retq
entry:
switch i32 %idx, label %bb0 [
i32 0, label %bb1
i32 1, label %bb2
i32 2, label %bb3
i32 3, label %bb4
]
bb0:
%a = load i32, i32* %a.ptr
br label %bb1
bb1:
%b.phi = phi i32 [ 0, %entry ], [ %a, %bb0 ]
%b = load i32, i32* %b.ptr
%b.sum = add i32 %b.phi, %b
br label %bb2
bb2:
%c.phi = phi i32 [ 0, %entry ], [ %b.sum, %bb1 ]
%c = load i32, i32* %c.ptr
%c.sum = add i32 %c.phi, %c
br label %bb3
bb3:
%d.phi = phi i32 [ 0, %entry ], [ %c.sum, %bb2 ]
%d = load i32, i32* %d.ptr
%d.sum = add i32 %d.phi, %d
br label %bb4
bb4:
%e.phi = phi i32 [ 0, %entry ], [ %d.sum, %bb3 ]
ret i32 %e.phi
}