maxjhandsome
/
llvm-project
forked from OSchip/llvm-project
1
0
Fork 0
Code Issues Pull Requests Packages Releases Wiki Activity
llvm-project/clang/test/CodeGenCXX/auto-var-init.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

1634 lines
88 KiB
C++
Raw Normal View History

CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks %s -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK,CHECK-O0
// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O0,PATTERN,PATTERN-O0
[clang][NewPM] Add -fno-experimental-new-pass-manager to tests As per the discussion on D58375, we disable test that have optimizations under the new PM. This patch adds -fno-experimental-new-pass-manager to RUNS that: - Already run with optimizations (-O1 or higher) that were missed in D58375. - Explicitly test new PM behavior along side some new PM RUNS, but are missing this flag if new PM is enabled by default. - Specify -O without the number. Based on getOptimizationLevel(), it seems the default is 2, and the IR appears to be the same when changed to -O2, so update the test to explicitly say -O2 and provide -fno-experimental-new-pass-manager`. Differential Revision: https://reviews.llvm.org/D63156 llvm-svn: 364066
2019-06-22 00:03:06 +08:00
// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -O1 -fno-experimental-new-pass-manager -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O1,PATTERN,PATTERN-O1,PATTERN-O1-LEGACY
// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -O1 -fexperimental-new-pass-manager -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O1,PATTERN,PATTERN-O1,PATTERN-O1-NEWPM
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O0,ZERO,ZERO-O0
[clang][NewPM] Add -fno-experimental-new-pass-manager to tests As per the discussion on D58375, we disable test that have optimizations under the new PM. This patch adds -fno-experimental-new-pass-manager to RUNS that: - Already run with optimizations (-O1 or higher) that were missed in D58375. - Explicitly test new PM behavior along side some new PM RUNS, but are missing this flag if new PM is enabled by default. - Specify -O without the number. Based on getOptimizationLevel(), it seems the default is 2, and the IR appears to be the same when changed to -O2, so update the test to explicitly say -O2 and provide -fno-experimental-new-pass-manager`. Differential Revision: https://reviews.llvm.org/D63156 llvm-svn: 364066
2019-06-22 00:03:06 +08:00
// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -O1 -fno-experimental-new-pass-manager -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O1,ZERO,ZERO-O1,ZERO-O1-LEGACY
// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -O1 -fexperimental-new-pass-manager -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O1,ZERO,ZERO-O1,ZERO-O1-NEWPM
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// RUN: %clang_cc1 -std=c++14 -triple i386-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O0,PATTERN,PATTERN-O0
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
CodeGen: Suppress c++ warnings in test llvm-svn: 365835
2019-07-12 05:59:09 +08:00
#pragma clang diagnostic ignored "-Winaccessible-base"
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
#ifdef __x86_64__
char inits[] = {"-86/-21846/-1431655766/i64/-6148914691236517206/-6148914691236517206/i128/-113427455640312821154458202477256070486/i64/-6148914691236517206/AA/"};
#else
CodeGet: Init 32bit pointers with 0xFFFFFFFF Summary: Patch makes D63967 effective for 32bit platforms and improves pattern initialization there. It cuts size of 32bit binary compiled with -ftrivial-auto-var-init=pattern by 2% (3% with -Os). Binary size change on CTMark, (with -fuse-ld=lld -Wl,--icf=all, similar results with default linker options) ``` master patch diff Os pattern 7.915580e+05 7.698424e+05 -0.028387 O3 pattern 9.953688e+05 9.752952e+05 -0.019325 ``` Zero vs Pattern on master ``` zero pattern diff Os 7.689712e+05 7.915580e+05 0.031380 O3 9.744796e+05 9.953688e+05 0.021133 ``` Zero vs Pattern with the patch ``` zero pattern diff Os 7.689712e+05 7.698424e+05 0.000789 O3 9.744796e+05 9.752952e+05 0.000742 ``` Reviewers: pcc, eugenis, glider, jfb Reviewed By: jfb Subscribers: hubert.reinterpretcast, dexonsmith, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D64597 llvm-svn: 365921
2019-07-13 01:21:55 +08:00
char inits[] = {"-1/-1/-1/i32/-1/-1/i32/-1/i32/-1/FF/"};
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
#define __int128 int;
#endif
// PATTERN: @inits = {{.*}} c"[[I8:[^/]+]]/[[I16:[^/]+]]/[[I32:[^/]+]]/[[ILONGT:[^/]+]]/[[ILONG:[^/]+]]/[[I64:[^/]+]]/[[I128T:[^/]+]]/[[I128:[^/]+]]/[[IPTRT:[^/]+]]/[[IPTR:[^/]+]]/[[IC:[^/]+]]/\00", align 1
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
template<typename T> void used(T &) noexcept;
#define TEST_UNINIT(NAME, TYPE) \
using type_##NAME = TYPE; \
void test_##NAME##_uninit() { \
type_##NAME uninit; \
used(uninit); \
}
// Value initialization on scalars, aggregate initialization on aggregates.
#define TEST_BRACES(NAME, TYPE) \
using type_##NAME = TYPE; \
void test_##NAME##_braces() { \
type_##NAME braces = {}; \
used(braces); \
}
#define TEST_CUSTOM(NAME, TYPE, ...) \
using type_##NAME = TYPE; \
void test_##NAME##_custom() { \
type_##NAME custom __VA_ARGS__; \
used(custom); \
}
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// None of the synthesized globals should contain `undef`.
// PATTERN-NOT: undef
// ZERO-NOT: undef
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_empty_uninit.uninit = private unnamed_addr constant %struct.empty { i8 [[I8]] }, align 1
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_empty_uninit.uninit
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct empty {};
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_small_uninit.uninit = private unnamed_addr constant %struct.small { i8 [[I8]] }, align 1
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: @__const.test_small_custom.custom = private unnamed_addr constant %struct.small { i8 42 }, align 1
// ZERO-O0: @__const.test_small_custom.custom = private unnamed_addr constant %struct.small { i8 42 }, align 1
// PATTERN-O1-NOT: @__const.test_small_uninit.uninit
// PATTERN-O1-NOT: @__const.test_small_custom.custom
// ZERO-O1-NOT: @__const.test_small_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct small { char c; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_smallinit_uninit.uninit = private unnamed_addr constant %struct.smallinit { i8 [[I8]] }, align 1
// PATTERN-O0: @__const.test_smallinit_braces.braces = private unnamed_addr constant %struct.smallinit { i8 [[I8]] }, align 1
// PATTERN-O0: @__const.test_smallinit_custom.custom = private unnamed_addr constant %struct.smallinit { i8 [[I8]] }, align 1
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_smallinit_uninit.uninit
// PATTERN-O1-NOT: @__const.test_smallinit_braces.braces
// PATTERN-O1-NOT: @__const.test_smallinit_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct smallinit { char c = 42; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_smallpartinit_uninit.uninit = private unnamed_addr constant %struct.smallpartinit { i8 [[I8]], i8 [[I8]] }, align 1
// PATTERN-O0: @__const.test_smallpartinit_braces.braces = private unnamed_addr constant %struct.smallpartinit { i8 [[I8]], i8 [[I8]] }, align 1
// PATTERN-O0: @__const.test_smallpartinit_custom.custom = private unnamed_addr constant %struct.smallpartinit { i8 [[I8]], i8 [[I8]] }, align 1
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_smallpartinit_uninit.uninit
// PATTERN-O1-NOT: @__const.test_smallpartinit_braces.braces
// PATTERN-O1-NOT: @__const.test_smallpartinit_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct smallpartinit { char c = 42, d; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_nullinit_uninit.uninit = private unnamed_addr constant %struct.nullinit { i8* inttoptr ([[IPTRT]] [[IPTR]] to i8*) }, align
// PATTERN-O0: @__const.test_nullinit_braces.braces = private unnamed_addr constant %struct.nullinit { i8* inttoptr ([[IPTRT]] [[IPTR]] to i8*) }, align
// PATTERN-O0: @__const.test_nullinit_custom.custom = private unnamed_addr constant %struct.nullinit { i8* inttoptr ([[IPTRT]] [[IPTR]] to i8*) }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_nullinit_uninit.uninit
// PATTERN-O1-NOT: @__const.test_nullinit_braces.braces
// PATTERN-O1-NOT: @__const.test_nullinit_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct nullinit { char* null = nullptr; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_padded_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i32 [[I32]] }, align 4
Fix auto-init test r359628 changed the initialization of padding to follow C, but I didn't update the C++ tests. llvm-svn: 359636
2019-05-01 07:27:28 +08:00
// PATTERN-O0: @__const.test_padded_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 42, [3 x i8] zeroinitializer, i32 13371337 }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: @__const.test_padded_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 42, [3 x i8] zeroinitializer, i32 13371337 }, align 4
// PATTERN-O1-NOT: @__const.test_padded_uninit.uninit
// PATTERN-O1-NOT: @__const.test_padded_custom.custom
// ZERO-O1-NOT: @__const.test_padded_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct padded { char c; int i; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_paddednullinit_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i32 [[I32]] }, align 4
// PATTERN-O0: @__const.test_paddednullinit_braces.braces = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i32 [[I32]] }, align 4
// PATTERN-O0: @__const.test_paddednullinit_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i32 [[I32]] }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_paddednullinit_uninit.uninit
// PATTERN-O1-NOT: @__const.test_paddednullinit_braces.braces
// PATTERN-O1-NOT: @__const.test_paddednullinit_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct paddednullinit { char c = 0; int i = 0; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_paddedpacked_uninit.uninit = private unnamed_addr constant %struct.paddedpacked <{ i8 [[I8]], i32 [[I32]] }>, align 1
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// PATTERN: @__const.test_paddedpacked_custom.custom = private unnamed_addr constant %struct.paddedpacked <{ i8 42, i32 13371337 }>, align 1
// ZERO: @__const.test_paddedpacked_custom.custom = private unnamed_addr constant %struct.paddedpacked <{ i8 42, i32 13371337 }>, align 1
struct paddedpacked { char c; int i; } __attribute__((packed));
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_paddedpackedarray_uninit.uninit = private unnamed_addr constant %struct.paddedpackedarray { [2 x %struct.paddedpacked] [%struct.paddedpacked <{ i8 [[I8]], i32 [[I32]] }>, %struct.paddedpacked <{ i8 [[I8]], i32 [[I32]] }>] }, align 1
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// PATTERN: @__const.test_paddedpackedarray_custom.custom = private unnamed_addr constant %struct.paddedpackedarray { [2 x %struct.paddedpacked] [%struct.paddedpacked <{ i8 42, i32 13371337 }>, %struct.paddedpacked <{ i8 43, i32 13371338 }>] }, align 1
// ZERO: @__const.test_paddedpackedarray_custom.custom = private unnamed_addr constant %struct.paddedpackedarray { [2 x %struct.paddedpacked] [%struct.paddedpacked <{ i8 42, i32 13371337 }>, %struct.paddedpacked <{ i8 43, i32 13371338 }>] }, align 1
struct paddedpackedarray { struct paddedpacked p[2]; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_unpackedinpacked_uninit.uninit = private unnamed_addr constant <{ { i8, [3 x i8], i32 }, i8 }> <{ { i8, [3 x i8], i32 } { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i32 [[I32]] }, i8 [[I8]] }>, align 1
CodeGen: Preserve packed attribute in constStructWithPadding. Otherwise the object may have an incorrect size due to tail padding. Differential Revision: https://reviews.llvm.org/D59446 llvm-svn: 356328
2019-03-17 03:25:39 +08:00
struct unpackedinpacked { padded a; char b; } __attribute__((packed));
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_paddednested_uninit.uninit = private unnamed_addr constant { { i8, [3 x i8], i32 }, { i8, [3 x i8], i32 } } { { i8, [3 x i8], i32 } { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i32 [[I32]] }, { i8, [3 x i8], i32 } { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i32 [[I32]] } }, align 4
Fix auto-init test r359628 changed the initialization of padding to follow C, but I didn't update the C++ tests. llvm-svn: 359636
2019-05-01 07:27:28 +08:00
// PATTERN: @__const.test_paddednested_custom.custom = private unnamed_addr constant { { i8, [3 x i8], i32 }, { i8, [3 x i8], i32 } } { { i8, [3 x i8], i32 } { i8 42, [3 x i8] zeroinitializer, i32 13371337 }, { i8, [3 x i8], i32 } { i8 43, [3 x i8] zeroinitializer, i32 13371338 } }, align 4
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// ZERO: @__const.test_paddednested_custom.custom = private unnamed_addr constant { { i8, [3 x i8], i32 }, { i8, [3 x i8], i32 } } { { i8, [3 x i8], i32 } { i8 42, [3 x i8] zeroinitializer, i32 13371337 }, { i8, [3 x i8], i32 } { i8 43, [3 x i8] zeroinitializer, i32 13371338 } }, align 4
struct paddednested { struct padded p1, p2; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_paddedpackednested_uninit.uninit = private unnamed_addr constant %struct.paddedpackednested { %struct.paddedpacked <{ i8 [[I8]], i32 [[I32]] }>, %struct.paddedpacked <{ i8 [[I8]], i32 [[I32]] }> }, align 1
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// PATTERN: @__const.test_paddedpackednested_custom.custom = private unnamed_addr constant %struct.paddedpackednested { %struct.paddedpacked <{ i8 42, i32 13371337 }>, %struct.paddedpacked <{ i8 43, i32 13371338 }> }, align 1
// ZERO: @__const.test_paddedpackednested_custom.custom = private unnamed_addr constant %struct.paddedpackednested { %struct.paddedpacked <{ i8 42, i32 13371337 }>, %struct.paddedpacked <{ i8 43, i32 13371338 }> }, align 1
struct paddedpackednested { struct paddedpacked p1, p2; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_bitfield_uninit.uninit = private unnamed_addr constant %struct.bitfield { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]" }, align 4
// PATTERN-O0: @__const.test_bitfield_custom.custom = private unnamed_addr constant %struct.bitfield { i8 20, [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]" }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: @__const.test_bitfield_custom.custom = private unnamed_addr constant %struct.bitfield { i8 20, [3 x i8] zeroinitializer }, align 4
// PATTERN-O1-NOT: @__const.test_bitfield_uninit.uninit
// PATTERN-O1-NOT: @__const.test_bitfield_custom.custom
// ZERO-O1-NOT: @__const.test_bitfield_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct bitfield { int i : 4; int j : 2; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_bitfieldaligned_uninit.uninit = private unnamed_addr constant %struct.bitfieldaligned { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]" }, align 4
// PATTERN-O0: @__const.test_bitfieldaligned_custom.custom = private unnamed_addr constant %struct.bitfieldaligned { i8 4, [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i8 1, [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]" }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: @__const.test_bitfieldaligned_custom.custom = private unnamed_addr constant %struct.bitfieldaligned { i8 4, [3 x i8] zeroinitializer, i8 1, [3 x i8] zeroinitializer }, align 4
// PATTERN-O1-NOT: @__const.test_bitfieldaligned_uninit.uninit
// PATTERN-O1-NOT: @__const.test_bitfieldaligned_custom.custom
// ZERO-O1-NOT: @__const.test_bitfieldaligned_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct bitfieldaligned { int i : 4; int : 0; int j : 2; };
struct big { unsigned a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_arraytail_uninit.uninit = private unnamed_addr constant %struct.arraytail { i32 [[I32]], [0 x i32] zeroinitializer }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: @__const.test_arraytail_custom.custom = private unnamed_addr constant %struct.arraytail { i32 57005, [0 x i32] zeroinitializer }, align 4
// ZERO-O0: @__const.test_arraytail_custom.custom = private unnamed_addr constant %struct.arraytail { i32 57005, [0 x i32] zeroinitializer }, align 4
// PATTERN-O1-NOT: @__const.test_arraytail_uninit.uninit
// PATTERN-O1-NOT: @__const.test_arraytail_custom.custom
// ZERO-O1-NOT: @__const.test_arraytail_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct arraytail { int i; int arr[]; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_int1_uninit.uninit = private unnamed_addr constant [1 x i32] {{\[}}i32 [[I32]]], align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: @__const.test_int1_custom.custom = private unnamed_addr constant [1 x i32] [i32 858993459], align 4
// ZERO-O0: @__const.test_int1_custom.custom = private unnamed_addr constant [1 x i32] [i32 858993459], align 4
// PATTERN-O1-NOT: @__const.test_int1_uninit.uninit
// PATTERN-O1-NOT: @__const.test_int1_custom.custom
// ZERO-O1-NOT: @__const.test_int1_custom.custom
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_bool4_uninit.uninit = private unnamed_addr constant [4 x i8] c"\[[IC]]\[[IC]]\[[IC]]\[[IC]]", align 1
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: @__const.test_bool4_custom.custom = private unnamed_addr constant [4 x i8] c"\01\01\01\01", align 1
// ZERO-O0: @__const.test_bool4_custom.custom = private unnamed_addr constant [4 x i8] c"\01\01\01\01", align 1
// PATTERN-O1-NOT: @__const.test_bool4_uninit.uninit
// PATTERN-O1-NOT: @__const.test_bool4_custom.custom
// ZERO-O1-NOT: @__const.test_bool4_custom.custom
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: @__const.test_intptr4_custom.custom = private unnamed_addr constant [4 x i32*] [i32* inttoptr ([[IPTRT]] 572662306 to i32*), i32* inttoptr ([[IPTRT]] 572662306 to i32*), i32* inttoptr ([[IPTRT]] 572662306 to i32*), i32* inttoptr ([[IPTRT]] 572662306 to i32*)], align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO: @__const.test_intptr4_custom.custom = private unnamed_addr constant [4 x i32*] [i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*)], align 16
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_tailpad4_uninit.uninit = private unnamed_addr constant [4 x { i16, i8, [1 x i8] }] [{ i16, i8, [1 x i8] } { i16 [[I16]], i8 [[I8]], [1 x i8] c"\[[IC]]" }, { i16, i8, [1 x i8] } { i16 [[I16]], i8 [[I8]], [1 x i8] c"\[[IC]]" }, { i16, i8, [1 x i8] } { i16 [[I16]], i8 [[I8]], [1 x i8] c"\[[IC]]" }, { i16, i8, [1 x i8] } { i16 [[I16]], i8 [[I8]], [1 x i8] c"\[[IC]]" }], align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_tailpad4_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: @__const.test_tailpad4_custom.custom = private unnamed_addr constant [4 x { i16, i8, [1 x i8] }] [{ i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }], align
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// ZERO: @__const.test_tailpad4_custom.custom = private unnamed_addr constant [4 x { i16, i8, [1 x i8] }] [{ i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }], align 16
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct tailpad { short s; char c; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_atomicnotlockfree_uninit.uninit = private unnamed_addr constant %struct.notlockfree { [4 x i64] {{\[}}i64 [[I64]], i64 [[I64]], i64 [[I64]], i64 [[I64]]] }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_atomicnotlockfree_uninit.uninit
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct notlockfree { long long a[4]; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_atomicpadded_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 [[I8]], [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]", i32 [[I32]] }, align 8
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_atomicpadded_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_atomictailpad_uninit.uninit = private unnamed_addr constant { i16, i8, [1 x i8] } { i16 [[I16]], i8 [[I8]], [1 x i8] c"\[[IC]]" }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_atomictailpad_uninit.uninit
// PATTERN-O0: @__const.test_complexfloat_uninit.uninit = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
// PATTERN-O1-NOT: @__const.test_complexfloat_uninit.uninit
// PATTERN-O0: @__const.test_complexfloat_braces.braces = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
// PATTERN-O1-NOT: @__const.test_complexfloat_braces.braces
// PATTERN-O0: @__const.test_complexfloat_custom.custom = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
// PATTERN-O1-NOT: @__const.test_complexfloat_custom.custom
// PATTERN-O0: @__const.test_complexdouble_uninit.uninit = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
// PATTERN-O1-NOT: @__const.test_complexdouble_uninit.uninit
// PATTERN-O0: @__const.test_complexdouble_braces.braces = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
// PATTERN-O1-NOT: @__const.test_complexdouble_braces.braces
// PATTERN-O0: @__const.test_complexdouble_custom.custom = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
// PATTERN-O1-NOT: @__const.test_complexdouble_custom.custom
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_semivolatile_uninit.uninit = private unnamed_addr constant %struct.semivolatile { i32 [[I32]], i32 [[I32]] }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: @__const.test_semivolatile_custom.custom = private unnamed_addr constant %struct.semivolatile { i32 1145324612, i32 1145324612 }, align 4
// PATTERN-O1-NOT: @__const.test_semivolatile_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct semivolatile { int i; volatile int vi; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_semivolatileinit_uninit.uninit = private unnamed_addr constant %struct.semivolatileinit { i32 [[I32]], i32 [[I32]] }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_semivolatileinit_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_semivolatileinit_braces.braces = private unnamed_addr constant %struct.semivolatileinit { i32 [[I32]], i32 [[I32]] }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_semivolatileinit_braces.braces
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_semivolatileinit_custom.custom = private unnamed_addr constant %struct.semivolatileinit { i32 [[I32]], i32 [[I32]] }, align 4
// PATTERN-O1-NOT: @__const.test_semivolatileinit_custom.custom = private unnamed_addr constant %struct.semivolatileinit { i32 [[I32]], i32 [[I32]] }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: @__const.test_semivolatile_custom.custom = private unnamed_addr constant %struct.semivolatile { i32 1145324612, i32 1145324612 }, align 4
// ZERO-O1-NOT: @__const.test_semivolatile_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct semivolatileinit { int i = 0x11111111; volatile int vi = 0x11111111; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_base_uninit.uninit = private unnamed_addr constant %struct.base { i32 (...)** inttoptr ([[IPTRT]] [[IPTR]] to i32 (...)**) }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_base_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_base_braces.braces = private unnamed_addr constant %struct.base { i32 (...)** inttoptr ([[IPTRT]] [[IPTR]] to i32 (...)**) }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_base_braces.braces
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct base { virtual ~base(); };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_derived_uninit.uninit = private unnamed_addr constant %struct.derived { %struct.base { i32 (...)** inttoptr ([[IPTRT]] [[IPTR]] to i32 (...)**) } }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_derived_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_derived_braces.braces = private unnamed_addr constant %struct.derived { %struct.base { i32 (...)** inttoptr ([[IPTRT]] [[IPTR]] to i32 (...)**) } }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_derived_braces.braces
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct derived : public base {};
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_virtualderived_uninit.uninit = private unnamed_addr constant %struct.virtualderived { %struct.base { i32 (...)** inttoptr ([[IPTRT]] [[IPTR]] to i32 (...)**) }, %struct.derived { %struct.base { i32 (...)** inttoptr ([[IPTRT]] [[IPTR]] to i32 (...)**) } } }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_virtualderived_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_virtualderived_braces.braces = private unnamed_addr constant %struct.virtualderived { %struct.base { i32 (...)** inttoptr ([[IPTRT]] [[IPTR]] to i32 (...)**) }, %struct.derived { %struct.base { i32 (...)** inttoptr ([[IPTRT]] [[IPTR]] to i32 (...)**) } } }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_virtualderived_braces.braces
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
struct virtualderived : public virtual base, public virtual derived {};
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_matching_uninit.uninit = private unnamed_addr constant %union.matching { i32 [[I32]] }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_matching_uninit.uninit
// PATTERN-O0: @__const.test_matching_custom.custom = private unnamed_addr constant { float } { float 6.145500e+04 }, align 4
// PATTERN-O1-NOT: @__const.test_matching_custom.custom
// ZERO-O0: @__const.test_matching_custom.custom = private unnamed_addr constant { float } { float 6.145500e+04 }, align 4
// ZERO-O1-NOT: @__const.test_matching_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
union matching { int i; float f; };
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: @__const.test_matchingreverse_uninit.uninit = private unnamed_addr constant %union.matchingreverse { float 0xFFFFFFFFE0000000 }, align 4
// PATTERN-O1-NOT: @__const.test_matchingreverse_uninit.uninit
// PATTERN-O0: @__const.test_matchingreverse_custom.custom = private unnamed_addr constant { i32 } { i32 61455 }, align 4
// PATTERN-O1-NOT: @__const.test_matchingreverse_custom.custom
// ZERO-O0: @__const.test_matchingreverse_custom.custom = private unnamed_addr constant { i32 } { i32 61455 }, align 4
// ZERO-O1-NOT: @__const.test_matchingreverse_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
union matchingreverse { float f; int i; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_unmatched_uninit.uninit = private unnamed_addr constant %union.unmatched { i32 [[I32]] }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_unmatched_uninit.uninit
// PATTERN-O0: @__const.test_unmatched_custom.custom = private unnamed_addr constant %union.unmatched { i32 1001242351 }, align 4
// PATTERN-O1-NOT: @__const.test_unmatched_custom.custom
// ZERO-O0: @__const.test_unmatched_custom.custom = private unnamed_addr constant %union.unmatched { i32 1001242351 }, align 4
// ZERO-O1-NOT: @__const.test_unmatched_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
union unmatched { char c; int i; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_unmatchedreverse_uninit.uninit = private unnamed_addr constant %union.unmatchedreverse { i32 [[I32]] }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_unmatchedreverse_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_unmatchedreverse_custom.custom = private unnamed_addr constant { i8, [3 x i8] } { i8 42, [3 x i8] c"\[[IC]]\[[IC]]\[[IC]]" }, align 4
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_unmatchedreverse_custom.custom
// ZERO-O0: @__const.test_unmatchedreverse_custom.custom = private unnamed_addr constant { i8, [3 x i8] } { i8 42, [3 x i8] zeroinitializer }, align 4
// ZERO-O1-NOT: @__const.test_unmatchedreverse_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
union unmatchedreverse { int i; char c; };
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_unmatchedfp_uninit.uninit = private unnamed_addr constant %union.unmatchedfp { double 0xFFFFFFFFFFFFFFFF }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_unmatchedfp_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: @__const.test_unmatchedfp_custom.custom = private unnamed_addr constant %union.unmatchedfp { double 0x400921FB54442D18 }, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1-NOT: @__const.test_unmatchedfp_custom.custom
// ZERO-O0: @__const.test_unmatchedfp_custom.custom = private unnamed_addr constant %union.unmatchedfp { double 0x400921FB54442D18 }, align 8
// ZERO-O1-NOT: @__const.test_unmatchedfp_custom.custom
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
union unmatchedfp { float f; double d; };
enum emptyenum {};
enum smallenum { VALUE };
extern "C" {
TEST_UNINIT(char, char);
// CHECK-LABEL: @test_char_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i8, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_char_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i8 [[I8]], i8* %uninit, align 1
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_char_uninit()
// ZERO: store i8 0, i8* %uninit, align 1
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(char, char);
// CHECK-LABEL: @test_char_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i8, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i8 0, i8* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(uchar, unsigned char);
// CHECK-LABEL: @test_uchar_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i8, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_uchar_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i8 [[I8]], i8* %uninit, align 1
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_uchar_uninit()
// ZERO: store i8 0, i8* %uninit, align 1
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(uchar, unsigned char);
// CHECK-LABEL: @test_uchar_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i8, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i8 0, i8* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(schar, signed char);
// CHECK-LABEL: @test_schar_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i8, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_schar_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i8 [[I8]], i8* %uninit, align 1
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_schar_uninit()
// ZERO: store i8 0, i8* %uninit, align 1
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(schar, signed char);
// CHECK-LABEL: @test_schar_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i8, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i8 0, i8* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(wchar_t, wchar_t);
// CHECK-LABEL: @test_wchar_t_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_wchar_t_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i32 [[I32]], i32* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_wchar_t_uninit()
// ZERO: store i32 0, i32* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(wchar_t, wchar_t);
// CHECK-LABEL: @test_wchar_t_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i32, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32 0, i32* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(short, short);
// CHECK-LABEL: @test_short_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i16, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_short_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i16 [[I16]], i16* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_short_uninit()
// ZERO: store i16 0, i16* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(short, short);
// CHECK-LABEL: @test_short_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i16, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i16 0, i16* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(ushort, unsigned short);
// CHECK-LABEL: @test_ushort_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i16, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_ushort_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i16 [[I16]], i16* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_ushort_uninit()
// ZERO: store i16 0, i16* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(ushort, unsigned short);
// CHECK-LABEL: @test_ushort_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i16, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i16 0, i16* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(int, int);
// CHECK-LABEL: @test_int_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_int_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i32 [[I32]], i32* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_int_uninit()
// ZERO: store i32 0, i32* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(int, int);
// CHECK-LABEL: @test_int_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i32, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32 0, i32* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(unsigned, unsigned);
// CHECK-LABEL: @test_unsigned_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_unsigned_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i32 [[I32]], i32* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_unsigned_uninit()
// ZERO: store i32 0, i32* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(unsigned, unsigned);
// CHECK-LABEL: @test_unsigned_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i32, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32 0, i32* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(long, long);
// CHECK-LABEL: @test_long_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i64, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_long_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store [[ILONGT]] [[ILONG]], [[ILONGT]]* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_long_uninit()
// ZERO: store i64 0, i64* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(long, long);
// CHECK-LABEL: @test_long_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i64, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i64 0, i64* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(ulong, unsigned long);
// CHECK-LABEL: @test_ulong_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i64, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_ulong_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store [[ILONGT]] [[ILONG]], [[ILONGT]]* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_ulong_uninit()
// ZERO: store i64 0, i64* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(ulong, unsigned long);
// CHECK-LABEL: @test_ulong_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i64, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i64 0, i64* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(longlong, long long);
// CHECK-LABEL: @test_longlong_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i64, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_longlong_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i64 [[I64]], i64* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_longlong_uninit()
// ZERO: store i64 0, i64* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(longlong, long long);
// CHECK-LABEL: @test_longlong_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i64, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i64 0, i64* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(ulonglong, unsigned long long);
// CHECK-LABEL: @test_ulonglong_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i64, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_ulonglong_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i64 [[I64]], i64* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_ulonglong_uninit()
// ZERO: store i64 0, i64* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(ulonglong, unsigned long long);
// CHECK-LABEL: @test_ulonglong_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i64, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i64 0, i64* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(int128, __int128);
// CHECK-LABEL: @test_int128_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i128, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_int128_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store [[I128T]] [[I128]], [[I128T]]* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_int128_uninit()
// ZERO: store i128 0, i128* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(int128, __int128);
// CHECK-LABEL: @test_int128_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i128, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i128 0, i128* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(uint128, unsigned __int128);
// CHECK-LABEL: @test_uint128_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i128, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_uint128_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store [[I128T]] [[I128]], [[I128T]]* %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_uint128_uninit()
// ZERO: store i128 0, i128* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(uint128, unsigned __int128);
// CHECK-LABEL: @test_uint128_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i128, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i128 0, i128* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(fp16, __fp16);
// CHECK-LABEL: @test_fp16_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca half, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_fp16_uninit()
// PATTERN: store half 0xHFFFF, half* %uninit, align
// ZERO-LABEL: @test_fp16_uninit()
// ZERO: store half 0xH0000, half* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(fp16, __fp16);
// CHECK-LABEL: @test_fp16_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca half, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store half 0xH0000, half* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(float, float);
// CHECK-LABEL: @test_float_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca float, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_float_uninit()
// PATTERN: store float 0xFFFFFFFFE0000000, float* %uninit, align
// ZERO-LABEL: @test_float_uninit()
// ZERO: store float 0.000000e+00, float* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(float, float);
// CHECK-LABEL: @test_float_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca float, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store float 0.000000e+00, float* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(double, double);
// CHECK-LABEL: @test_double_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca double, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_double_uninit()
// PATTERN: store double 0xFFFFFFFFFFFFFFFF, double* %uninit, align
// ZERO-LABEL: @test_double_uninit()
// ZERO: store double 0.000000e+00, double* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(double, double);
// CHECK-LABEL: @test_double_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca double, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store double 0.000000e+00, double* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(longdouble, long double);
// CHECK-LABEL: @test_longdouble_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca x86_fp80, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_longdouble_uninit()
// PATTERN: store x86_fp80 0xKFFFFFFFFFFFFFFFFFFFF, x86_fp80* %uninit, align
// ZERO-LABEL: @test_longdouble_uninit()
// ZERO: store x86_fp80 0xK00000000000000000000, x86_fp80* %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(longdouble, long double);
// CHECK-LABEL: @test_longdouble_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca x86_fp80, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store x86_fp80 0xK00000000000000000000, x86_fp80* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(intptr, int*);
// CHECK-LABEL: @test_intptr_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32*, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_intptr_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i32* inttoptr ([[IPTRT]] [[IPTR]] to i32*), i32** %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_intptr_uninit()
// ZERO: store i32* null, i32** %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(intptr, int*);
// CHECK-LABEL: @test_intptr_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i32*, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32* null, i32** %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(intptrptr, int**);
// CHECK-LABEL: @test_intptrptr_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32**, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_intptrptr_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i32** inttoptr ([[IPTRT]] [[IPTR]] to i32**), i32*** %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_intptrptr_uninit()
// ZERO: store i32** null, i32*** %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(intptrptr, int**);
// CHECK-LABEL: @test_intptrptr_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i32**, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32** null, i32*** %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(function, void(*)());
// CHECK-LABEL: @test_function_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca void ()*, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_function_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store void ()* inttoptr ([[IPTRT]] [[IPTR]] to void ()*), void ()** %uninit, align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_function_uninit()
// ZERO: store void ()* null, void ()** %uninit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(function, void(*)());
// CHECK-LABEL: @test_function_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca void ()*, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store void ()* null, void ()** %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(bool, bool);
// CHECK-LABEL: @test_bool_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i8, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_bool_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i8 [[I8]], i8* %uninit, align 1
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_bool_uninit()
// ZERO: store i8 0, i8* %uninit, align 1
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(bool, bool);
// CHECK-LABEL: @test_bool_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i8, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i8 0, i8* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(empty, empty);
// CHECK-LABEL: @test_empty_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.empty, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_empty_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_empty_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i8 [[I8]], {{.*}} align 1
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_empty_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i8 0, {{.*}} align 1
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(empty, empty);
// CHECK-LABEL: @test_empty_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.empty, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(small, small);
// CHECK-LABEL: @test_small_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.small, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_small_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_small_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i8 [[I8]], {{.*}} align 1
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_small_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i8 0, {{.*}} align 1
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(small, small);
// CHECK-LABEL: @test_small_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.small, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 1, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
TEST_CUSTOM(small, small, { 42 });
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-LABEL: @test_small_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.small, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(smallinit, smallinit);
// CHECK-LABEL: @test_smallinit_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.smallinit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}smallinit{{.*}}%uninit)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
TEST_BRACES(smallinit, smallinit);
// CHECK-LABEL: @test_smallinit_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.smallinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[C:[^ ]*]] = getelementptr inbounds %struct.smallinit, %struct.smallinit* %braces, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8 42, i8* %[[C]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(smallinit, smallinit, { 100 });
// CHECK-LABEL: @test_smallinit_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.smallinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[C:[^ ]*]] = getelementptr inbounds %struct.smallinit, %struct.smallinit* %custom, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8 100, i8* %[[C]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(smallpartinit, smallpartinit);
// CHECK-LABEL: @test_smallpartinit_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.smallpartinit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}smallpartinit{{.*}}%uninit)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_smallpartinit_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_smallpartinit_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i8 [[I8]], {{.*}} align 1
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1: store i8 42, {{.*}} align 1
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_smallpartinit_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
[clang][NewPM] Add -fno-experimental-new-pass-manager to tests As per the discussion on D58375, we disable test that have optimizations under the new PM. This patch adds -fno-experimental-new-pass-manager to RUNS that: - Already run with optimizations (-O1 or higher) that were missed in D58375. - Explicitly test new PM behavior along side some new PM RUNS, but are missing this flag if new PM is enabled by default. - Specify -O without the number. Based on getOptimizationLevel(), it seems the default is 2, and the IR appears to be the same when changed to -O2, so update the test to explicitly say -O2 and provide -fno-experimental-new-pass-manager`. Differential Revision: https://reviews.llvm.org/D63156 llvm-svn: 364066
2019-06-22 00:03:06 +08:00
// ZERO-O1-LEGACY: store i16 0, i16* %uninit, align 2
Revert "Revert "As a follow-up to my initial mail to llvm-dev here's a first pass at the O1 described there."" This reapplies: 8ff85ed905a7306977d07a5cd67ab4d5a56fafb4 Original commit message: As a follow-up to my initial mail to llvm-dev here's a first pass at the O1 described there. This change doesn't include any change to move from selection dag to fast isel and that will come with other numbers that should help inform that decision. There also haven't been any real debuggability studies with this pipeline yet, this is just the initial start done so that people could see it and we could start tweaking after. Test updates: Outside of the newpm tests most of the updates are coming from either optimization passes not run anymore (and without a compelling argument at the moment) that were largely used for canonicalization in clang. Original post: http://lists.llvm.org/pipermail/llvm-dev/2019-April/131494.html Tags: #llvm Differential Revision: https://reviews.llvm.org/D65410 This reverts commit c9ddb02659e3ece7a0d9d6b4dac7ceea4ae46e6d.
2019-11-27 12:28:52 +08:00
// ZERO-O1-NEWPM: store i16 0, i16* %uninit, align 2
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(smallpartinit, smallpartinit);
// CHECK-LABEL: @test_smallpartinit_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.smallpartinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[C:[^ ]*]] = getelementptr inbounds %struct.smallpartinit, %struct.smallpartinit* %braces, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8 42, i8* %[[C]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[D:[^ ]*]] = getelementptr inbounds %struct.smallpartinit, %struct.smallpartinit* %braces, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8 0, i8* %[[D]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(smallpartinit, smallpartinit, { 100, 42 });
// CHECK-LABEL: @test_smallpartinit_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.smallpartinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[C:[^ ]*]] = getelementptr inbounds %struct.smallpartinit, %struct.smallpartinit* %custom, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8 100, i8* %[[C]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[D:[^ ]*]] = getelementptr inbounds %struct.smallpartinit, %struct.smallpartinit* %custom, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8 42, i8* %[[D]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(nullinit, nullinit);
// CHECK-LABEL: @test_nullinit_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.nullinit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}nullinit{{.*}}%uninit)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
TEST_BRACES(nullinit, nullinit);
// CHECK-LABEL: @test_nullinit_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.nullinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[N:[^ ]*]] = getelementptr inbounds %struct.nullinit, %struct.nullinit* %braces, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8* null, i8** %[[N]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(nullinit, nullinit, { (char*)"derp" });
// CHECK-LABEL: @test_nullinit_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.nullinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[N:[^ ]*]] = getelementptr inbounds %struct.nullinit, %struct.nullinit* %custom, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8* getelementptr inbounds {{.*}}, i8** %[[N]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(padded, padded);
// CHECK-LABEL: @test_padded_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.padded, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_padded_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_padded_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i64 [[I64]], i64* %uninit, align 8
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_padded_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i64 0, i64* %uninit, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(padded, padded);
// CHECK-LABEL: @test_padded_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.padded, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 8, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(padded, padded, { 42, 13371337 });
// CHECK-LABEL: @test_padded_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.padded, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(paddednullinit, paddednullinit);
// CHECK-LABEL: @test_paddednullinit_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.paddednullinit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}paddednullinit{{.*}}%uninit)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_paddednullinit_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddednullinit_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1-LEGACY: store i64 [[I64]], i64* %uninit, align 8
Revert "Revert "As a follow-up to my initial mail to llvm-dev here's a first pass at the O1 described there."" This reapplies: 8ff85ed905a7306977d07a5cd67ab4d5a56fafb4 Original commit message: As a follow-up to my initial mail to llvm-dev here's a first pass at the O1 described there. This change doesn't include any change to move from selection dag to fast isel and that will come with other numbers that should help inform that decision. There also haven't been any real debuggability studies with this pipeline yet, this is just the initial start done so that people could see it and we could start tweaking after. Test updates: Outside of the newpm tests most of the updates are coming from either optimization passes not run anymore (and without a compelling argument at the moment) that were largely used for canonicalization in clang. Original post: http://lists.llvm.org/pipermail/llvm-dev/2019-April/131494.html Tags: #llvm Differential Revision: https://reviews.llvm.org/D65410 This reverts commit c9ddb02659e3ece7a0d9d6b4dac7ceea4ae46e6d.
2019-11-27 12:28:52 +08:00
// PATTERN-O1-NEWPM: store i64 [[I64]], i64* %uninit, align 8
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_paddednullinit_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i64 0, i64* %uninit, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(paddednullinit, paddednullinit);
// CHECK-LABEL: @test_paddednullinit_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.paddednullinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[C:[^ ]*]] = getelementptr inbounds %struct.paddednullinit, %struct.paddednullinit* %braces, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8 0, i8* %[[C]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[I:[^ ]*]] = getelementptr inbounds %struct.paddednullinit, %struct.paddednullinit* %braces, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i32 0, i32* %[[I]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(paddednullinit, paddednullinit, { 42, 13371337 });
// CHECK-LABEL: @test_paddednullinit_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.paddednullinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[C:[^ ]*]] = getelementptr inbounds %struct.paddednullinit, %struct.paddednullinit* %custom, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i8 42, i8* %[[C]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[I:[^ ]*]] = getelementptr inbounds %struct.paddednullinit, %struct.paddednullinit* %custom, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i32 13371337, i32* %[[I]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
TEST_UNINIT(paddedpacked, paddedpacked);
// CHECK-LABEL: @test_paddedpacked_uninit()
// CHECK: %uninit = alloca %struct.paddedpacked, align
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
// PATTERN-LABEL: @test_paddedpacked_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddedpacked_uninit.uninit
// PATTERN-O1: %[[C:[^ ]*]] = getelementptr inbounds {{.*}}%uninit, i64 0, i32 0
[clang] Add missing FileCheck colons
2020-04-15 00:46:21 +08:00
// PATTERN-O1: store i8 [[I8]], i8* %[[C]], align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1: %[[I:[^ ]*]] = getelementptr inbounds {{.*}}%uninit, i64 0, i32 1
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i32 [[I32]], i32* %[[I]], align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// ZERO-LABEL: @test_paddedpacked_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
TEST_BRACES(paddedpacked, paddedpacked);
// CHECK-LABEL: @test_paddedpacked_braces()
// CHECK: %braces = alloca %struct.paddedpacked, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 5, i1 false)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(paddedpacked, paddedpacked, { 42, 13371337 });
// CHECK-LABEL: @test_paddedpacked_custom()
// CHECK: %custom = alloca %struct.paddedpacked, align
// CHECK-NEXT: bitcast
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-NEXT: call void @llvm.memcpy{{.*}}({{.*}}@__const.test_paddedpacked_custom.custom
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(paddedpackedarray, paddedpackedarray);
// CHECK-LABEL: @test_paddedpackedarray_uninit()
// CHECK: %uninit = alloca %struct.paddedpackedarray, align
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
// PATTERN-LABEL: @test_paddedpackedarray_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddedpackedarray_uninit.uninit
// PATTERN-O1: getelementptr
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: call void @llvm.memset{{.*}}({{.*}}i8 [[I8]], i64 10
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// ZERO-LABEL: @test_paddedpackedarray_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
TEST_BRACES(paddedpackedarray, paddedpackedarray);
// CHECK-LABEL: @test_paddedpackedarray_braces()
// CHECK: %braces = alloca %struct.paddedpackedarray, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 10, i1 false)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(paddedpackedarray, paddedpackedarray, { {{ 42, 13371337 }, { 43, 13371338 }} });
// CHECK-LABEL: @test_paddedpackedarray_custom()
// CHECK: %custom = alloca %struct.paddedpackedarray, align
// CHECK-NEXT: bitcast
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-NEXT: call void @llvm.memcpy{{.*}}({{.*}}@__const.test_paddedpackedarray_custom.custom
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
CodeGen: Preserve packed attribute in constStructWithPadding. Otherwise the object may have an incorrect size due to tail padding. Differential Revision: https://reviews.llvm.org/D59446 llvm-svn: 356328
2019-03-17 03:25:39 +08:00
TEST_UNINIT(unpackedinpacked, unpackedinpacked);
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-LABEL: @test_unpackedinpacked_uninit()
// PATTERN-O0: call void @llvm.memcpy{{.*}} 9, i1 false)
CodeGen: Preserve packed attribute in constStructWithPadding. Otherwise the object may have an incorrect size due to tail padding. Differential Revision: https://reviews.llvm.org/D59446 llvm-svn: 356328
2019-03-17 03:25:39 +08:00
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
TEST_UNINIT(paddednested, paddednested);
// CHECK-LABEL: @test_paddednested_uninit()
// CHECK: %uninit = alloca %struct.paddednested, align
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
// PATTERN-LABEL: @test_paddednested_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddednested_uninit.uninit
// PATTERN-O1: getelementptr
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: call void @llvm.memset{{.*}}({{.*}}, i8 [[I8]], i64 16
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// ZERO-LABEL: @test_paddednested_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
TEST_BRACES(paddednested, paddednested);
// CHECK-LABEL: @test_paddednested_braces()
// CHECK: %braces = alloca %struct.paddednested, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 16, i1 false)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(paddednested, paddednested, { { 42, 13371337 }, { 43, 13371338 } });
// CHECK-LABEL: @test_paddednested_custom()
// CHECK: %custom = alloca %struct.paddednested, align
// CHECK-NEXT: bitcast
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-NEXT: call void @llvm.memcpy{{.*}}({{.*}}@__const.test_paddednested_custom.custom
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(paddedpackednested, paddedpackednested);
// CHECK-LABEL: @test_paddedpackednested_uninit()
// CHECK: %uninit = alloca %struct.paddedpackednested, align
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
// PATTERN-LABEL: @test_paddedpackednested_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddedpackednested_uninit.uninit
// PATTERN-O1: getelementptr
[NFC] Updated tests after r368657 llvm-svn: 368658
2019-08-13 17:12:07 +08:00
// PATTERN-O1: call void @llvm.memset.p0i8.i64(i8* nonnull align 1 dereferenceable(10) %0, i8 [[I8]], i64 10, i1 false
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// ZERO-LABEL: @test_paddedpackednested_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
TEST_BRACES(paddedpackednested, paddedpackednested);
// CHECK-LABEL: @test_paddedpackednested_braces()
// CHECK: %braces = alloca %struct.paddedpackednested, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 10, i1 false)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(paddedpackednested, paddedpackednested, { { 42, 13371337 }, { 43, 13371338 } });
// CHECK-LABEL: @test_paddedpackednested_custom()
// CHECK: %custom = alloca %struct.paddedpackednested, align
// CHECK-NEXT: bitcast
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-NEXT: call void @llvm.memcpy{{.*}}({{.*}}@__const.test_paddedpackednested_custom.custom
CodeGen: Explicitly initialize structure padding in the -ftrivial-auto-var-init mode When generating initializers for local structures in the -ftrivial-auto-var-init mode, explicitly wipe the padding bytes with either 0x00 or 0xAA. This will allow us to automatically handle the padding when splitting the initialization stores (see https://reviews.llvm.org/D57898). Reviewed at https://reviews.llvm.org/D58188 llvm-svn: 354861
2019-02-26 18:46:21 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(bitfield, bitfield);
// CHECK-LABEL: @test_bitfield_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.bitfield, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_bitfield_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_bitfield_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i32 [[I32]], i32* %uninit, align 4
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_bitfield_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, i32* %uninit, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(bitfield, bitfield);
// CHECK-LABEL: @test_bitfield_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.bitfield, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(bitfield, bitfield, { 4, 1 });
// CHECK-LABEL: @test_bitfield_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.bitfield, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(bitfieldaligned, bitfieldaligned);
// CHECK-LABEL: @test_bitfieldaligned_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.bitfieldaligned, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_bitfieldaligned_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_bitfieldaligned_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i64 [[IPTR]], i64* %uninit, align 8
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_bitfieldaligned_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i64 0, i64* %uninit, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(bitfieldaligned, bitfieldaligned);
// CHECK-LABEL: @test_bitfieldaligned_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.bitfieldaligned, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(bitfieldaligned, bitfieldaligned, { 4, 1 });
// CHECK-LABEL: @test_bitfieldaligned_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.bitfieldaligned, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(big, big);
// CHECK-LABEL: @test_big_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.big, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_big_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: call void @llvm.memset{{.*}}, i8 [[I8]],
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_big_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(big, big);
// CHECK-LABEL: @test_big_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.big, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 104, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(big, big, { 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA, 0xAAAAAAAA });
// CHECK-LABEL: @test_big_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.big, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 -86, i64 104, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(arraytail, arraytail);
// CHECK-LABEL: @test_arraytail_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.arraytail, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_arraytail_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_arraytail_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i32 [[I32]], {{.*}} align 4
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_arraytail_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, {{.*}} align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(arraytail, arraytail);
// CHECK-LABEL: @test_arraytail_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.arraytail, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(arraytail, arraytail, { 0xdead });
// CHECK-LABEL: @test_arraytail_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.arraytail, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(int0, int[0]);
// CHECK-LABEL: @test_int0_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca [0 x i32], align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_int0_uninit()
// PATTERN: %uninit = alloca [0 x i32], align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_int0_uninit()
// ZERO: %uninit = alloca [0 x i32], align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0-NEXT: call void @{{.*}}used{{.*}}%uninit)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(int0, int[0]);
// CHECK-LABEL: @test_int0_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca [0 x i32], align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(int1, int[1]);
// CHECK-LABEL: @test_int1_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca [1 x i32], align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_int1_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_int1_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i32 [[I32]], {{.*}} align 4
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_int1_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, {{.*}} align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(int1, int[1]);
// CHECK-LABEL: @test_int1_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca [1 x i32], align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(int1, int[1], { 0x33333333 });
// CHECK-LABEL: @test_int1_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca [1 x i32], align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(int64, int[64]);
// CHECK-LABEL: @test_int64_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca [64 x i32], align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_int64_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: call void @llvm.memset{{.*}}, i8 [[I8]],
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_int64_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(int64, int[64]);
// CHECK-LABEL: @test_int64_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca [64 x i32], align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 256, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(int64, int[64], = { 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111, 0x11111111 });
// CHECK-LABEL: @test_int64_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca [64 x i32], align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 17, i64 256, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(bool4, bool[4]);
// CHECK-LABEL: @test_bool4_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca [4 x i8], align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_bool4_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_bool4_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i32 [[I32]], i32* %uninit, align 4
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_bool4_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, i32* %uninit, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(bool4, bool[4]);
// CHECK-LABEL: @test_bool4_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca [4 x i8], align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(bool4, bool[4], { true, true, true, true });
// CHECK-LABEL: @test_bool4_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca [4 x i8], align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(intptr4, int*[4]);
Variable auto-init: split out small arrays Summary: Following up with r355181, initialize small arrays as well. LLVM stage2 shows a tiny size gain. <rdar://48523005> Reviewers: glider, pcc, kcc, rjmccall Subscribers: jkorous, dexonsmith, jdoerfert, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D58885 llvm-svn: 355660
2019-03-08 09:26:49 +08:00
// CHECK-LABEL: @test_intptr4_uninit()
// CHECK: %uninit = alloca [4 x i32*], align
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
// PATTERN-O1-LABEL: @test_intptr4_uninit()
[clang][codegen][NFC] Make test patterns more permissive. See the discussion in: http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20190909/692736.html llvm-svn: 371522
2019-09-10 22:20:08 +08:00
// PATTERN-O1: call void @llvm.memset.p0i8.i64(i8* nonnull align 16 dereferenceable(32) %{{[0-9*]}}, i8 -86, i64 32, i1 false)
Variable auto-init: split out small arrays Summary: Following up with r355181, initialize small arrays as well. LLVM stage2 shows a tiny size gain. <rdar://48523005> Reviewers: glider, pcc, kcc, rjmccall Subscribers: jkorous, dexonsmith, jdoerfert, cfe-commits Tags: #clang Differential Revision: https://reviews.llvm.org/D58885 llvm-svn: 355660
2019-03-08 09:26:49 +08:00
// ZERO-LABEL: @test_intptr4_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(intptr4, int*[4]);
// CHECK-LABEL: @test_intptr4_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca [4 x i32*], align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 32, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(intptr4, int*[4], = { (int*)0x22222222, (int*)0x22222222, (int*)0x22222222, (int*)0x22222222 });
// CHECK-LABEL: @test_intptr4_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca [4 x i32*], align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(tailpad4, tailpad[4]);
// CHECK-LABEL: @test_tailpad4_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca [4 x %struct.tailpad], align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_tailpad4_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_tailpad4_uninit.uninit
// PATTERN-O1: bitcast
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: call void @llvm.memset{{.*}}({{.*}}0, i8 [[I8]], i64 16
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_tailpad4_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(tailpad4, tailpad[4]);
// CHECK-LABEL: @test_tailpad4_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca [4 x %struct.tailpad], align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 16, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
CDDecl More automatic variable tail padding test Test tail padded automatic variable at different width, because they encounter different codegen. llvm-svn: 339273
2018-08-09 01:05:17 +08:00
TEST_CUSTOM(tailpad4, tailpad[4], { {257, 1}, {257, 1}, {257, 1}, {257, 1} });
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-LABEL: @test_tailpad4_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca [4 x %struct.tailpad], align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
CDDecl More automatic variable tail padding test Test tail padded automatic variable at different width, because they encounter different codegen. llvm-svn: 339273
2018-08-09 01:05:17 +08:00
TEST_UNINIT(tailpad9, tailpad[9]);
// CHECK-LABEL: @test_tailpad9_uninit()
// CHECK: %uninit = alloca [9 x %struct.tailpad], align
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_tailpad9_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O0: call void @llvm.memset{{.*}}, i8 [[I8]],
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_tailpad9_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
CDDecl More automatic variable tail padding test Test tail padded automatic variable at different width, because they encounter different codegen. llvm-svn: 339273
2018-08-09 01:05:17 +08:00
TEST_BRACES(tailpad9, tailpad[9]);
// CHECK-LABEL: @test_tailpad9_braces()
// CHECK: %braces = alloca [9 x %struct.tailpad], align [[ALIGN:[0-9]*]]
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 36, i1 false)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(tailpad9, tailpad[9], { {257, 1}, {257, 1}, {257, 1}, {257, 1}, {257, 1}, {257, 1}, {257, 1}, {257, 1}, {257, 1} });
// CHECK-LABEL: @test_tailpad9_custom()
// CHECK: %custom = alloca [9 x %struct.tailpad], align [[ALIGN:[0-9]*]]
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 1, i64 36, i1 false)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(atomicbool, _Atomic(bool));
// CHECK-LABEL: @test_atomicbool_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i8, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_atomicbool_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i8 [[I8]], i8* %uninit, align 1
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_atomicbool_uninit()
// ZERO: store i8 0, i8* %uninit, align 1
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(atomicint, _Atomic(int));
// CHECK-LABEL: @test_atomicint_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_atomicint_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i32 [[I32]], i32* %uninit, align 4
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_atomicint_uninit()
// ZERO: store i32 0, i32* %uninit, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(atomicdouble, _Atomic(double));
// CHECK-LABEL: @test_atomicdouble_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca double, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_atomicdouble_uninit()
// PATTERN: store double 0xFFFFFFFFFFFFFFFF, double* %uninit, align 8
// ZERO-LABEL: @test_atomicdouble_uninit()
// ZERO: store double 0.000000e+00, double* %uninit, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(atomicnotlockfree, _Atomic(notlockfree));
// CHECK-LABEL: @test_atomicnotlockfree_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.notlockfree, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_atomicnotlockfree_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_atomicnotlockfree_uninit.uninit
// PATTERN-O1: bitcast
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: call void @llvm.memset{{.*}}({{.*}}, i8 [[I8]], i64 32
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_atomicnotlockfree_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(atomicpadded, _Atomic(padded));
// CHECK-LABEL: @test_atomicpadded_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.padded, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_atomicpadded_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_atomicpadded_uninit.uninit
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN-O1: store i64 [[IPTR]], i64* %uninit, align 8
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_atomicpadded_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i64 0, i64* %uninit, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(atomictailpad, _Atomic(tailpad));
// CHECK-LABEL: @test_atomictailpad_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.tailpad, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_atomictailpad_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_atomictailpad_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_atomictailpad_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, i32* %uninit, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(complexfloat, _Complex float);
// CHECK-LABEL: @test_complexfloat_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca { float, float }, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_complexfloat_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_complexfloat_uninit.uninit
// PATTERN-O1: %[[F1:[^ ]*]] = getelementptr inbounds {{.*}}%uninit, i64 0, i32 0
[clang] Add missing FileCheck colons
2020-04-15 00:46:21 +08:00
// PATTERN-O1: store float 0xFFFFFFFFE0000000, float* %[[F1]], align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O1: %[[F2:[^ ]*]] = getelementptr inbounds {{.*}}%uninit, i64 0, i32 1
// PATTERN-O1: store float 0xFFFFFFFFE0000000, float* %[[F2]], align
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_complexfloat_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i64 0, i64* %uninit, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(complexfloat, _Complex float);
// CHECK-LABEL: @test_complexfloat_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca { float, float }, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[R:[^ ]*]] = getelementptr inbounds { float, float }, { float, float }* %braces, i32 0, i32 0
// CHECK-NEXT: %[[I:[^ ]*]] = getelementptr inbounds { float, float }, { float, float }* %braces, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store float 0.000000e+00, float* %[[R]], align [[ALIGN]]
// CHECK-NEXT: store float 0.000000e+00, float* %[[I]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(complexfloat, _Complex float, { 3.1415926535897932384626433, 3.1415926535897932384626433 });
// CHECK-LABEL: @test_complexfloat_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca { float, float }, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[R:[^ ]*]] = getelementptr inbounds { float, float }, { float, float }* %custom, i32 0, i32 0
// CHECK-NEXT: %[[I:[^ ]*]] = getelementptr inbounds { float, float }, { float, float }* %custom, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store float 0x400921FB60000000, float* %[[R]], align [[ALIGN]]
// CHECK-NEXT: store float 0x400921FB60000000, float* %[[I]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(complexdouble, _Complex double);
// CHECK-LABEL: @test_complexdouble_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca { double, double }, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_complexdouble_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_complexdouble_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_complexdouble_uninit()
// ZERO: call void @llvm.memset{{.*}}, i8 0,
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(complexdouble, _Complex double);
// CHECK-LABEL: @test_complexdouble_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca { double, double }, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[R:[^ ]*]] = getelementptr inbounds { double, double }, { double, double }* %braces, i32 0, i32 0
// CHECK-NEXT: %[[I:[^ ]*]] = getelementptr inbounds { double, double }, { double, double }* %braces, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store double 0.000000e+00, double* %[[R]], align [[ALIGN]]
// CHECK-NEXT: store double 0.000000e+00, double* %[[I]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(complexdouble, _Complex double, { 3.1415926535897932384626433, 3.1415926535897932384626433 });
// CHECK-LABEL: @test_complexdouble_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca { double, double }, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[R:[^ ]*]] = getelementptr inbounds { double, double }, { double, double }* %custom, i32 0, i32 0
// CHECK-NEXT: %[[I:[^ ]*]] = getelementptr inbounds { double, double }, { double, double }* %custom, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store double 0x400921FB54442D18, double* %[[R]], align [[ALIGN]]
// CHECK-NEXT: store double 0x400921FB54442D18, double* %[[I]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(volatileint, volatile int);
// CHECK-LABEL: @test_volatileint_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_volatileint_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store volatile i32 [[I32]], i32* %uninit, align 4
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_volatileint_uninit()
// ZERO: store volatile i32 0, i32* %uninit, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(volatileint, volatile int);
// CHECK-LABEL: @test_volatileint_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i32, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store volatile i32 0, i32* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(semivolatile, semivolatile);
// CHECK-LABEL: @test_semivolatile_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.semivolatile, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_semivolatile_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_semivolatile_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_semivolatile_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i64 0, i64* %uninit, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(semivolatile, semivolatile);
// CHECK-LABEL: @test_semivolatile_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.semivolatile, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 8, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(semivolatile, semivolatile, { 0x44444444, 0x44444444 });
// CHECK-LABEL: @test_semivolatile_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.semivolatile, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-O0: bitcast
// CHECK-O0: call void @llvm.memcpy
// CHECK-O0: call void @{{.*}}used{{.*}}%custom)
// CHECK-O1: store i64 4919131752989213764, i64* %custom, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(semivolatileinit, semivolatileinit);
// CHECK-LABEL: @test_semivolatileinit_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.semivolatileinit, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}semivolatileinit{{.*}}%uninit)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
TEST_BRACES(semivolatileinit, semivolatileinit);
// CHECK-LABEL: @test_semivolatileinit_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.semivolatileinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[I:[^ ]*]] = getelementptr inbounds %struct.semivolatileinit, %struct.semivolatileinit* %braces, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i32 286331153, i32* %[[I]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[VI:[^ ]*]] = getelementptr inbounds %struct.semivolatileinit, %struct.semivolatileinit* %braces, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store volatile i32 286331153, i32* %[[VI]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(semivolatileinit, semivolatileinit, { 0x44444444, 0x44444444 });
// CHECK-LABEL: @test_semivolatileinit_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %struct.semivolatileinit, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[I:[^ ]*]] = getelementptr inbounds %struct.semivolatileinit, %struct.semivolatileinit* %custom, i32 0, i32 0
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store i32 1145324612, i32* %[[I]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: %[[VI:[^ ]*]] = getelementptr inbounds %struct.semivolatileinit, %struct.semivolatileinit* %custom, i32 0, i32 1
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: store volatile i32 1145324612, i32* %[[VI]], align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(base, base);
// CHECK-LABEL: @test_base_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.base, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}base{{.*}}%uninit)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_base_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_base_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_base_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
[clang][NewPM] Add -fno-experimental-new-pass-manager to tests As per the discussion on D58375, we disable test that have optimizations under the new PM. This patch adds -fno-experimental-new-pass-manager to RUNS that: - Already run with optimizations (-O1 or higher) that were missed in D58375. - Explicitly test new PM behavior along side some new PM RUNS, but are missing this flag if new PM is enabled by default. - Specify -O without the number. Based on getOptimizationLevel(), it seems the default is 2, and the IR appears to be the same when changed to -O2, so update the test to explicitly say -O2 and provide -fno-experimental-new-pass-manager`. Differential Revision: https://reviews.llvm.org/D63156 llvm-svn: 364066
2019-06-22 00:03:06 +08:00
// ZERO-O1-LEGACY: store i64 0, {{.*}} align 8
// ZERO-O1-NEWPM: store i32 (...)** bitcast (i8** getelementptr inbounds ({ [4 x i8*] }, { [4 x i8*] }* @_ZTV4base, i64 0, inrange i32 0, i64 2) to i32 (...)**), {{.*}}, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(base, base);
// CHECK-LABEL: @test_base_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.base, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 8, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}base{{.*}}%braces)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(derived, derived);
// CHECK-LABEL: @test_derived_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.derived, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}derived{{.*}}%uninit)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_derived_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_derived_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_derived_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
[clang][NewPM] Add -fno-experimental-new-pass-manager to tests As per the discussion on D58375, we disable test that have optimizations under the new PM. This patch adds -fno-experimental-new-pass-manager to RUNS that: - Already run with optimizations (-O1 or higher) that were missed in D58375. - Explicitly test new PM behavior along side some new PM RUNS, but are missing this flag if new PM is enabled by default. - Specify -O without the number. Based on getOptimizationLevel(), it seems the default is 2, and the IR appears to be the same when changed to -O2, so update the test to explicitly say -O2 and provide -fno-experimental-new-pass-manager`. Differential Revision: https://reviews.llvm.org/D63156 llvm-svn: 364066
2019-06-22 00:03:06 +08:00
// ZERO-O1-LEGACY: store i64 0, {{.*}} align 8
// ZERO-O1-NEWPM: store i32 (...)** bitcast (i8** getelementptr inbounds ({ [4 x i8*] }, { [4 x i8*] }* @_ZTV7derived, i64 0, inrange i32 0, i64 2) to i32 (...)**), {{.*}} align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(derived, derived);
// CHECK-LABEL: @test_derived_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.derived, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 8, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}derived{{.*}}%braces)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(virtualderived, virtualderived);
// CHECK-LABEL: @test_virtualderived_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %struct.virtualderived, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}virtualderived{{.*}}%uninit)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_virtualderived_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_virtualderived_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_virtualderived_uninit()
[clang][NewPM] Add -fno-experimental-new-pass-manager to tests As per the discussion on D58375, we disable test that have optimizations under the new PM. This patch adds -fno-experimental-new-pass-manager to RUNS that: - Already run with optimizations (-O1 or higher) that were missed in D58375. - Explicitly test new PM behavior along side some new PM RUNS, but are missing this flag if new PM is enabled by default. - Specify -O without the number. Based on getOptimizationLevel(), it seems the default is 2, and the IR appears to be the same when changed to -O2, so update the test to explicitly say -O2 and provide -fno-experimental-new-pass-manager`. Differential Revision: https://reviews.llvm.org/D63156 llvm-svn: 364066
2019-06-22 00:03:06 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1-LEGACY: call void @llvm.memset{{.*}}, i8 0,
Revert "Revert "As a follow-up to my initial mail to llvm-dev here's a first pass at the O1 described there."" This reapplies: 8ff85ed905a7306977d07a5cd67ab4d5a56fafb4 Original commit message: As a follow-up to my initial mail to llvm-dev here's a first pass at the O1 described there. This change doesn't include any change to move from selection dag to fast isel and that will come with other numbers that should help inform that decision. There also haven't been any real debuggability studies with this pipeline yet, this is just the initial start done so that people could see it and we could start tweaking after. Test updates: Outside of the newpm tests most of the updates are coming from either optimization passes not run anymore (and without a compelling argument at the moment) that were largely used for canonicalization in clang. Original post: http://lists.llvm.org/pipermail/llvm-dev/2019-April/131494.html Tags: #llvm Differential Revision: https://reviews.llvm.org/D65410 This reverts commit c9ddb02659e3ece7a0d9d6b4dac7ceea4ae46e6d.
2019-11-27 12:28:52 +08:00
// ZERO-O1-NEWPM: call void @llvm.memset{{.*}}, i8 0,
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(virtualderived, virtualderived);
// CHECK-LABEL: @test_virtualderived_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %struct.virtualderived, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 16, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}virtualderived{{.*}}%braces)
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_UNINIT(matching, matching);
// CHECK-LABEL: @test_matching_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %union.matching, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_matching_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_matching_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_matching_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, {{.*}} align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(matching, matching);
// CHECK-LABEL: @test_matching_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %union.matching, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(matching, matching, { .f = 0xf00f });
// CHECK-LABEL: @test_matching_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %union.matching, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-O0: bitcast
// CHECK-O0: call void @llvm.memcpy
// CHECK-O0: call void @{{.*}}used{{.*}}%custom)
// CHECK-O1: getelementptr
// CHECK-O1: store i32 1198526208, i32* {{.*}}, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(matchingreverse, matchingreverse);
// CHECK-LABEL: @test_matchingreverse_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %union.matchingreverse, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_matchingreverse_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_matchingreverse_uninit.uninit
// PATTERN-O1: store float 0xFFFFFFFFE0000000
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_matchingreverse_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, {{.*}} align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(matchingreverse, matchingreverse);
// CHECK-LABEL: @test_matchingreverse_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %union.matchingreverse, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(matchingreverse, matchingreverse, { .i = 0xf00f });
// CHECK-LABEL: @test_matchingreverse_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %union.matchingreverse, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-O0: bitcast
// CHECK-O0: call void @llvm.memcpy
// CHECK-O0: call void @{{.*}}used{{.*}}%custom)
// CHECK-O1: store i32 61455, i32* %1, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(unmatched, unmatched);
// CHECK-LABEL: @test_unmatched_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %union.unmatched, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_unmatched_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_unmatched_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_unmatched_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, {{.*}} align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(unmatched, unmatched);
// CHECK-LABEL: @test_unmatched_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %union.unmatched, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(unmatched, unmatched, { .i = 0x3badbeef });
// CHECK-LABEL: @test_unmatched_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %union.unmatched, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-O0: bitcast
// CHECK-O0: call void @llvm.memcpy
// CHECK-O0: call void @{{.*}}used{{.*}}%custom)
// CHECK-O1: store i32 1001242351, i32* {{.*}}, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(unmatchedreverse, unmatchedreverse);
// CHECK-LABEL: @test_unmatchedreverse_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %union.unmatchedreverse, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_unmatchedreverse_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_unmatchedreverse_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_unmatchedreverse_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i32 0, {{.*}} align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(unmatchedreverse, unmatchedreverse);
// CHECK-LABEL: @test_unmatchedreverse_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %union.unmatchedreverse, align [[ALIGN:[0-9]*]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK-NEXT: call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(unmatchedreverse, unmatchedreverse, { .c = 42 });
// CHECK-LABEL: @test_unmatchedreverse_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %union.unmatchedreverse, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-O0: bitcast
// CHECK-O0: call void @llvm.memcpy
// CHECK-O0: call void @{{.*}}used{{.*}}%custom)
// PATTERN-O1: store i32 -1431655894, i32* {{.*}}, align 4
// ZERO-O1: store i32 42, i32* {{.*}}, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(unmatchedfp, unmatchedfp);
// CHECK-LABEL: @test_unmatchedfp_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca %union.unmatchedfp, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_unmatchedfp_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_unmatchedfp_uninit.uninit
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_unmatchedfp_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
// ZERO-O1: store i64 0, {{.*}} align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(unmatchedfp, unmatchedfp);
// CHECK-LABEL: @test_unmatchedfp_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca %union.unmatchedfp, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: bitcast
// CHECK-NEXT: call void @llvm.memcpy
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(unmatchedfp, unmatchedfp, { .d = 3.1415926535897932384626433 });
// CHECK-LABEL: @test_unmatchedfp_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca %union.unmatchedfp, align
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// CHECK-O0: bitcast
// CHECK-O0: call void @llvm.memcpy
// CHECK-O0: call void @{{.*}}used{{.*}}%custom)
// CHECK-O1: store i64 4614256656552045848, i64* %1, align 8
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_UNINIT(emptyenum, emptyenum);
// CHECK-LABEL: @test_emptyenum_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_emptyenum_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i32 [[I32]], i32* %uninit, align 4
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_emptyenum_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO: store i32 0, i32* %uninit, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(emptyenum, emptyenum);
// CHECK-LABEL: @test_emptyenum_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i32, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32 0, i32* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(emptyenum, emptyenum, { (emptyenum)42 });
// CHECK-LABEL: @test_emptyenum_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca i32, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32 42, i32* %custom, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(smallenum, smallenum);
// CHECK-LABEL: @test_smallenum_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca i32, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_smallenum_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store i32 [[I32]], i32* %uninit, align 4
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_smallenum_uninit()
CodeGen: Fix PR40605 by splitting constant struct initializers When emitting initializers for local structures for code built with -ftrivial-auto-var-init, replace constant structures with sequences of stores. This appears to greatly help removing dead initialization stores to those locals that are later overwritten by other data. This also removes a lot of .rodata constants (see PR40605), replacing most of them with immediate values (for Linux kernel the .rodata size is reduced by ~1.9%) llvm-svn: 355181
2019-03-01 17:00:41 +08:00
// ZERO: store i32 0, i32* %uninit, align 4
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(smallenum, smallenum);
// CHECK-LABEL: @test_smallenum_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca i32, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32 0, i32* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(smallenum, smallenum, { (smallenum)42 });
// CHECK-LABEL: @test_smallenum_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca i32, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store i32 42, i32* %custom, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(intvec16, int __attribute__((vector_size(16))));
// CHECK-LABEL: @test_intvec16_uninit()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %uninit = alloca <4 x i32>, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_intvec16_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store <4 x i32> <i32 [[I32]], i32 [[I32]], i32 [[I32]], i32 [[I32]]>, <4 x i32>* %uninit, align 16
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_intvec16_uninit()
// ZERO: store <4 x i32> zeroinitializer, <4 x i32>* %uninit, align 16
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(intvec16, int __attribute__((vector_size(16))));
// CHECK-LABEL: @test_intvec16_braces()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %braces = alloca <4 x i32>, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store <4 x i32> zeroinitializer, <4 x i32>* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
TEST_CUSTOM(intvec16, int __attribute__((vector_size(16))), { 0x44444444, 0x44444444, 0x44444444, 0x44444444 });
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-LABEL: @test_intvec16_custom()
[NFC] Improve auto-var-init alignment check We're not actually testing for alignment, we just want to know that whatever incoming alignment got propagated. Do that by capturing the alignment and checking that it's actually what's passed later, instead of hard-coding an alignment value. llvm-svn: 339196
2018-08-08 06:43:44 +08:00
// CHECK: %custom = alloca <4 x i32>, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store <4 x i32> <i32 1145324612, i32 1145324612, i32 1145324612, i32 1145324612>, <4 x i32>* %custom, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(longlongvec32, long long __attribute__((vector_size(32))));
// CHECK-LABEL: @test_longlongvec32_uninit()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %uninit = alloca <4 x i64>, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_longlongvec32_uninit()
Codegen, NFC: 32bit test in auto-var-init.cpp llvm-svn: 365857
2019-07-12 09:36:11 +08:00
// PATTERN: store <4 x i64> <i64 [[I64]], i64 [[I64]], i64 [[I64]], i64 [[I64]]>, <4 x i64>* %uninit, align 32
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// ZERO-LABEL: @test_longlongvec32_uninit()
// ZERO: store <4 x i64> zeroinitializer, <4 x i64>* %uninit, align 32
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(longlongvec32, long long __attribute__((vector_size(32))));
// CHECK-LABEL: @test_longlongvec32_braces()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %braces = alloca <4 x i64>, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store <4 x i64> zeroinitializer, <4 x i64>* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(longlongvec32, long long __attribute__((vector_size(32))), { 0x3333333333333333, 0x3333333333333333, 0x3333333333333333, 0x3333333333333333 });
// CHECK-LABEL: @test_longlongvec32_custom()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %custom = alloca <4 x i64>, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store <4 x i64> <i64 3689348814741910323, i64 3689348814741910323, i64 3689348814741910323, i64 3689348814741910323>, <4 x i64>* %custom, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(floatvec16, float __attribute__((vector_size(16))));
// CHECK-LABEL: @test_floatvec16_uninit()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %uninit = alloca <4 x float>, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_floatvec16_uninit()
// PATTERN: store <4 x float> <float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000>, <4 x float>* %uninit, align 16
// ZERO-LABEL: @test_floatvec16_uninit()
// ZERO: store <4 x float> zeroinitializer, <4 x float>* %uninit, align 16
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(floatvec16, float __attribute__((vector_size(16))));
// CHECK-LABEL: @test_floatvec16_braces()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %braces = alloca <4 x float>, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store <4 x float> zeroinitializer, <4 x float>* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(floatvec16, float __attribute__((vector_size(16))), { 3.1415926535897932384626433, 3.1415926535897932384626433, 3.1415926535897932384626433, 3.1415926535897932384626433 });
// CHECK-LABEL: @test_floatvec16_custom()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %custom = alloca <4 x float>, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store <4 x float> <float 0x400921FB60000000, float 0x400921FB60000000, float 0x400921FB60000000, float 0x400921FB60000000>, <4 x float>* %custom, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
TEST_UNINIT(doublevec32, double __attribute__((vector_size(32))));
// CHECK-LABEL: @test_doublevec32_uninit()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %uninit = alloca <4 x double>, align
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
Automatic variable initialization Summary: Add an option to initialize automatic variables with either a pattern or with zeroes. The default is still that automatic variables are uninitialized. Also add attributes to request uninitialized on a per-variable basis, mainly to disable initialization of large stack arrays when deemed too expensive. This isn't meant to change the semantics of C and C++. Rather, it's meant to be a last-resort when programmers inadvertently have some undefined behavior in their code. This patch aims to make undefined behavior hurt less, which security-minded people will be very happy about. Notably, this means that there's no inadvertent information leak when: - The compiler re-uses stack slots, and a value is used uninitialized. - The compiler re-uses a register, and a value is used uninitialized. - Stack structs / arrays / unions with padding are copied. This patch only addresses stack and register information leaks. There's many more infoleaks that we could address, and much more undefined behavior that could be tamed. Let's keep this patch focused, and I'm happy to address related issues elsewhere. To keep the patch simple, only some `undef` is removed for now, see `replaceUndef`. The padding-related infoleaks are therefore not all gone yet. This will be addressed in a follow-up, mainly because addressing padding-related leaks should be a stand-alone option which is implied by variable initialization. There are three options when it comes to automatic variable initialization: 0. Uninitialized This is C and C++'s default. It's not changing. Depending on code generation, a programmer who runs into undefined behavior by using an uninialized automatic variable may observe any previous value (including program secrets), or any value which the compiler saw fit to materialize on the stack or in a register (this could be to synthesize an immediate, to refer to code or data locations, to generate cookies, etc). 1. Pattern initialization This is the recommended initialization approach. Pattern initialization's goal is to initialize automatic variables with values which will likely transform logic bugs into crashes down the line, are easily recognizable in a crash dump, without being values which programmers can rely on for useful program semantics. At the same time, pattern initialization tries to generate code which will optimize well. You'll find the following details in `patternFor`: - Integers are initialized with repeated 0xAA bytes (infinite scream). - Vectors of integers are also initialized with infinite scream. - Pointers are initialized with infinite scream on 64-bit platforms because it's an unmappable pointer value on architectures I'm aware of. Pointers are initialize to 0x000000AA (small scream) on 32-bit platforms because 32-bit platforms don't consistently offer unmappable pages. When they do it's usually the zero page. As people try this out, I expect that we'll want to allow different platforms to customize this, let's do so later. - Vectors of pointers are initialized the same way pointers are. - Floating point values and vectors are initialized with a negative quiet NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff). NaNs are nice (here, anways) because they propagate on arithmetic, making it more likely that entire computations become NaN when a single uninitialized value sneaks in. - Arrays are initialized to their homogeneous elements' initialization value, repeated. Stack-based Variable-Length Arrays (VLAs) are runtime-initialized to the allocated size (no effort is made for negative size, but zero-sized VLAs are untouched even if technically undefined). - Structs are initialized to their heterogeneous element's initialization values. Zero-size structs are initialized as 0xAA since they're allocated a single byte. - Unions are initialized using the initialization for the largest member of the union. Expect the values used for pattern initialization to change over time, as we refine heuristics (both for performance and security). The goal is truly to avoid injecting semantics into undefined behavior, and we should be comfortable changing these values when there's a worthwhile point in doing so. Why so much infinite scream? Repeated byte patterns tend to be easy to synthesize on most architectures, and otherwise memset is usually very efficient. For values which aren't entirely repeated byte patterns, LLVM will often generate code which does memset + a few stores. 2. Zero initialization Zero initialize all values. This has the unfortunate side-effect of providing semantics to otherwise undefined behavior, programs therefore might start to rely on this behavior, and that's sad. However, some programmers believe that pattern initialization is too expensive for them, and data might show that they're right. The only way to make these programmers wrong is to offer zero-initialization as an option, figure out where they are right, and optimize the compiler into submission. Until the compiler provides acceptable performance for all security-minded code, zero initialization is a useful (if blunt) tool. I've been asked for a fourth initialization option: user-provided byte value. This might be useful, and can easily be added later. Why is an out-of band initialization mecanism desired? We could instead use -Wuninitialized! Indeed we could, but then we're forcing the programmer to provide semantics for something which doesn't actually have any (it's uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`, or whether it's just there to shut that warning up. It's also way easier to use a compiler flag than it is to manually and intelligently initialize all values in a program. Why not just rely on static analysis? Because it cannot reason about all dynamic code paths effectively, and it has false positives. It's a great tool, could get even better, but it's simply incapable of catching all uses of uninitialized values. Why not just rely on memory sanitizer? Because it's not universally available, has a 3x performance cost, and shouldn't be deployed in production. Again, it's a great tool, it'll find the dynamic uses of uninitialized variables that your test coverage hits, but it won't find the ones that you encounter in production. What's the performance like? Not too bad! Previous publications [0] have cited 2.7 to 4.5% averages. We've commmitted a few patches over the last few months to address specific regressions, both in code size and performance. In all cases, the optimizations are generally useful, but variable initialization benefits from them a lot more than regular code does. We've got a handful of other optimizations in mind, but the code is in good enough shape and has found enough latent issues that it's a good time to get the change reviewed, checked in, and have others kick the tires. We'll continue reducing overheads as we try this out on diverse codebases. Is it a good idea? Security-minded folks think so, and apparently so does the Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this feature would have killed 49 MSRC cases that involved uninitialized struct data leaking across a trust boundary. It would have also mitigated a number of bugs involving uninitialized struct data being used directly.". They seem to use pure zero initialization, and claim to have taken the overheads down to within noise. Don't just trust Microsoft though, here's another relevant person asking for this [2]. It's been proposed for GCC [3] and LLVM [4] before. What are the caveats? A few! - Variables declared in unreachable code, and used later, aren't initialized. This goto, Duff's device, other objectionable uses of switch. This should instead be a hard-error in any serious codebase. - Volatile stack variables are still weird. That's pre-existing, it's really the language's fault and this patch keeps it weird. We should deprecate volatile [5]. - As noted above, padding isn't fully handled yet. I don't think these caveats make the patch untenable because they can be addressed separately. Should this be on by default? Maybe, in some circumstances. It's a conversation we can have when we've tried it out sufficiently, and we're confident that we've eliminated enough of the overheads that most codebases would want to opt-in. Let's keep our precious undefined behavior until that point in time. How do I use it: 1. On the command-line: -ftrivial-auto-var-init=uninitialized (the default) -ftrivial-auto-var-init=pattern -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang 2. Using an attribute: int dont_initialize_me __attribute((uninitialized)); [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf [1]: https://twitter.com/JosephBialek/status/1062774315098112001 [2]: https://outflux.net/slides/2018/lss/danger.pdf [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c [5]: http://wg21.link/p1152 I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html <rdar://problem/39131435> Reviewers: pcc, kcc, rsmith Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D54604 llvm-svn: 349442
2018-12-18 13:12:21 +08:00
// PATTERN-LABEL: @test_doublevec32_uninit()
// PATTERN: store <4 x double> <double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF>, <4 x double>* %uninit, align 32
// ZERO-LABEL: @test_doublevec32_uninit()
// ZERO: store <4 x double> zeroinitializer, <4 x double>* %uninit, align 32
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
TEST_BRACES(doublevec32, double __attribute__((vector_size(32))));
// CHECK-LABEL: @test_doublevec32_braces()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %braces = alloca <4 x double>, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store <4 x double> zeroinitializer, <4 x double>* %braces, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%braces)
TEST_CUSTOM(doublevec32, double __attribute__((vector_size(32))), { 3.1415926535897932384626433, 3.1415926535897932384626433, 3.1415926535897932384626433, 3.1415926535897932384626433 });
// CHECK-LABEL: @test_doublevec32_custom()
Auto var init test fix #2 It turns out that the AVX bots have different alignment for their vectors, and my test mistakenly assumed a particular vector alignent on the stack. Instead, capture the alignment and test for it in subsequent operations. llvm-svn: 339093
2018-08-07 12:44:13 +08:00
// CHECK: %custom = alloca <4 x double>, align [[ALIGN:[0-9]*]]
// CHECK-NEXT: store <4 x double> <double 0x400921FB54442D18, double 0x400921FB54442D18, double 0x400921FB54442D18, double 0x400921FB54442D18>, <4 x double>* %custom, align [[ALIGN]]
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
// CHECK-NEXT: call void @{{.*}}used{{.*}}%custom)
[clang codegen] Clean up handling of vectors with trivial-auto-var-init. The code was pretending to be doing something useful with vectors, but really it was doing nothing: the element type of a vector is always a scalar type, so constWithPadding would always just return the input constant. Split off from D75661 so it can be reviewed separately. While I'm here, also add testcase to show missing vector handling. Differential Revision: https://reviews.llvm.org/D76528
2020-03-21 05:18:04 +08:00
// TODO: This vector has tail padding
TEST_UNINIT(doublevec24, double __attribute__((vector_size(24))));
// CHECK-LABEL: @test_doublevec24_uninit()
// CHECK: %uninit = alloca <3 x double>, align
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
// PATTERN-LABEL: @test_doublevec24_uninit()
// PATTERN: store <3 x double> <double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF>, <3 x double>* %uninit, align 32
// ZERO-LABEL: @test_doublevec24_uninit()
// ZERO: store <3 x double> zeroinitializer, <3 x double>* %uninit, align 32
// TODO: This vector has tail padding
TEST_UNINIT(longdoublevec32, long double __attribute__((vector_size(sizeof(long double)*2))));
// CHECK-LABEL: @test_longdoublevec32_uninit()
// CHECK: %uninit = alloca <2 x x86_fp80>, align
// CHECK-NEXT: call void @{{.*}}used{{.*}}%uninit)
// PATTERN-LABEL: @test_longdoublevec32_uninit()
// PATTERN: store <2 x x86_fp80> <x86_fp80 0xKFFFFFFFFFFFFFFFFFFFF, x86_fp80 0xKFFFFFFFFFFFFFFFFFFFF>, <2 x x86_fp80>* %uninit, align 32
// ZERO-LABEL: @test_longdoublevec32_uninit()
// ZERO: store <2 x x86_fp80> zeroinitializer, <2 x x86_fp80>* %uninit, align 32
[NFC] Test automatic variable initialization Summary: r337887 started using memset for automatic variable initialization where sensible. A follow-up discussion leads me to believe that we should better test automatic variable initialization, and that there are probably follow-up patches in clang and LLVM to improve codegen. It’ll be important to measure -O0 compile time, and figure out which transforms should be in the frontend versus the backend. This patch is just a test of the current behavior, no questions asked. Follow-up patches will tune the code generation. <rdar://problem/42981573> Subscribers: dexonsmith, cfe-commits Differential Revision: https://reviews.llvm.org/D50361 llvm-svn: 339089
2018-08-07 11:12:52 +08:00
} // extern "C"