There seems to be an bug related to table relocations not being
written correctly in this case. This change is intended simply
to increase the coverage, not fix the issue.
llvm-svn: 323282
The reinterpret cast to uint32_t to read the little-endian instructions
will only work on a little endian system. Use ulittle32_t to always read
little-endian (AArch64 instructions are always little endian).
Fixes PR36056
Differential Revision: https://reviews.llvm.org/D42421
llvm-svn: 323243
TABLE relocations now store the function that is being refered
to indirectly.
See rL323165.
Also extend the call-indirect.ll a little.
Based on a patch by Nicholas Wilson!
llvm-svn: 323168
Summary:
This detects when an import library is about to be overwritten with a
newly built one with the same contents, and keeps the old library
instead. The use case for this is to avoid needlessly rebuilding
targets that depend on the import library in build systems that rely
on timestamps to determine whether a target requires rebuilding.
This feature was requested in PR35917.
Reviewers: rnk, ruiu, zturner, pcc
Reviewed By: ruiu
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D42326
llvm-svn: 323164
Summary:
First, we need to explain the core of the vulnerability. Note that this
is a very incomplete description, please see the Project Zero blog post
for details:
https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html
The basis for branch target injection is to direct speculative execution
of the processor to some "gadget" of executable code by poisoning the
prediction of indirect branches with the address of that gadget. The
gadget in turn contains an operation that provides a side channel for
reading data. Most commonly, this will look like a load of secret data
followed by a branch on the loaded value and then a load of some
predictable cache line. The attacker then uses timing of the processors
cache to determine which direction the branch took *in the speculative
execution*, and in turn what one bit of the loaded value was. Due to the
nature of these timing side channels and the branch predictor on Intel
processors, this allows an attacker to leak data only accessible to
a privileged domain (like the kernel) back into an unprivileged domain.
The goal is simple: avoid generating code which contains an indirect
branch that could have its prediction poisoned by an attacker. In many
cases, the compiler can simply use directed conditional branches and
a small search tree. LLVM already has support for lowering switches in
this way and the first step of this patch is to disable jump-table
lowering of switches and introduce a pass to rewrite explicit indirectbr
sequences into a switch over integers.
However, there is no fully general alternative to indirect calls. We
introduce a new construct we call a "retpoline" to implement indirect
calls in a non-speculatable way. It can be thought of loosely as
a trampoline for indirect calls which uses the RET instruction on x86.
Further, we arrange for a specific call->ret sequence which ensures the
processor predicts the return to go to a controlled, known location. The
retpoline then "smashes" the return address pushed onto the stack by the
call with the desired target of the original indirect call. The result
is a predicted return to the next instruction after a call (which can be
used to trap speculative execution within an infinite loop) and an
actual indirect branch to an arbitrary address.
On 64-bit x86 ABIs, this is especially easily done in the compiler by
using a guaranteed scratch register to pass the target into this device.
For 32-bit ABIs there isn't a guaranteed scratch register and so several
different retpoline variants are introduced to use a scratch register if
one is available in the calling convention and to otherwise use direct
stack push/pop sequences to pass the target address.
This "retpoline" mitigation is fully described in the following blog
post: https://support.google.com/faqs/answer/7625886
We also support a target feature that disables emission of the retpoline
thunk by the compiler to allow for custom thunks if users want them.
These are particularly useful in environments like kernels that
routinely do hot-patching on boot and want to hot-patch their thunk to
different code sequences. They can write this custom thunk and use
`-mretpoline-external-thunk` *in addition* to `-mretpoline`. In this
case, on x86-64 thu thunk names must be:
```
__llvm_external_retpoline_r11
```
or on 32-bit:
```
__llvm_external_retpoline_eax
__llvm_external_retpoline_ecx
__llvm_external_retpoline_edx
__llvm_external_retpoline_push
```
And the target of the retpoline is passed in the named register, or in
the case of the `push` suffix on the top of the stack via a `pushl`
instruction.
There is one other important source of indirect branches in x86 ELF
binaries: the PLT. These patches also include support for LLD to
generate PLT entries that perform a retpoline-style indirection.
The only other indirect branches remaining that we are aware of are from
precompiled runtimes (such as crt0.o and similar). The ones we have
found are not really attackable, and so we have not focused on them
here, but eventually these runtimes should also be replicated for
retpoline-ed configurations for completeness.
For kernels or other freestanding or fully static executables, the
compiler switch `-mretpoline` is sufficient to fully mitigate this
particular attack. For dynamic executables, you must compile *all*
libraries with `-mretpoline` and additionally link the dynamic
executable and all shared libraries with LLD and pass `-z retpolineplt`
(or use similar functionality from some other linker). We strongly
recommend also using `-z now` as non-lazy binding allows the
retpoline-mitigated PLT to be substantially smaller.
When manually apply similar transformations to `-mretpoline` to the
Linux kernel we observed very small performance hits to applications
running typical workloads, and relatively minor hits (approximately 2%)
even for extremely syscall-heavy applications. This is largely due to
the small number of indirect branches that occur in performance
sensitive paths of the kernel.
When using these patches on statically linked applications, especially
C++ applications, you should expect to see a much more dramatic
performance hit. For microbenchmarks that are switch, indirect-, or
virtual-call heavy we have seen overheads ranging from 10% to 50%.
However, real-world workloads exhibit substantially lower performance
impact. Notably, techniques such as PGO and ThinLTO dramatically reduce
the impact of hot indirect calls (by speculatively promoting them to
direct calls) and allow optimized search trees to be used to lower
switches. If you need to deploy these techniques in C++ applications, we
*strongly* recommend that you ensure all hot call targets are statically
linked (avoiding PLT indirection) and use both PGO and ThinLTO. Well
tuned servers using all of these techniques saw 5% - 10% overhead from
the use of retpoline.
We will add detailed documentation covering these components in
subsequent patches, but wanted to make the core functionality available
as soon as possible. Happy for more code review, but we'd really like to
get these patches landed and backported ASAP for obvious reasons. We're
planning to backport this to both 6.0 and 5.0 release streams and get
a 5.0 release with just this cherry picked ASAP for distros and vendors.
This patch is the work of a number of people over the past month: Eric, Reid,
Rui, and myself. I'm mailing it out as a single commit due to the time
sensitive nature of landing this and the need to backport it. Huge thanks to
everyone who helped out here, and everyone at Intel who helped out in
discussions about how to craft this. Also, credit goes to Paul Turner (at
Google, but not an LLVM contributor) for much of the underlying retpoline
design.
Reviewers: echristo, rnk, ruiu, craig.topper, DavidKreitzer
Subscribers: sanjoy, emaste, mcrosier, mgorny, mehdi_amini, hiraditya, llvm-commits
Differential Revision: https://reviews.llvm.org/D41723
llvm-svn: 323155
This was added to mimic ELF, but maintaining it has cost
and we currently don't have any use for it outside of the
test code.
Differential Revision: https://reviews.llvm.org/D42324
llvm-svn: 323154
It is possible for a link to fail with an undefined reference, unless
--gc-sections is specified, removing the reference in the process. This
doesn't look to be tested anywhere explicitly, so I thought it useful
to add a test for it to ensure the behaviour is maintained.
Reviewers: ruiu
Differential Revision: https://reviews.llvm.org/D42299
llvm-svn: 323099
Its much easier to export it via setHidden(false), now that
that is a thing.
As a side effect the start function is not longer always exports first
(becuase its being exported just like all the other function).
Differential Revision: https://reviews.llvm.org/D42321
llvm-svn: 323025
This code was needed back when we were not able to write
out the synthetic symbol for main.
Add tests to make sure we can handle this now.
Differential Revision: https://reviews.llvm.org/D42322
llvm-svn: 323020
Compiler doesn't know the fact that Config->WordSize * 8 is always a
power of two, so it had to use the div instruction to divide some
number with C.
llvm-svn: 323014
I created https://reviews.llvm.org/D42202 to see how large the bloom
filter should be. With that patch, I tested various bloom filter sizes
with the following commands:
$ cmake -GNinja -DCMAKE_BUILD_TYPE=Debug -DLLVM_ENABLE_LLD=true \
-DLLVM_ENABLE_PROJECTS='clang;lld' -DBUILD_SHARED_LIBS=ON \
-DCMAKE_SHARED_LINKER_FLAGS=-Wl,-bloom-filter-bits=<some integer> \
../llvm-project/llvm
$ rm -f $(find . -name \*.so.7.0.0svn)
$ ninja lld
$ LD_BIND_NOW=1 perf stat bin/ld.lld
Here is the result:
-bloom-filter-bits=8 0.220351609 seconds
-bloom-filter-bits=10 0.217146597 seconds
-bloom-filter-bits=12 0.206870826 seconds
-bloom-filter-bits=16 0.209456312 seconds
-bloom-filter-bits=32 0.195092075 seconds
Currently we allocate 8 bits for a symbol, but according to the above
result, that number is not optimal. Even though the numbers follow the
diminishing return rule, the point where a marginal improvement becomes
too small is not -bloom-filter-bits=8 but 12. So this patch sets it to 12.
Differential Revision: https://reviews.llvm.org/D42204
llvm-svn: 323010
We need these import since relocations are generated against them.
Patch by Nicholas Wilson!
Differential Revision: https://reviews.llvm.org/D42305
llvm-svn: 322990
This solves the problem that --emit-relocs needs the stack-pointer
to be exported, in order to write out any relocations that reference
the __stack_pointer symbol by its symbol index.
Patch by Nicholas Wilson!
Differential Revision: https://reviews.llvm.org/D42237
llvm-svn: 322911
When writing relocatable files we were exporting for all globals
(including file-local syms), but not for functions. Oops. To be
consistent with non-relocatable output, all symbols (file-local
and global) should be exported. Any symbol targetted by further
relocations needs to be exported. The lack of local function
exports was just an omission, I think.
Second bug: Local symbol names can collide, causing an illegal
Wasm file to be generated! Oops again. This only previously affected
producing relocatable output from two files, where each had a global
with the same name. We need to "budge" the symbol names for locals
that are exported on relocatable output.
Third bug: LLD's relocatable output wasn't writing out any symbol
flags! Thus the local globals weren't being marked as local, and
the hidden flag was also stripped...
Added tests to exercise colliding local names with/without
relocatable flag
Patch by Nicholas Wilson!
Differential Revision: https://reviews.llvm.org/D42105
llvm-svn: 322908
There's some abstraction overhead in the underlying
mechanisms that were being used, and it was leading to an
abundance of small but not-free copies being made. This
showed up on a profile. Eliminating this and going back to
a low-level byte-based implementation speeds up lld with
/DEBUG between 10 and 15%.
Differential Revision: https://reviews.llvm.org/D42148
llvm-svn: 322871
We need to decompose relocation type for N32 / N64 ABI. Let's do it
before any other manipulations with relocation type in the `relocateOne`
routine.
llvm-svn: 322860
The problem we had with it is that anything inside an AT is an
expression, so we failed to parse the section name because of the - in
it.
llvm-svn: 322801
Simplify generation of "names" section by simply iterating
over the DefinedFunctions array.
This even fixes some bugs, judging by the test changes required.
Some tests are asserting that functions are named multiple times,
other tests are asserting that the "names" section contains the
function's alias rather than its original name
Patch by Nicholas Wilson!
Differential Revision: https://reviews.llvm.org/D42076
llvm-svn: 322751
The classes used to print and update time information are in
common, so other linkers could use this as well if desired.
Differential Revision: https://reviews.llvm.org/D41915
llvm-svn: 322736
Previously we always handled -defsym after other commands in command line.
That made impossible to overload values set by -defsym from linker script:
test.script:
foo = 0x22;
-defsym=foo=0x11 -script t.script
would always set foo to 0x11.
That is inconstent with common logic which allows to override command line
options. it is inconsistent with bfd behavior and seems breaks assumption that
-defsym is the same as linker script assignment, as -defsyms always handled out of
command line order.
Patch fixes the handling order.
Differential revision: https://reviews.llvm.org/D42054
llvm-svn: 322625
This is an immutable exported global representing
the start of the heap area. It is a page aligned.
Differential Revision: https://reviews.llvm.org/D42030
llvm-svn: 322609
This is used by __cxa_ataxit to determine the currently
executing DLL. Once we fully support DLLs this will need
to be set to some address within the DLL.
The ELF linker added support for this symbol here:
https://reviews.llvm.org/D33856
Differential Revision: https://reviews.llvm.org/D42024
llvm-svn: 322606
Patch by Colden Cullen.
Currently, when a large PE (>4 GiB) is to be produced, a crash occurs
because:
1. Calling setOffset with a number greater than UINT32_MAX causes the
PointerToRawData to overflow
2. When adding the symbol table to the end of the file, the last section's
offset was used to calculate file size. Because this had overflowed,
this number was too low, and the file created would not be large enough.
This lead to the actual crash I saw, which was a buffer overrun.
This change:
1. Adds comment to setOffset, clarifying that overflow can occur, but it's
somewhat safe because the error will be handled elsewhere
2. Adds file size check after all output data has been created This matches
the MS link.exe error, which looks prints as: "LINK : fatal error
LNK1248: image size (10000EFC9) exceeds maximum allowable size
(FFFFFFFF)"
3. Changes calculate of the symbol table offset to just use the existing
FileSize. This should match the previous calculations, but doesn't rely
on the use of a u32 that can overflow.
4. Removes trivial usage of a magic number that bugged me while I was
debugging the issue
I'm not sure how to add a test for this outside of adding 4GB of object
files to the repo. If there's an easier way, let me know and I'll be
happy to add a test.
Differential Revision: https://reviews.llvm.org/D42010
llvm-svn: 322605
Symbol had both Visibility and getVisibility() and they had different
meanings. That is just too easy to get wrong.
getVisibility() would compute the visibility of a particular symbol
(foo in bar.o), and Visibility stores the computed value we will put
in the output.
There is only one case when we want what getVisibility() provides, so
inline it.
llvm-svn: 322590
We track both the combined visibility that will be used for the output
symbol and the original input visibility of the selected symbol.
Almost everything should use the computed visibility.
I will make the names less confusing an a followup patch.
llvm-svn: 322576
Rui Ueyama