Relocations.cpp is still head-scratching. Even though relocations are
processed by multiple functions, the functions are effectively one
gigantic function with lots of local and global shared states, because
they are really tightly coupled. It is really hard to predict whether
a change to a function will or will not affect other functions behaviors.
What I'm trying to do is to rewrite the code without breaking the
existing tests so that the code can tolerate a more aggressive
refactoring (i.e. splitting it to logically separated steps).
llvm-svn: 315673
This is not a mechanical transformation. Even though I believe this
patch is correct, I'm not 100% sure if lld with this patch behaves
exactly the same way as before on all edge cases. At least all tests
still pass.
I'm submitting this patch because it took almost a day to understand
this function, and I don't want to lose it.
llvm-svn: 315658
This patch merges computeAddend and computeMipsAddend.
Getting an addend for a relocation is usually pretty easy:
it is either in the r_addend field (if RELA) or in a target
section (if REL).
However, MIPS has many special rules that are different from
other ELF ABIs. I don't think there were technical reasons to
be different, but the reality is that they are different.
It is unfortunate that we had to pass many parameters to
computeAddend, but it seems unavoidable because of MIPS.
llvm-svn: 315617
This is an attempt to make lld's relocation handler code understandable.
Since I don't fully understand what exactly this function does for all
possible cases (I believe no one can), I'm not really sure if this patch
is NFC, but at least no functionality change intended. All tests still pass.
llvm-svn: 315612
It doesn't make sense to link from an alloc section to a non alloc
one as we were doing.
This matches the behavior of both gold and bfd.
llvm-svn: 315584
Previously we did not support following:
foo = ~0xFF;
and had to add space before numeric value:
foo = ~ 0xFF
That was constistent with ld.bfd < 2.30, which shows:
script.txt:3: undefined symbol `~2' referenced in expression,
but inconsistent with gold.
It was fixed for ld.bfd 2.30 as well:
https://sourceware.org/bugzilla/show_bug.cgi?id=22267
Differential revision: https://reviews.llvm.org/D36508
llvm-svn: 315569
A section was passed to getRelExpr just to create an error message.
But if there's an invalid relocation, we would eventually report it
in relocateOne. So we don't have to pass a section to getRelExpr.
llvm-svn: 315552
We were using uint32_t as the type of relocation kind. It has a
readability issue because what Type really means in `uint32_t Type`
is not obvious. It could be a section type, a symbol type or a
relocation type.
Since we do not do any arithemetic operations on relocation types
(e.g. adding one to R_X86_64_PC32 doesn't make sense), it would be
more natural if they are represented as enums. Unfortunately, that
is not doable because relocation type definitions are spread into
multiple header files.
So I decided to use typedef. This still should be better than the
plain uint32_t because the intended type is now obvious.
llvm-svn: 315525
This patch doesn't change the behavior of the program because it
would eventually return None at end of the function. But it is
better to return None early if we know it will eventually happen.
llvm-svn: 315495
This is PR34546.
Currently LLD creates output sections even if it has no input sections,
but its command contains an assignment.
Committed code just assigns the same flag that was used in previous
live section.
That does not work sometimes. For example if we have following script:
.ARM.exidx : { *(.ARM.exidx*) }
.foo : { _foo = 0; } }
Then first section has SHF_LINK_ORDER flag. But section foo should not.
That was a reason of crash in OutputSection::finalize(). LLD tried to calculate
Link value, calling front() on empty input sections list.
We should only keep access flags and omit all others when creating such sections.
Patch fixes the crash observed.
Differential revision: https://reviews.llvm.org/D37736
llvm-svn: 315441
Usually, a function that does symbol lookup takes symbol name as
its first argument. Also, if a function takes a source location hint,
it is usually the last parameter. So the previous parameter order
was counter-intuitive.
llvm-svn: 315433
Because of r314495, DefinedCommon symbols cannot reach to
getSymbolValue function. When they reach the fucntion, they have
already been converted to DefinedRegular symbols.
llvm-svn: 315432
We used CurAddressState to capture a dynamic context just like
we use lambdas to capture static contexts. So, CurAddressState
is used everywhere in LinkerScript.cpp. It is worth a shorter
name.
llvm-svn: 315418
"Commands" was ambiguous because in the linker script, everything is
a command. We used to handle only SECTIONS commands, and at the time,
it might make sense to call them the commands, but it is no longer
the case. We handle not only SECTIONS but also MEMORY, PHDRS, VERSION,
etc., and they are all commands.
llvm-svn: 315409
HasSections is true if there is at least one SECTIONS linker
script command, and it is not directly related to whether we have
section objects or not. So I think the new name is better.
llvm-svn: 315405
ScriptConfiguration was a class to contain parsed results of
linker scripts. LinkerScript is a class to interpret it.
That ditinction was needed because we haven't instantiated
LinkerScript early (because, IIRC, LinkerScript class was a
ELFT template function). So, when we parse linker scripts,
we couldn't directly store the result to a LinkerScript instance.
Now, that limitation is gone. We instantiate LinkerScript
at the very beginning of our main function. We can directly
store parse results to a LinkerScript instance.
llvm-svn: 315403
Because addRegular's functionality is tightly coupled with
addSymbol, and the former is called only once, it makes sense
to merge the two functions. This patch also adds comments.
llvm-svn: 315401
The condition whether a section is alive or not by default
is becoming increasingly complex, so the decision of garbage
collection is spreading over InputSection.h and MarkLive.cpp,
which is not a good state.
This moves the code to MarkLive.cpp, to keep the file the central
place to make decisions about garbage collection.
llvm-svn: 315384
We just don't need one with the current setup.
We only error on undefined references that are used by some
relocation.
If we managed to relax all uses of __tls_get_addr, no relocation uses
it and we don't produce an error.
This is less code and fixes the case were we fail to relax. Before we
would produce a broken output, but now we produce an error.
llvm-svn: 315334
When parsing linker scripts, LLD previously started with a '.' value of 0,
regardless of the internal default image base for the target, and regardless of
switches such as --image-base. It seems reasonable to use a different image base
value when using linker scripts and --image-base is specified, since otherwise the
switch has no effect. This change does this, as well as removing unnecessary
initialisation of Dot where it is not used.
The default image base should not be used when processing linker
scripts, because this will change the behaviour for existing linker script users,
and potentially result in invalid output being produced, as a subsequent assignment
to Dot could move the location counter backwards. Instead, we maintain the existing
behaviour of starting from 0 if --image-base is not specified.
Reviewers: ruiu
Differential Revision: https://reviews.llvm.org/D38360
llvm-svn: 315293
When findOrphanPos does the reverse search to find the OutputSection
preceding the orphan's insertion point, look for a live OutputSection
and ignore "dead" OutputSection's. This matches the behaviour of the
forward search performed earlier in this function.
Added test which without the above fix fails as a result of an orphan
executable section being incorrectly placed in a non-executable segment.
Differential Review: https://reviews.llvm.org/D38690
llvm-svn: 315292
Previously, when we added an input section to an output section, we
called `OutputSectionFactory::addInputSec`. This isn't a good design
because, a factory class is intended to create a new object and
return it, but in this use case, it will never create a new object.
This patch fixes the design flaw.
llvm-svn: 315138
Factory::addInputSec added an output section to Script->Opt.Commands,
but that is too subtle. This patch makes it explicit so that it is easy
to see when a new element is added to Script->Opt.Commands.
llvm-svn: 315129
This patch moves a std::find to a new function. It also removes
the following piece of code. I believe it should be fine because all
tests still pass.
unsigned Index = std::distance(Opt.Commands.begin(), I);
assert(Sec->SectionIndex == INT_MAX || Sec->SectionIndex == Index);
Sec->SectionIndex = Index;
llvm-svn: 315125
addSection function was hard to read because it behaves differently
depending on its arguments but what exactly it does is not clear.
Now it should be better. Still, it is not clear (not what but) why
it does what it does, but I'll take a look at it later.
llvm-svn: 315124
I hadn't synced past the change that changed the default hash style
to --hash-style=both, so my test had the symbols in the wrong order.
llvm-svn: 315119
Dynamic lists in an executable are additive, not restrictive, so we
must continue to export preempted symbols even with a dynamic list.
This fixes sanitizer interception of libc symbols (and should also fix
symbol preemption by users of sanitizers).
Differential Revision: https://reviews.llvm.org/D38647
llvm-svn: 315114
This reverts commit r314616 because nested parallel-for loops can hang.
Our ThreadPoolExecutor is not necessarily able to handle nested calls.
llvm-svn: 315098
Before this patch we would copy foo into real_foo and wrap_foo into
foo. The net result is that __wrap_foo shows up twice in the symbol
table.
With this patch we:
* save a copy of __real_foo before copying foo.
* drop one of the __wrap_foo from the symbol table.
* if __real_foo was not undefined, add a *new* symbol with that content to
the symbol table.
The net result is that
Anything using foo now uses __wrap_foo
Anything using __real_foo now uses foo.
Anything using __wrap_foo still does.
And the symbol table has foo, __wrap_foo and __real_foo (if defined).
Which I think is the desired behavior.
llvm-svn: 315097
Summary:
These are 16 bit relocations and not part of a HI/LO pair so we need to
check that they don't overflow.
Reviewers: atanasyan
Reviewed By: atanasyan
Subscribers: ruiu, llvm-commits, emaste, sdardis
Tags: #lld
Differential Revision: https://reviews.llvm.org/D38614
llvm-svn: 315073
LLD produces broken .dynamic section when --no-rosegment and at least
one of following options is present:
1) -z rodynamic is given.
2) MIPS target.
That happens because code that writes .dynamic assumes target buffer
is zero-filled, what can be not true after LLD fills it with trap
instructions. With one of two options above, .dynamic becomes
SHF_ALLOC section, so can be affected.
Differential revision: https://reviews.llvm.org/D38580
llvm-svn: 315054
We produce broken output currently.
Code that writes .dynsym assumes output buffer is zero-filled,
though that is not always true. When --no-rosegment is given,
buffer can be filled with trap instructions. Patch fixes the issue.
It is relative with PR34705.
Differential revision: https://reviews.llvm.org/D38579
llvm-svn: 315053
Its PR34712,
GNU linkers recently changed default values to "both" of "sysv".
Patch do the same for all targets except MIPS, where .gnu.hash
section is not yet supported.
Code suggested by Rui Ueyama.
Differential revision: https://reviews.llvm.org/D38407
llvm-svn: 315051
Summary:
We were crashing when linking telnetd in FreeBSD because lld was emitting
corrupted output files for --norosegment. In this file the version index of some symbols
was set to 9 but lld only found 8 version definitions.
I am not sure how to create a minimal .so file that also exposes this behaviour so I just added the one that initially caused the error to Inputs/
This partially addresses https://bugs.llvm.org/show_bug.cgi?id=34705
Reviewers: ruiu, rafael, pcc, grimar
Reviewed By: ruiu
Subscribers: emaste, krytarowski
Tags: #lld
Differential Revision: https://reviews.llvm.org/D38397
llvm-svn: 315036
Summary:
We were crashing when linking telnetd in FreeBSD because lld was emitting
corrupted output files for --norosegment. In this file the version index of some symbols
was set to 9 but lld only found 8 version definitions.
I am not sure how to create a minimal .so file that also exposes this behaviour so I just added the one that initially caused the error to Inputs/
This partially addresses https://bugs.llvm.org/show_bug.cgi?id=34705
Reviewers: ruiu, rafael, pcc, grimar
Reviewed By: ruiu
Subscribers: emaste, krytarowski
Tags: #lld
Differential Revision: https://reviews.llvm.org/D38397
llvm-svn: 315035
I think it is not defined what would happen to detached threads
when the main thread tries to exit. That means it was not guaranteed
that unlinkAsync correctly removes a temporary file. It was also
reported that this unlinkAsync caused a crash on Windows.
This patch adds a few new functions so that the main thread always
waits for non-main threads before exitting.
I don't actually like the new two functions, runBackground and
waitForBackgroundThreads, because it looks like it is a bit
overdesigned. After all, what we are doing with these functions
is to just remove a file.
An alternative would be to do fork(2) and make the child process
remove a file asynchronously. However, it has its own problems.
Correctly forking and reclaiming a resource using waitpid(2) is not
doable unless we know our process-wide settings (such as signal mask),
but we can't make any assumption on it when lld is embedded to other
process. So I chose to stick with threads instead of multi-processes.
Differential Revision: https://reviews.llvm.org/D38571
llvm-svn: 315033
We used to call exitLld() from a leaf function, Writer::run(), because
we had objects on the stack whose dtors are expensive. Now we no longer
have such objects on the stack, so there's no reason to exist from the
leaf function.
llvm-svn: 314869
When reporting a symbol conflict, LLD parses the debug info to report
source location information. Sections have not been decompressed at this
point, so if an object file contains zlib compressed debug info, LLD
ends up passing this compressed debug info to the DWARF parser, which
causes debug info parsing failures and can trigger assertions in the
parser (as the test case demonstrates).
Decompress debug sections when constructing the LLDDwarfObj to avoid
this issue. This doesn't handle GNU-style compressed debug info sections
(.zdebug_*), which at present are simply ignored by LLDDwarfObj; those
can be done in a follow-up.
Differential Revision: https://reviews.llvm.org/D38491
llvm-svn: 314866
We have this comment in LinkerDriver::link
After this, no new names except a few linker-synthesized ones
will be added to the symbol table.
but that was not true because new symbols could be added by processing
the -u option.
llvm-svn: 314842
The issue with std:🧵:hardware_concurrency is that it forwards
to libc and some implementations (like glibc) don't take thread
affinity into consideration.
With this change a llvm program that can execute in only 2 cores will
use 2 threads, even if the machine has 32 cores.
This makes benchmarking a lot easier, but should also help if someone
doesn't want to use all cores for compilation for example.
llvm-svn: 314810
If symbol has the STO_MIPS_MICROMIPS flag and requires a thunk to perform
call PIC from non-PIC functions, we need to generate a thunk with microMIPS
code.
llvm-svn: 314797
New lld's files are spread under lib subdirectory, and it isn't easy
to find which files are actually maintained. This patch moves maintained
files to Common subdirectory.
Differential Revision: https://reviews.llvm.org/D37645
llvm-svn: 314719
Reads from `Live` and writes to `OutputOff` in the following code race
even though they are logically independent because they are bitfields
sharing the same word.
for (size_t I = 0, E = Sec->Pieces.size(); I != E; ++I) {
if (!Sec->Pieces[I].Live)
continue;
CachedHashStringRef Str = Sec->getData(I);
size_t ShardId = getShardId(Str.hash());
if ((ShardId & (Concurrency - 1)) == ThreadId)
Sec->Pieces[I].OutputOff = Shards[ShardId].add(Str);
}
llvm-svn: 314711
Currently LLD calls the `isMicroMips` routine to determine type of PLT entries
needs to be generated: regular or microMIPS. This routine checks ELF
header flags in the `FirstObj` to retrieve type of linked object files.
So if the first file does not contain microMIPS code, LLD will generate
PLT entries with regular (non-microMIPS) code only.
Ideally, if a PLT entry is referenced by microMIPS code only this entry
should contain microMIPS code, if a PLT entry is referenced by regular
code this entry should contain regular code. In a "mixed" case the PLT
entry can be either microMIPS or regular, but each "cross-mode-call" has
additional cost.
It's rather difficult to implement this ideal solution. But we can
assume that if there is an input object file with microMIPS code, the
most part of the code is microMIPS too. So we need to deduce type of PLT
entries based on finally calculated ELF header flags and do not check
only the first input object file.
This change implements this.
- The `getMipsEFlags` renamed to the `calcMipsEFlags`. The function
called from the `LinkerDriver::link`. Result is stored in
the Configuration::MipsEFlags field.
- The `isMicroMips` and `isMipsR6` routines access the `MipsEFlags`
field to get and check calculated ELF flags.
- New types of PLT records created when necessary.
Differential revision: https://reviews.llvm.org/D37747
llvm-svn: 314675
That makes code a bit more consistent. Instead of removing sections there
we can just mark them as dead. So that removeEmptyCommands() will
handle the rest.
Differential revision: https://reviews.llvm.org/D38393
llvm-svn: 314654
Computing section content hashes early seems like a win in terms of
performance. It increases a chance that two different sections will get
different class IDs from the beginning.
Without threads, this patch improves Chromium link time by about 0.3
seconds. With threads, by 0.1 seconds. That's less than 1% time saving
but not bad for a small patch.
llvm-svn: 314644
I don't know why we didn't use parallelForEach to call writeTo,
but there should be no reason to not do that, as most writeTo
functions are safe to run concurrently.
llvm-svn: 314616
The result of hash_value(StringRef) depends on sizeof(size_t).
That causes lld to create different mergeable table contents on
32-bit machines.
This patch is to use xxHash64 so that we get the same hash values
on 32-bit machines.
llvm-svn: 314603
String merging is one of the most time-consuming functions in lld.
This patch parallelize it to speed it up. On my 2-socket 20-core
40-threads Xeon E5-2680 @ 2.8 GHz machine, this patch shorten the
clang debug build link time from 7.11s to 5.16s. It's a 27%
improvement and actually pretty noticeable. In this test condition,
lld is now 4x faster than gold.
Differential Revision: https://reviews.llvm.org/D38266
llvm-svn: 314588
Convert all common symbols to regular symbols after scan.
This means that the downstream code does not to handle common symbols as a special case.
Differential Revision: https://reviews.llvm.org/D38137
llvm-svn: 314495
This is "Bug 34688 - lld much slower than bfd when linking the linux kernel"
Inside copyRelocations() we have O(N*M) algorithm, where N - amount of
relocations and M - amount of symbols in symbol table. It isincredibly slow
for linking linux kernel.
Patch creates local search tables to speedup.
With this fix link time goes for me from 12.95s to 0.55s what is almost 23x
faster. (used release LLD).
Differential revision: https://reviews.llvm.org/D38129
llvm-svn: 314282
SymbolTable::insert() is a hot path function. When linking a clang debug
build, the function is called 3.7 million times. The total amount of "Name"
string contents is 300 MiB. That means this `Name.find("@@")` scans almost
300 MiB of data. That's far from negligible.
StringRef::find(StringRef) uses a sophisticated algorithm, but the
function is slow for a short needle. This patch replaces it with
StringRef::find(char).
This patch alone speeds up a clang debug build link time by 0.5 seconds
from 8.2s to 7.7s. That's 6% speed up. It seems too good for this tiny
change, but looks like it's real.
llvm-svn: 314192
[Synopsys]
Using function elf::link(...) leads to segmentation fault on its second call. First call finishes correctly.
[Solution]
Clear the rest of globals.
Reviewed by: George Rimar and Rui Ueyama
Differential Revision: http://reviews.llvm.org/D38131
llvm-svn: 314108
Previously`InX::Got` and InX::MipsGot synthetic sections
were not removed if ElfSym::GlobalOffsetTable was defined.
ElfSym::GlobalOffsetTable is a symbol for _GLOBAL_OFFSET_TABLE_.
Patch moves ElfSym::GlobalOffsetTable check out from removeUnusedSyntheticSections.
Also note that there was no point to check ElfSym::GlobalOffsetTable for MIPS case
because InX::MipsGot::empty() always returns false for non-relocatable case, and in case
of relocatable output we do not create special symbols anyways.
Differential revision: https://reviews.llvm.org/D37623
llvm-svn: 314099
Rafael Espindola