Previously, all input files were owned by the symbol table.
Files were created at various places, such as the Driver, the lazy
symbols, or the bitcode compiler, and the ownership of new files
was transferred to the symbol table using std::unique_ptr.
All input files were then free'd when the symbol table is freed
which is on program exit.
I think we don't have to transfer ownership just to free all
instance at once on exit.
In this patch, all instances are automatically collected to a
vector and freed on exit. In this way, we no longer have to
use std::unique_ptr.
Differential Revision: https://reviews.llvm.org/D24493
llvm-svn: 281425
Implemented by building an ELF file in memory.
elf, default, and binary match gold behavior.
Differential Revision: https://reviews.llvm.org/D24060
llvm-svn: 281108
This section supersedes .reginfo and .MIPS.options sections. But for now
we have to support all three sections for ABI transition period.
llvm-svn: 278482
This flag is implemented similarly to --reproduce in the ELF linker.
This patch implements /linkrepro by moving the cpio writer and associated
utility functions to lldCore, and using that implementation in both linkers.
One COFF-specific detail is that we store the object file from which the
resource files were created in our reproducer, rather than the resource
files themselves. This allows the reproducer to be used on non-Windows
systems for example.
Differential Revision: https://reviews.llvm.org/D22418
llvm-svn: 276719
Symbol's dtors are not called because they are allocated using
BumpPtrAllocators. So, members of std::unique_ptr type are not
freed when symbols are deallocated.
This patch is to allocate Thunks using BumpPtrAllocators.
llvm-svn: 274896
Previously, we initialized Config->EKind and Config->EMachine when
we instantiate ELF objects. That was not an ideal location to do that
because the logic was buried too deep inside a concrete logic.
This patch moves the code to the driver so that the initialization
becomes explicit.
Differential Revision: http://reviews.llvm.org/D21784
llvm-svn: 274089
Patch by Shridhar Joshi.
This option provides names of all the link time modules which define and
reference symbols requested by user. This helps to speed up application
development by detecting references causing undefined symbols.
It also helps in detecting symbols being resolved to wrong (unintended)
definitions in case of applications containing multiple definitions for
same symbols with different types, bindings.
Implements PR28226.
llvm-svn: 273536
This should never happen with correct programs, but it is trivial
write a testcase where lld would crash or report duplicated
symbols. We now behave like when an archive is used and include the
file only once.
llvm-svn: 272724
MIPS N64 ABI introduces .MIPS.options section which specifies miscellaneous
options to be applied to an object/shared/executable file. LLVM as well as
modern versions of GNU tools read and write the only type of the options -
ODK_REGINFO. It is exact copy of .reginfo section used by O32 ABI.
llvm-svn: 268485
This patch implements a new design for the symbol table that stores
SymbolBodies within a memory region of the Symbol object. Symbols are mutated
by constructing SymbolBodies in place over existing SymbolBodies, rather
than by mutating pointers. As mentioned in the initial proposal [1], this
memory layout helps reduce the cache miss rate by improving memory locality.
Performance numbers:
old(s) new(s)
Without debug info:
chrome 7.178 6.432 (-11.5%)
LLVMgold.so 0.505 0.502 (-0.5%)
clang 0.954 0.827 (-15.4%)
llvm-as 0.052 0.045 (-15.5%)
With debug info:
scylla 5.695 5.613 (-1.5%)
clang 14.396 14.143 (-1.8%)
Performance counter results show that the fewer required indirections is
indeed the cause of the improved performance. For example, when linking
chrome, stalled cycles decreases from 14,556,444,002 to 12,959,238,310, and
instructions per cycle increases from 0.78 to 0.83. We are also executing
many fewer instructions (15,516,401,933 down to 15,002,434,310), probably
because we spend less time allocating SymbolBodies.
The new mechanism by which symbols are added to the symbol table is by calling
add* functions on the SymbolTable.
In this patch, I handle local symbols by storing them inside "unparented"
SymbolBodies. This is suboptimal, but if we do want to try to avoid allocating
these SymbolBodies, we can probably do that separately.
I also removed a few members from the SymbolBody class that were only being
used to pass information from the input file to the symbol table.
This patch implements the new design for the ELF linker only. I intend to
prepare a similar patch for the COFF linker.
[1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098832.html
Differential Revision: http://reviews.llvm.org/D19752
llvm-svn: 268178
Using multiple context used to be a really big memory saving because we
could free memory from each file while the linker proceeded with the
symbol resolution. We are getting lazier about reading data from the
bitcode, so I was curious if this was still a good tradeoff.
One thing that is a bit annoying is that we still have to copy the
symbol names. The problem is that the names are stored in the Module and
get freed when we move the module bits during linking.
Long term I think the solution is to add a symbol table to the bitcode.
That way IRObject file will not need to use a Module or a Context and we
can drop it while still keeping a StringRef to the names.
This patch is still be an interesting medium term improvement.
When linking llvm-as without debug info this patch is a small speedup:
master: 29.861877513 seconds
patch: 29.814533787 seconds
With debug info the numbers are
master: 34.765181469 seconds
patch: 34.563351584 seconds
The peak memory usage when linking llvm-as with debug info was
master: 599.10MB
patch: 600.13MB
llvm-svn: 267921
It turns out that this will read data from the section to properly
handle Elf_Rel implicit addends.
Sorry for the noise.
Original messages:
Try to fix Windows lld build.
Move getRelocTarget to ObjectFile.
It doesn't use anything from the InputSection.
llvm-svn: 267163
start-lib and end-lib are options to link object files in the same
semantics as archive files. If an object is in start-lib and end-lib,
the object is linked only when the file is needed to resolve
undefined symbols. That means, if an object is in start-lib and end-lib,
it behaves as if it were in an archive file.
In this patch, I introduced a new notion, LazyObjectFile. That is
analogous to Archive file type, but that works for a single object
file instead of for an archive file.
http://reviews.llvm.org/D18814
llvm-svn: 265710
Summary:
This bug was introduced by http://reviews.llvm.org/rL265059,
where InputSectionBase got Thunks field, which can do memory allocations.
Since InputSectionBase destructors were never called (I count it as another bug),
that caused a memory leak when 2 or more thunks are added to a section.
The fix to is properly call InputSectionBase destructors from ~ObjectFile.
Reviewers: atanasyan, ruiu, rafael
Subscribers: rafael, krasin, pcc
Differential Revision: http://reviews.llvm.org/D18809
llvm-svn: 265497
Our symbol representation was redundant, and some times would get out of
sync. It had an Elf_Sym, but some fields were copied to SymbolBody.
Different parts of the code were checking the bits in SymbolBody and
others were checking Elf_Sym.
There are two general approaches to fix this:
* Copy the required information and don't store and Elf_Sym.
* Don't copy the information and always use the Elf_Smy.
The second way sounds tempting, but has a big problem: we would have to
template SymbolBody. I started doing it, but it requires templeting
*everything* and creates a bit chicken and egg problem at the driver
where we have to find ELFT before we can create an ArchiveFile for
example.
As much as possible I compared the test differences with what gold and
bfd produce to make sure they are still valid. In most cases we are just
adding hidden visibility to a local symbol, which is harmless.
In most tests this is a small speedup. The only slowdown was scylla
(1.006X). The largest speedup was clang with no --build-id, -O3 or
--gc-sections (i.e.: focus on the relocations): 1.019X.
llvm-svn: 265293
That is directly opposite to http://reviews.llvm.org/D18045,
which was reverted.
This patch changes all messages to start from lowercase letter if
they were not before.
That is done to be consistent with clang.
Differential revision: http://reviews.llvm.org/D18085
llvm-svn: 263252
In lld we usually avoid hash lookups. In addition to that, IR names are
not fully mangled, so it is best to avoid using them whenever possible.
llvm-svn: 263248
Summary:
More generally, appending linkage is a special case that we don't want
to create a SymbolBody for.
Reviewers: rafael, ruiu
Subscribers: Bigcheese, llvm-commits, joker.eph
Differential Revision: http://reviews.llvm.org/D18012
llvm-svn: 263179
All MemoryBuffers for archive files are guaranteed to exist as long
as their children are used in the linker. So we don't need to copy
strings here. Thanks to Sean Silva for pointing this out.
llvm-svn: 259554
If object files are drawn from archive files, the error message should
be something like "conflict symbols in foo.a(bar.o) and baz.o" instead
of "conflict symbols in bar.o and baz.o". This patch implements that.
llvm-svn: 259475
This avoids the need to have reserve and addString in sync.
We avoid hashing the global symbols again. This means that we don't
merge a global symbol that has the same name as some other string, but
that doesn't seem very common. The string table size is the same in
clang an scylladb with or without hashing global symbols again.
llvm-svn: 259136
Unlike ObjectFile or ArchiveFile, SharedFile had two parse functions,
parseSoName() and parse(). parse must have been called after parseSoName,
but that requirement was not obvious from their names. (So it looked
like you could call parse() on a shared object file right away.)
This patch rename parseRest. It is now obvious that there's no single
parse function for the shared object file.
llvm-svn: 256898
Previously, we handle archive files with --whole-archive this way:
create instances of ArchiveFile, call getMembers to obtain memory
buffers of archive members, and create ObjectFiles for the members.
We didn't call anything except getMembers if --whole-archive was
specified.
I noticed that we didn't actually have to create ArchiveFile instaces
at all for that case. All we need is to get a list of memory buffers
for members, which can be done by a non-member function.
This patch removes getMembers member function from ArchiveFile.
Also removed unnecessary code for memory management.
llvm-svn: 256893
The R_MIPS_GPREL16 / R_MIPS_GPREL32 relocations use the following
expressions for calculations:
```
local symbol: S + A + GP0 - GP
global symbol: S + A - GP
GP - Represents the final gp value, i.e. _gp symbol
GP0 - Represents the gp value used to create the relocatable object
```
The GP0 value is taken from the .reginfo data section defined by an object
file. To implement that I keep a reference to `MipsReginfoInputSection`
in the `ObjectFile` class. This reference is used by the
`ObjectFile::getMipsGp0` method to return the GP0 value.
Differential Revision: http://reviews.llvm.org/D15760
llvm-svn: 256416
Section garbage collection is a feature to remove unused sections
from outputs. Unused sections are sections that cannot be reachable
from known GC-root symbols or sections. Naturally the feature is
implemented as a mark-sweep garbage collector.
In this patch, I added Live bit to InputSectionBase. If and only
if Live bit is on, the section will be written to the output.
Starting from GC-root symbols or sections, a new function, markLive(),
visits all reachable sections and sets their Live bits. Writer then
ignores sections whose Live bit is off, so that such sections are
excluded from the output.
This change has small negative impact on performance if you use
the feature because making sections means more work. The time to
link Clang changes from 0.356s to 0.386s, or +8%.
It reduces Clang size from 57,764,984 bytes to 55,296,600 bytes.
That is 4.3% reduction.
http://reviews.llvm.org/D13950
llvm-svn: 251043
BSD's DSO files have undefined symbol "__progname" which is defined
in crt1.o. On that system, both user programs and system shared
libraries depend on each other.
In general, we need to put symbols defined by user programs which are
referenced by shared libraries to user program's .dynsym.
http://reviews.llvm.org/D13637
llvm-svn: 250176
This patch adds AsNeeded and IsUsed bool fields to SharedFile. AsNeeded bit
is set if the DSO is enclosed with --as-needed and --no-as-needed. IsUsed
bit is off by default. When we adds a symbol to the symbol table for dynamic
linking, we set its SharedFile's IsUsed bit.
If AsNeeded is set but IsUsed is not set, we don't want to write that
file's SO name to DT_NEEDED field.
http://reviews.llvm.org/D13579
llvm-svn: 249998
Parse and apply emulation given with -m option.
Check input files to match ELF type and machine architecture provided with -m.
Differential Revision: http://reviews.llvm.org/D13055
llvm-svn: 249529
Summary:
If --whole-archive is used, all symbols from the following archives are added to the output. --no-whole-archive restores default behavior. These switches can be used multiple times.
NB. We have to keep an ArchiveFile instance within SymbolTable even if --whole-archive mode is active since it can be a thin archive which contains just names of external files. In that case actual memory buffers for the archive members will be stored within the File member of ArchiveFile class.
Reviewers: rafael, ruiu
Subscribers: grimar, llvm-commits
Projects: #lld
Differential Revision: http://reviews.llvm.org/D13286
llvm-svn: 249045
If a shared library has a DT_SONAME entry, that is what should be included
in the DT_NEEDED of a program using it.
We don't implement -soname yet, so check in a .so for now.
llvm-svn: 249025
Opening a file and dispatching to readLinkerScript() or createFile()
is a common operation. We want to use that at least from Driver and
from LinkerScript. In COFF, we had the same problem. This patch
resolves the problem in the same way as we did for COFF.
Now, if you have a path that you want to open, just call
Driver->addFile(StringRef). That function opens the file and handles
that as if that were given by command line. This function is the
only place we call identify_magic().
llvm-svn: 249023
This linker script parser and evaluator is powerful enough to read
Linux's libc.so, which is (despite its name) a linker script that
contains OUTPUT_FORMAT, GROUP and AS_NEEDED directives.
The parser implemented in this patch is a recursive-descendent one.
It does *not* construct an AST but consumes directives in place and
sets the results to Symtab object, like what Driver is doing.
This should be very fast since less objects are allocated, and
this is also more readable.
http://reviews.llvm.org/D13232
llvm-svn: 248918
Since FreeBSD 4.1, the kernel expects binaries to be marked with
ELFOSABI_FREEBSD in the ELF header to exec() them. LLD unconditionally
sets OSABI to ELF_OSABINONE, and everything linked with it won't run
on FreeBSD (unless explicitly rebranded).
Example:
% ./aarch64-hello
ELF binary type "0" not known.
zsh: exec format error: ./aarch64-hello
FreeBSD could be modified to accept ELF_OSABINONE, but that would break all
existing binaries, so the kernel needs to support both ABINONE and ABIFREEBSD.
I plan to push this change in FreeBSD one day, which, unfortunately, is
not today. This code patches lld so it sets the header field correctly.
For completeness, the rationale of this change is explained in the FreeBSD
commit message, and it's apparently to pleasee binutils maintainers at the time.
https://svnweb.freebsd.org/base?view=revision&revision=59342
Differential Revision: http://reviews.llvm.org/D13140
llvm-svn: 248554
This is more consistent with OutputSection. This is also part of removing
the "Chunk" term from the ELF linker, since we just have input/output sections
and program headers.
llvm-svn: 248183
Symbol table is now populated correctly, but some fields are missing,
they'll be added in the future. This patch also adds --discard-all
flag, which was the default behavior until now.
Differential Revision: http://reviews.llvm.org/D12874
llvm-svn: 247849
There were at least two issues with having them together:
* For compatibility checks, we only want to look at the ELF kind.
* Adding support for shared libraries should introduce one InputFile kind,
not 4.
llvm-svn: 246707
InputFiles.h:98:53: error: invalid use of incomplete type ‘class lld::elf2::SymbolBody’
return SymbolBodies[SymbolIndex - FirstNonLocal]->getReplacement();
llvm-svn: 246262
The others we have in sight are
* common symbols.
* entries in SHF_MERGE sections.
They will have a substantially different treatment. It is not clear if it is
worth it putting them all in a single list just to dispatch based on the kind on
the other side.
I hope to implement common symbols soon, and then we will be in a position
to have a concrete discussion. For now this is simpler for the the implemented
features.
llvm-svn: 244042
This is a direct port of the new PE/COFF linker to ELF.
It can take a single object file and generate a valid executable that executes at the first byte in the text section.
llvm-svn: 242088
Davide Italiano