Handle resolution of symbols coming from linked object files lazily.
Add implementation of handling _GLOBAL_OFFSET_TABLE_ and __exidx_start/_end symbols for ARM platform.
Differential Revision: http://reviews.llvm.org/D8159
llvm-svn: 232261
The Segment Chunk had two functions one to append a section and one to append a
chunk. A section is a subclass of a chunk and clearly this can be merged into
one single function.
llvm-svn: 232249
GNU LD has an option named -T/--script which allows a user to specify
a linker script to be used [1]. LLD already accepts linker scripts
without this option, but the option is widely used. Therefore it is
best to support it in LLD as well.
[1] https://sourceware.org/binutils/docs/ld/Options.html#Options
llvm-svn: 232183
This makes it a bit more like a 'real' iterator though I still haven't
gone through to make sure it meets the full requirements. Copy
assignability seems to be required by MSVC's std::find_if, which is its
right.
llvm-svn: 232097
The canonical LLVM directory arrangement places binaries in the 'utils/'
tree when they are used as part of building the project. For example,
the tblgen binaries are built out of 'utils/' trees.
Tools which are not used by any other part of the build, including
testing utilities, are just in the 'tools' directory. For example, in
Clang we have 'c-index-test' which is exactly the same kind of thing as
'linker-script-test'.
Differential Revision: http://reviews.llvm.org/D8269
llvm-svn: 231973
the spec required by std::sort and friends.
Ordering things this way also dramatically simplifies the code as
short-circuit ensures we can skip all of the negative tests.
I've left one FIXME where we're establishing a fairly arbitrary
ordering. Previously, the function compared all types as equal except
for the ones it explicitly handled, but it didn't delegate correctly to
the atomflags when doing so, and so it would fail to be a SWO. The two
possible fixes are to stop comparing the atom flags entirely, or to
establish some arbitrary ordering of the types.
Since it was pure luck which ordering of unequal types we ended up with
previously (the caller was std::sort, not std::stable_sort) I chose to
make the ordering explicit and guaranteed. This seems like the best
conservative approach as I suspect we would want to switch to
stable_sort otherwise in order to have deterministic output.
Differential Revision: http://reviews.llvm.org/D8266
llvm-svn: 231968
This patch implements parsing of the GNU ld MEMORY command [1].
The command and the memory block definitions are parsed as
specified (including the slightly strange "o" and "l" keywords).
Evaluation will be added at a later point in time.
[1] https://sourceware.org/binutils/docs-2.25/ld/MEMORY.html
llvm-svn: 231928
This will be replaced by a more generic class to handle
all the default symbols in an executable, e.g. __init_array.
Differential Revision: http://reviews.llvm.org/D8234
Reviewed by: shankare
llvm-svn: 231906
of the vector. For a vector 'v', '&v[v.size()]' isn't a valid way to
compute a pointer one-past-the-end of the vector. Instead, write the
loop in terms of iterators and save the beginning iterator. Once we have
that we can compute the beginning pointer from the beginning iterator,
and compute the distance which we should increment the beginning pointer
by subtracting the iterators.
What might be simpler would be to convert the function accepting a raw
pointer for begin and end to accept iterators or a range or some other
construct, but I wanted to keep this to a minimal bug-fix change.
This fixes a crash on any debug STL implementation which checks for
indexing out of bounds.
llvm-svn: 231765
The expression evaluation is needed when interpreting linker scripts, in order
to calculate the value for new symbols or to determine a new position to load
sections in memory. This commit extends Expression nodes from the linker script
AST with evaluation functions, and also contains a unit test.
http://reviews.llvm.org/D8156
llvm-svn: 231707
1. Move relocation addendum reading code to the MipsRelocationHandler
class to reduce code duplication.
2. Factor out the relocations calculation code into the separate
function to be ready to handle MIPS N64 ABI relocation chains.
No functional changes.
llvm-svn: 231641
We should not take in account a type of "source" symbol. Cross mode jump
adjustment is requred when target symbol and relocation belong to
different (regular/microMIPS) instruction sets.
llvm-svn: 231639
Previously, getEmptyKey and getTombstoneKey return the same value
in the sense of isEqual defined by the same class, although they
need to be distinct values. This could confuse DenseMap.
We didn't see any issue by this wrong code because we don't delete
elements from the symbol table. We only add or replace elements.
But this is a bug and needs to be fixed anyway.
llvm-svn: 231618
All readers except PE/COFF reader create layout-after edges to preserve
the original symbol order. PE/COFF uses layout-before edges as primary
edges for no reason.
This patch makes PE/COFF reader to create layout-after edges.
Resolver is updated to recognize reverse edges of layout-after edges
in the garbage collection pass.
Now we can retire layout-before edges. I don't do that in this patch
because if I do, I would have updated many tests to replace all
occurrrences of "layout-before" with "layout-after". So that's a TODO.
llvm-svn: 231615
If an output is large, its base relocation section can be also large.
For example, chrome.dll is almost 300 MB, and it has about 9 million
base relocations. Creating the section took 1.5 seconds on my machine.
This patch changes the way to create the section so that we can use
parallel_sort to group base relocations by high bits. This change
makes the linker almost 4% faster for the above test case on my machine.
If I replace parallel_sort with std::sort, performance remains the same,
so single thread performance should remain the same.
This has no functionality change. The output should be identical as
before.
llvm-svn: 231585
All defined symbols from all archive files are inserted to _archiveMap,
so performance of hash table matters here (I'm not trying to convert
all std::maps with DenseMaps). This change seems to make the linker
0.5% - 1% faster for my test case.
llvm-svn: 231584
This patch reverts r231545 "PECOFF: Do not add extraneous symbols
to the dead strip root." CrWinClangLLD buildbot is currently broken.
Since I can't reproduce the issue locally, I'm reverting the most
relevant change.
llvm-svn: 231582
Atoms with fallback atoms are never be added to the symbol table.
However, we added such atoms to _undefines array. We had to call
isCoalescedAway to identify and skip them. We should just stop
adding them in the first place.
This seems to make the linker ~1% faster in my test case.
llvm-svn: 231552
If an undefined symbol is added to the symbol table by the previous
call of SymbolTable::add, SymbolTable::isDefined will always return
false for the same symbol.
llvm-svn: 231551
This is yet another optimization patch. Previously we called
SymbolTable::isDefined() and SymbolTable::findByName() from a very
frequently executed function. Because isDefined calls findByName,
findByName is called twice on each iteration.
findByName is not a cheap function. It computes a hash value for a
given symbol name. When linking C++ programs, it can be expensive
because of C++ mangled long symbols.
This patch reduces the number of call from 2 to 1. Performance
improvements by this patch was larger than I expected. Linking time
of chrome.dll gets almost 5% shorter.
llvm-svn: 231549
Previously we added all undefined symbols found in object files to
the dead strip root. This patch makes the linker to stop doing that.
Undefined symbols would be resolved anyway, so this patch doesn't
change the linker behavior. It should slightly improve performance
but it's really marginal. This is a cleanup.
llvm-svn: 231545
Previously applying 1 million relocations took about 2 seconds on my
Xeon 2.4GHz 8 core workstation. After this patch, it takes about 300
milliseconds. As a result, time to link chrome.dll becomes 23 seconds
to 21 seconds.
llvm-svn: 231454
In the resolver, we maintain a list of undefined symbols, and when we
visit an archive file, we check that file if undefined symbols can be
resolved using files in the archive. The archive file class provides
find() function to lookup a symbol.
Previously, we call find() for each undefined symbols. Archive files
may be visited multiple times if they are in a --start-group and
--end-group. If we visit a file M times and if we have N undefined
symbols, find() is called M*N times. I found that that is one of the
most significant bottlenecks in LLD when linking a large executable.
find() is not a very cheap operation because it looks up a hash table
for a given string. And a string, or a symbol name, can be pretty long
if you are dealing with C++ symbols.
We can eliminate the bottleneck.
Calling find() with the same symbol multiple times is a waste. If a
result of looking up a symbol is "not found", it stays "not found"
forever because the symbol simply doesn't exist in the archive.
Thus, we should call find() only for newly-added undefined symbols.
This optimization makes O(M*N) O(N).
In this patch, all undefined symbols are added to a vector. For each
archive/shared library file, we maintain a start position P. All
symbols [0, P) are already searched. [P, end of the vector) are not
searched yet. For each file, we scan the vector only once.
This patch changes the order in which undefined symbols are looked for.
Previously, we iterated over the result of _symbolTable.undefines().
Now we iterate over the new vector. This is a benign change but caused
differences in output if remaining undefines exist. This is why some
tests are updated.
The performance improvement of this patch seems sometimes significant.
Previously, linking chrome.dll on my workstation (Xeon 2.4GHz 8 cores)
took about 70 seconds. Now it takes (only?) 30 seconds!
http://reviews.llvm.org/D8091
llvm-svn: 231434
_reverseRef is a multimap from atoms to atoms. The map contains
reverse edges of "layout-before" and "group" edges for dead-stripping.
The type of the variable was DenseMap<Atom *, DenseSet<Atom *>>.
This patch changes that to std::unordered_multimap<Atom *, Atom *>.
A DenseMap with a value type of DenseSet was not fast. Inserting 900k
items to the map took about 1.6 seconds on my workstation.
unordered_multimap on the other hand took only 0.6 seconds.
Use of the map also got faster -- originally markLive took 1.3 seconds
in the same test case, and it now took 1.0 seconds. In total we shove
off 1.3 seconds out of 27 seconds in that test case.
llvm-svn: 231432
We maintain a map from symbols to archive files for the archive file
pre-loading. That map is created at the beginning of the resolve()
and is never updated. However, the input file list may be updated by
File::beforeLink(). This is a patch to update the map after beforeLink.
llvm-svn: 231395
I converted them to non-range-based loops in r226883 and r226893
because at that moment File::parse() may have side effects and
may update the vector that the reference returned from
LinkingContext::nodes().
Now File::parse() is free from side effects. We can use range-based
loops again.
llvm-svn: 231321
The last use of layout-after edge for PE/COFF was removed in r231290.
Now layout-after edges do nothing. We can stop adding them to the graph.
No functionality change intended.
llvm-svn: 231301
Merge::mergeByLargestSection is half-baked since it's defined
in terms of section size, there's no way to get the section size
of an atom.
Currently we work around the issue by traversing the layout edges
to both directions and calculate the sum of all atoms reachable.
I wrote that code but I knew it's hacky. It's even not guaranteed
to work. If you add layout edges before the core linking, it
miscalculates a size.
Also it's of course slow. It's basically a linked list traversal.
In this patch I added DefinedAtom::sectionSize so that we can use
that for mergeByLargestSection. I'm not very happy to add a new
field to DefinedAtom base class, but I think it's legitimate since
mergeByLargestSection is defined for section size, and the section
size is currently just missing.
http://reviews.llvm.org/D7966
llvm-svn: 231290
File objects are not really const in the resolver. We set ordinals to
them and call beforeLink hooks. Also, File's member functions marked
as const are not really const. ArchiveFile never returns the same
member file twice, so it remembers files returned before. find() has
side effects.
In order to deal with the inconsistencies, we sprinkled const_casts
and marked member varaibles as mutable.
This patch removes const from there to reflect the reality.
llvm-svn: 231212
std::promise and std::future in old version of libstdc++ are buggy.
I think that's the reason why LLD tests were flaky on Ubuntu 13
buildbots until we disabled file preloading.
In this patch, I implemented very simple future and used that in
FileArchive. Compared to std::promise and std::future, it lacks
many features, but should serve our purpose.
http://reviews.llvm.org/D8025
llvm-svn: 231153
Yet another chapter in the story. We're getting there, finally.
Note for the future: the tests for relocation have a lot of duplication
and probably can be unified in a single file. Let's reevaluate this once
the support will be complete (hopefully, soon).
llvm-svn: 231057
"virtual" was present at a wrong place. FileArchive is a subclass of
ArchiveLibraryFile, and a FileArchive can be deleted through a
pointer of ArchiveLibraryFile. We want to make the destructor of the
base class virtual.
llvm-svn: 231033
This reverts commit r228955. Previously files appear in a .drectve
section are parsed synchronously to avoid threading issues. I believe
it's now safe to do that asynchronously.
llvm-svn: 230905
This reverts commit r230086. I added a lock to guard FileCOFF::doParse(),
which killed parallel file parsing. Now the buildbots got back to green,
I believe the threading issue was resolved, so it's time to remove the
guard to see if it works with the buildbots.
llvm-svn: 230886
Previously we didn't call the hook on a file in an archive, which
let the PE/COFF port fail to link files in archives. It was a
simple mistake. Added a call to the hook and also added a test to
catch that error.
const_cast is an unfortunate hack. Files in the resolver are usually
const, but they are not actually const objects, since they are
mutated if either a file is taken from an archive (an archive file
does never return the same file twice) or the beforeLink hook is
called. Maybe we should just remove const from there -- because they
are not const.
llvm-svn: 230808
In doParse, we shouldn't do anything that has side effects. That function may be
called speculatively and possibly in parallel.
We called WinLinkDriver::parse from doParse to parse a command line in a .drectve
section. The parse function updates a linking context object, so it has many side
effects. It was not safe to call that function from doParse. beforeLink is a
function for a File object to do something that has side effects. Moving a call
of WinLinkDriver::parse to there.
llvm-svn: 230791
If no initial live symbols are set up, and deadStrip() == true,
the Resolver ends up reclaiming all the symbols that aren't absolute. This is wrong.
This patch fixes the issue by setting entrySymbolName() as live, and this allows
us to self-host lld when --gc-sections is enabled. There are still quite a few problems
with --gc-sections (test failures), so the option can't be enabled by default.
Differential Revision: D7926
Reviewed by: ruiu, shankarke
llvm-svn: 230737
It is observed that the function throws std::future_error on a few buildbots.
That cannot be easily reproducible on local machines. Kill the feature
temporarily to see if this is going to fix the buildbot issue.
llvm-svn: 230735
This reverts commit r230732.
sectionSize() in lib/Core/SymbolTable.cpp still depends on the layout-
after edges, so we couldn't remove them yet.
llvm-svn: 230734
Shankar Easwaran