This flag is implemented similarly to --reproduce in the ELF linker.
This patch implements /linkrepro by moving the cpio writer and associated
utility functions to lldCore, and using that implementation in both linkers.
One COFF-specific detail is that we store the object file from which the
resource files were created in our reproducer, rather than the resource
files themselves. This allows the reproducer to be used on non-Windows
systems for example.
Differential Revision: https://reviews.llvm.org/D22418
llvm-svn: 276719
make_dynamic_error_code was used to create a std::error_code with
a std::string message. Now that we are migrating to llvm::Error,
there are no calls to these make_dynamic_error_code methods.
There is one single call to make_dynamic_error_code remaining, the one
inside GenericError::convertToErrorCode(). That method is only called
from File::doParse() which should be a temporary situation. We need
to work out how to deal with File::parse() caching the error result from
doParse(). Caching errors isn't supported in the new scheme, and probably
isn't needed here, but we need to work that out.
Once thats done, dynamic error and all utilities around it can be deleted.
llvm-svn: 264982
Adds a GenericError class to lld/Core which can carry a string. This is
analygous to the dynamic_error we currently use in lld/Core.
Use this GenericError instead of make_dynamic_error_code. Also, provide
an implemention of GenericError::convertToErrorCode which for now converts
it in to the dynamic_error_code we used to have. This will go away once
all the APIs are converted.
llvm-svn: 264910
searchArchivesToOverrideTentativeDefinitions and
searchSharedLibrariesToOverrideTentativeDefinitions are always false.
For the dead flags, we have a fairly large amount of code which is
never be executed.
http://reviews.llvm.org/D17791
llvm-svn: 264653
This is a re-commit of r264022 with a fix for MSVC. The issue there was
that the code was running DefinedAtom::~Atom() for some value and instead
needed to cast to Atom before running ~Atom. Original commit message follows.
Currently each File contains an BumpPtrAllocator in which Atom's are
allocated. Some Atom's contain data structures like std::vector which
leak as we don't run ~Atom when they are BumpPtrAllocate'd.
Now each File actually owns its Atom's using an OwningAtomPtr. This
is analygous to std::unique_ptr and may be replaced by it if possible.
An Atom can therefore only be owned by a single File, so the Resolver now
moves them from one File to another. The MachOLinkingContext owns the File's
and so clears all the Atom's in ~MachOLinkingContext, then delete's all the
File's. This makes sure all Atom's have been destructed before any of the
BumpPtrAllocator's in which they run have gone away.
Should hopefully fix the remaining leaks. Will keep an eye on the bots to
make sure.
llvm-svn: 264067
This reverts commit r264022.
This breaks the Window's bots which don't like that i'm calling ~Atom when
the this pointer is a sublcass of Atom.
Reverting for now until I try find a better fix. I tried using std::unique_ptr with
a custom deleter as a quick fix, but it didn't work well in the YAML parser.
llvm-svn: 264023
Currently each File contains an BumpPtrAllocator in which Atom's are
allocated. Some Atom's contain data structures like std::vector which
leak as we don't run ~Atom when they are BumpPtrAllocate'd.
Now each File actually owns its Atom's using an OwningAtomPtr. This
is analygous to std::unique_ptr and may be replaced by it if possible.
An Atom can therefore only be owned by a single File, so the Resolver now
moves them from one File to another. The MachOLinkingContext owns the File's
and so clears all the Atom's in ~MachOLinkingContext, then delete's all the
File's. This makes sure all Atom's have been destructed before any of the
BumpPtrAllocator's in which they run have gone away.
Should hopefully fix the remaining leaks. Will keep an eye on the bots to
make sure.
llvm-svn: 264022
The non lazy atoms generated in the stubs pass use an image cache to
hold all of the pointers. On arm archs, this is the __got section,
but on x86 archs it should be __nl_symbol_ptr.
rdar://problem/24572729
llvm-svn: 260271
The magic file which contained these symbols inherited from archive
which meant that the resolver didn't add the required atoms as archive
members only get added when referenced. Instead we now inherit from
SimpleFile which always links in the atoms needed.
The second issue was in the handling of these symbols when we emit
the MachO. The mach header symbol needs to be in the atom list as
it gets an offset (0), and being in the atom list makes sure it is
emitted to the symbol table. DSO handles are not emitted to the
symbol table.
rdar://problem/24450654
llvm-svn: 259574
An upcoming patch will use this to create lists of ObjC methods.
Adding it now to reduce the amount of code in that patch.
Test cases will follow in the other patch too.
llvm-svn: 259440
This pass currently emits an objc image info section if one is required.
This section contains the aggregated version and flags for all of the input
files.
llvm-svn: 258197
This is to enable isa<> support for any files which need it.
It will be used in an upcoming patch to differentiate MachOFile from other implicitly generated files.
Reviewed by Lang Hames.
Differential Revision: http://reviews.llvm.org/D16103
llvm-svn: 257830
This is called from the resolver on each file we decide we actually want to use.
Future commits will make use of this to extract useful information from the files and do
error checking against the context. For example, ensure that files are the same arch as
each other.
Reviewed by Lang Hames.
Differential Revision: http://reviews.llvm.org/D16093
llvm-svn: 257814
In a UI such as XCode, it can group the headers for a library with that library.
This is done in the CMakeLists.txt for the library itself by setting the path(s)
as ADDITIONAL_HEADER_DIRS.
LLVM already does this for all of its libraries, so just adding this to lld to
make things easier. Should be NFC.
llvm-svn: 257002
In a UI such as XCode, LLVM source files are in 'libraries' while clang
files are in 'clang libraries'.
This change moves the lld source to 'lld libraries' to make code browsing easier.
It should be NFC as the build itself is still the same, just the structure in a
UI differs.
llvm-svn: 257001
We had some DEBUG prints these passes, but add more so that its clear where we are dumping
things, and what state we are in when we do so.
I'll be adding more and more DEBUG printing to try make it easier to observe whats going on
without having to attach a debugger.
llvm-svn: 255805
Add PT_PHDR segment depending on its availability in linker script's
PHDRS command, fallback if no linker script is given.
Handle FILEHDR, PHDRS and FLAGS attributes of program header.
Differential Revision: http://reviews.llvm.org/D11589
llvm-svn: 244743
Add method to query segments for specified output section name.
Return error if the section is assigned to unknown segment.
Check matching of sections to segments during layout on the subject of correctness.
NOTE: no actual functionality of using custom segments is implemented.
Differential Revision: http://reviews.llvm.org/D10359
llvm-svn: 239719
This patch provides generation of .ARM.exidx & .ARM.extab sections which are
used for unwinding. The patch adds new content type typeARMExidx for atoms from
.ARM.exidx section and integration of atoms with such type to the ELF
ReaderWriter. exidx.test has been added with checking of contents of .ARM.exidx
section and .ARM.extab section.
Differential Revision: http://reviews.llvm.org/D9324
llvm-svn: 236873
loadFile could load mulitple files just because yaml has a feature for
putting multiple documents in one file.
Designing a linker around what yaml can do seems like a bad idea to
me. This patch changes it to read a single file.
There are further improvements to be done to the api and they
will follow shortly.
llvm-svn: 235724
The function took either StringRef or Twine. Since string literals are
ambiguous when resolving the overloading, many code calls used this
function with explicit type conversion. That led awkward code like
make_dynamic_error_code(Twine("Error occurred")).
This patch adds a function definition for string literals, so that
you can directly call the function with literals.
llvm-svn: 234841
I believe this patch eliminates all remaining uses of _context
or _linkingContext variable names. Consistent naming improves
readability.
llvm-svn: 234645
The Native file format was designed to be the fastest on-memory or
on-disk file format for object files. The problem is that no one
is working on that. No LLVM tools can produce object files in
the Native, thus the feature of supporting the format is useless
in the linker.
This patch removes the Native file support. We can add it back
if we really want it in future.
llvm-svn: 234641
It's not our business to resolve those undefined symbols.
We just trust the linker will load the library and its dependencies
correctly, which is actually what happens, modulo bugs in the dynamic
linker itself.
PR: 23035
Differential Revision: http://reviews.llvm.org/D8886
llvm-svn: 234378
canParse took three parameters -- file magic, filename extension and
memory buffer. All but YAMLReader ignored the second parameter.
This patch removes the parameter.
llvm-svn: 234080
The expression evaluation is needed when interpreting linker scripts, in order
to calculate the value for new symbols or to determine a new position to load
sections in memory. This commit extends Expression nodes from the linker script
AST with evaluation functions, and also contains a unit test.
http://reviews.llvm.org/D8156
llvm-svn: 231707
All readers except PE/COFF reader create layout-after edges to preserve
the original symbol order. PE/COFF uses layout-before edges as primary
edges for no reason.
This patch makes PE/COFF reader to create layout-after edges.
Resolver is updated to recognize reverse edges of layout-after edges
in the garbage collection pass.
Now we can retire layout-before edges. I don't do that in this patch
because if I do, I would have updated many tests to replace all
occurrrences of "layout-before" with "layout-after". So that's a TODO.
llvm-svn: 231615
Atoms with fallback atoms are never be added to the symbol table.
However, we added such atoms to _undefines array. We had to call
isCoalescedAway to identify and skip them. We should just stop
adding them in the first place.
This seems to make the linker ~1% faster in my test case.
llvm-svn: 231552
If an undefined symbol is added to the symbol table by the previous
call of SymbolTable::add, SymbolTable::isDefined will always return
false for the same symbol.
llvm-svn: 231551
This is yet another optimization patch. Previously we called
SymbolTable::isDefined() and SymbolTable::findByName() from a very
frequently executed function. Because isDefined calls findByName,
findByName is called twice on each iteration.
findByName is not a cheap function. It computes a hash value for a
given symbol name. When linking C++ programs, it can be expensive
because of C++ mangled long symbols.
This patch reduces the number of call from 2 to 1. Performance
improvements by this patch was larger than I expected. Linking time
of chrome.dll gets almost 5% shorter.
llvm-svn: 231549
In the resolver, we maintain a list of undefined symbols, and when we
visit an archive file, we check that file if undefined symbols can be
resolved using files in the archive. The archive file class provides
find() function to lookup a symbol.
Previously, we call find() for each undefined symbols. Archive files
may be visited multiple times if they are in a --start-group and
--end-group. If we visit a file M times and if we have N undefined
symbols, find() is called M*N times. I found that that is one of the
most significant bottlenecks in LLD when linking a large executable.
find() is not a very cheap operation because it looks up a hash table
for a given string. And a string, or a symbol name, can be pretty long
if you are dealing with C++ symbols.
We can eliminate the bottleneck.
Calling find() with the same symbol multiple times is a waste. If a
result of looking up a symbol is "not found", it stays "not found"
forever because the symbol simply doesn't exist in the archive.
Thus, we should call find() only for newly-added undefined symbols.
This optimization makes O(M*N) O(N).
In this patch, all undefined symbols are added to a vector. For each
archive/shared library file, we maintain a start position P. All
symbols [0, P) are already searched. [P, end of the vector) are not
searched yet. For each file, we scan the vector only once.
This patch changes the order in which undefined symbols are looked for.
Previously, we iterated over the result of _symbolTable.undefines().
Now we iterate over the new vector. This is a benign change but caused
differences in output if remaining undefines exist. This is why some
tests are updated.
The performance improvement of this patch seems sometimes significant.
Previously, linking chrome.dll on my workstation (Xeon 2.4GHz 8 cores)
took about 70 seconds. Now it takes (only?) 30 seconds!
http://reviews.llvm.org/D8091
llvm-svn: 231434
_reverseRef is a multimap from atoms to atoms. The map contains
reverse edges of "layout-before" and "group" edges for dead-stripping.
The type of the variable was DenseMap<Atom *, DenseSet<Atom *>>.
This patch changes that to std::unordered_multimap<Atom *, Atom *>.
A DenseMap with a value type of DenseSet was not fast. Inserting 900k
items to the map took about 1.6 seconds on my workstation.
unordered_multimap on the other hand took only 0.6 seconds.
Use of the map also got faster -- originally markLive took 1.3 seconds
in the same test case, and it now took 1.0 seconds. In total we shove
off 1.3 seconds out of 27 seconds in that test case.
llvm-svn: 231432
We maintain a map from symbols to archive files for the archive file
pre-loading. That map is created at the beginning of the resolve()
and is never updated. However, the input file list may be updated by
File::beforeLink(). This is a patch to update the map after beforeLink.
llvm-svn: 231395
Merge::mergeByLargestSection is half-baked since it's defined
in terms of section size, there's no way to get the section size
of an atom.
Currently we work around the issue by traversing the layout edges
to both directions and calculate the sum of all atoms reachable.
I wrote that code but I knew it's hacky. It's even not guaranteed
to work. If you add layout edges before the core linking, it
miscalculates a size.
Also it's of course slow. It's basically a linked list traversal.
In this patch I added DefinedAtom::sectionSize so that we can use
that for mergeByLargestSection. I'm not very happy to add a new
field to DefinedAtom base class, but I think it's legitimate since
mergeByLargestSection is defined for section size, and the section
size is currently just missing.
http://reviews.llvm.org/D7966
llvm-svn: 231290
File objects are not really const in the resolver. We set ordinals to
them and call beforeLink hooks. Also, File's member functions marked
as const are not really const. ArchiveFile never returns the same
member file twice, so it remembers files returned before. find() has
side effects.
In order to deal with the inconsistencies, we sprinkled const_casts
and marked member varaibles as mutable.
This patch removes const from there to reflect the reality.
llvm-svn: 231212
std::promise and std::future in old version of libstdc++ are buggy.
I think that's the reason why LLD tests were flaky on Ubuntu 13
buildbots until we disabled file preloading.
In this patch, I implemented very simple future and used that in
FileArchive. Compared to std::promise and std::future, it lacks
many features, but should serve our purpose.
http://reviews.llvm.org/D8025
llvm-svn: 231153
Previously we didn't call the hook on a file in an archive, which
let the PE/COFF port fail to link files in archives. It was a
simple mistake. Added a call to the hook and also added a test to
catch that error.
const_cast is an unfortunate hack. Files in the resolver are usually
const, but they are not actually const objects, since they are
mutated if either a file is taken from an archive (an archive file
does never return the same file twice) or the beforeLink hook is
called. Maybe we should just remove const from there -- because they
are not const.
llvm-svn: 230808
It is observed that the function throws std::future_error on a few buildbots.
That cannot be easily reproducible on local machines. Kill the feature
temporarily to see if this is going to fix the buildbot issue.
llvm-svn: 230735
This fixes a linker crash (found out while testing --gc-sections,
testcase provided by Rafael Avila de Espindola).
While this behaviour was found while testing ELF, it' not necessarily
ELF specific and this change is (apparently) harmless on all the
other drivers.
Differential Revision: D7823
Reviewed by: ruiu
llvm-svn: 230614
Use the environment variable "LLD_RUN_ROUNDTRIP_TEST" in the test that you want
to disable, as
RUN: env LLD_RUN_ROUNDTRIP_TEST= <run>
This was a patch that I made, but I find this a better way to accomplish what we
want to do.
llvm-svn: 228376
This caused some tests to fail on FreeBSD, and Mac OS X.
Some std::sort() implementations will check for strict-weak-ordering
by comparing with the same element, or will compare an element to
itself for 1-element sequence. Take care of this case. Thanks to
chandlerc for explaning that to me.
Reviewed by: ruiu
llvm-svn: 227709
Previously we applied the LayoutPass to order atoms and then
apply elf::ArrayOrderPass to sort them again. The first pass is
basically supposed to sort atoms in the normal fashion (which
is to sort symbols in the same order as the input files).
The second pass sorts atoms in {init,fini}_array.<priority> by
priority.
The problem is that the LayoutPass is overkill. It analyzes
references between atoms to make a decision how to sort them.
It's slow, hard to understand, and above all, it doesn't seem
that we need its feature for ELF in the first place.
This patch remove the LayoutPass from ELF pass list. Now all
reordering is done in elf::OrderPass. That pass sorts atoms by
{init,fini}_array, and if they are not in the special section,
they are ordered as the same order as they appear in the command
line. The new code is far easier to understand, faster, and
still able to create valid executables.
Unlike the previous layout pass, elf::OrderPass doesn't count
any attributes of an atom (e.g. permissions) except its
position. It's OK because the writer takes care of them if we
have to.
This patch changes the order of final output, although that's
benign. Tests are updated.
http://reviews.llvm.org/D7278
llvm-svn: 227666
That kind of reference was used only in ELFFile, and the use of
that reference there didn't seem to make sense. All test still
pass (after adjusting symbol names) without that code. LLD is
still be able to link LLD and Clang. Looks like we just don't
need this.
http://reviews.llvm.org/D7189
llvm-svn: 227259
Before this patch there was a cyclic dependency between lldCore and
lldReaderWriter. Only lldConfig could be built as a shared library.
* Moved Reader and Writer base classes into lldCore.
* The following shared libraries can now be built:
lldCore
lldYAML
lldNative
lldPasses
lldReaderWriter
Differential Revision: http://reviews.llvm.org/D7105
From: Greg Fitzgerald <garious@gmail.com>
llvm-svn: 226732
I believe the original code is valid, but on Windows it failed with an
assertion error saying "Expression: vector iterator is not decrementable."
Don't use rbegin and rend to workaround that error.
llvm-svn: 226706
We used to manage the state whether we are in a group or not
using a counter. The counter is incremented by one if we jump from
end-group to start-group, and decremented by one if we don't.
The counter was assumed to be either zero or one, but obviously it
could be negative (if there's a group which is not repeated at all).
This is a fix for that issue.
llvm-svn: 226632
LLD parses archive file index table only at first. When it finds a symbol
it is looking for is defined in a member file in an archive file, it actually
reads the member from the archive file. That's done in the core linker.
That's a single-thread process since the core linker is single threaded.
If your command line contains a few object files and a lot of archive files
(which is quite often the case), LLD hardly utilizes hardware parallelism.
This patch improves parallelism by speculatively instantiating archive
file members. At the beginning of the core linking, we first create a map
containing all symbols defined in all members, and each time we find a
new undefined symbol, we instantiate a member file containing the
symbol (if such file exists). File instantiation is side effect free, so this
should not affect correctness.
This is a quick benchmark result. Time to link self-link LLD executable:
Linux 9.78s -> 8.50s (0.86x)
Windows 6.18s -> 4.51s (0.73x)
http://reviews.llvm.org/D7015
llvm-svn: 226336
This patch makes File::parse() multi-thread safe. If one thread is running
File::parse(), other threads will block if they try to call the same method.
File::parse() is idempotent, so you can safely call multiple times.
With this change, we don't have to wait for all worker threads to finish
in Driver::link(). Previously, Driver::link() calls TaskGroup::sync() to
wait for all threads running File::parse(). This was not ideal because
we couldn't start the resolver until we parse all files.
This patch increase parallelism by making Driver::link() to not wait for
worker threads. The resolver calls parse() to make sure that the file
being read has been parsed, and then uses the file. In this approach,
the resolver can run with the parser threads in parallel.
http://reviews.llvm.org/D6994
llvm-svn: 226281
InputElement was named that because it's an element of an InputGraph.
It's losing the origin because the InputGraph is now being removed.
InputElement's subclass is FileNode, that naming inconsistency needed
to be fixed.
llvm-svn: 226147
Davide Italiano