This caused some tests to fail on FreeBSD, and Mac OS X.
Some std::sort() implementations will check for strict-weak-ordering
by comparing with the same element, or will compare an element to
itself for 1-element sequence. Take care of this case. Thanks to
chandlerc for explaning that to me.
Reviewed by: ruiu
llvm-svn: 227709
Previously we applied the LayoutPass to order atoms and then
apply elf::ArrayOrderPass to sort them again. The first pass is
basically supposed to sort atoms in the normal fashion (which
is to sort symbols in the same order as the input files).
The second pass sorts atoms in {init,fini}_array.<priority> by
priority.
The problem is that the LayoutPass is overkill. It analyzes
references between atoms to make a decision how to sort them.
It's slow, hard to understand, and above all, it doesn't seem
that we need its feature for ELF in the first place.
This patch remove the LayoutPass from ELF pass list. Now all
reordering is done in elf::OrderPass. That pass sorts atoms by
{init,fini}_array, and if they are not in the special section,
they are ordered as the same order as they appear in the command
line. The new code is far easier to understand, faster, and
still able to create valid executables.
Unlike the previous layout pass, elf::OrderPass doesn't count
any attributes of an atom (e.g. permissions) except its
position. It's OK because the writer takes care of them if we
have to.
This patch changes the order of final output, although that's
benign. Tests are updated.
http://reviews.llvm.org/D7278
llvm-svn: 227666
That kind of reference was used only in ELFFile, and the use of
that reference there didn't seem to make sense. All test still
pass (after adjusting symbol names) without that code. LLD is
still be able to link LLD and Clang. Looks like we just don't
need this.
http://reviews.llvm.org/D7189
llvm-svn: 227259
Before this patch there was a cyclic dependency between lldCore and
lldReaderWriter. Only lldConfig could be built as a shared library.
* Moved Reader and Writer base classes into lldCore.
* The following shared libraries can now be built:
lldCore
lldYAML
lldNative
lldPasses
lldReaderWriter
Differential Revision: http://reviews.llvm.org/D7105
From: Greg Fitzgerald <garious@gmail.com>
llvm-svn: 226732
I believe the original code is valid, but on Windows it failed with an
assertion error saying "Expression: vector iterator is not decrementable."
Don't use rbegin and rend to workaround that error.
llvm-svn: 226706
We used to manage the state whether we are in a group or not
using a counter. The counter is incremented by one if we jump from
end-group to start-group, and decremented by one if we don't.
The counter was assumed to be either zero or one, but obviously it
could be negative (if there's a group which is not repeated at all).
This is a fix for that issue.
llvm-svn: 226632
LLD parses archive file index table only at first. When it finds a symbol
it is looking for is defined in a member file in an archive file, it actually
reads the member from the archive file. That's done in the core linker.
That's a single-thread process since the core linker is single threaded.
If your command line contains a few object files and a lot of archive files
(which is quite often the case), LLD hardly utilizes hardware parallelism.
This patch improves parallelism by speculatively instantiating archive
file members. At the beginning of the core linking, we first create a map
containing all symbols defined in all members, and each time we find a
new undefined symbol, we instantiate a member file containing the
symbol (if such file exists). File instantiation is side effect free, so this
should not affect correctness.
This is a quick benchmark result. Time to link self-link LLD executable:
Linux 9.78s -> 8.50s (0.86x)
Windows 6.18s -> 4.51s (0.73x)
http://reviews.llvm.org/D7015
llvm-svn: 226336
This patch makes File::parse() multi-thread safe. If one thread is running
File::parse(), other threads will block if they try to call the same method.
File::parse() is idempotent, so you can safely call multiple times.
With this change, we don't have to wait for all worker threads to finish
in Driver::link(). Previously, Driver::link() calls TaskGroup::sync() to
wait for all threads running File::parse(). This was not ideal because
we couldn't start the resolver until we parse all files.
This patch increase parallelism by making Driver::link() to not wait for
worker threads. The resolver calls parse() to make sure that the file
being read has been parsed, and then uses the file. In this approach,
the resolver can run with the parser threads in parallel.
http://reviews.llvm.org/D6994
llvm-svn: 226281
InputElement was named that because it's an element of an InputGraph.
It's losing the origin because the InputGraph is now being removed.
InputElement's subclass is FileNode, that naming inconsistency needed
to be fixed.
llvm-svn: 226147
These changes depended on r225674 and had been rolled back in r225859.
Because r225674 has been re-submitted, it's safe to re-submit them.
llvm-svn: 226132
r225764 broke a basic functionality on Mac OS. This change reverts
r225764, r225766, r225767, r225769, r225814, r225816, r225829, and r225832.
llvm-svn: 225859
getNextFile used to have a complex logic to determine which file
should be processed by the Resolver on next iteration.
Now, it is just a sequential accessor to the internal array and
provides no sensible feature.
This patch also removes InputGraph::getGroupSize and InputGraph::
skipGroup to simplify the code.
llvm-svn: 225832
PECOFF was the only user of the API, and the reason why we created
the API is because, although the driver creates a list of input files,
it has no knowledge on what files are being created. It was because
everything was hidden behind the InputGraph abstraction.
Now the driver knows what that's doing. We no longer need this
indirection to get the file list being processed.
llvm-svn: 225767
This is a part of InputGraph cleanup to represent input files as a flat
list of Files (and some meta-nodes for group etc.)
We cannot achieve that goal in one gigantic patch, so I split the task
into small steps as shown below.
(Recap the progress so far: Currently InputGraph contains a list of
InputElements. Each InputElement contain one File (that used to have
multiple Files, but I eliminated that use case in r223867). Files are
currently instantiated in Driver::link(), but I already made a change
to separate file parsing from object instantiation (r224102), so we
can safely instantiate Files when we need them, instead of wrapping
a file with the wrapper class (FileNode class). InputGraph used to
act like a generator class by interpreting groups by itself, but it's
now just a container of a list of InputElements (r223867).)
1. Instantiate Files in the driver and wrap them with WrapperNode.
WrapperNode is a temporary class that allows us to instantiate Files
in the driver while keep using the current InputGraph data structure.
This patch demonstrates how this step 1 looks like, using Core driver
as an example.
2. Do the same thing for the other drivers.
When step 2 is done, an InputGraph consists of GroupEnd objects or
WrapperNodes each of which contains one File. Other types of
FileNode subclasses are removed.
3. Replace InputGraph with std::vector<std::unique_ptr<InputElement>>.
InputGraph is already just a container of list of InputElements,
so this step removes that needless class.
4. Remove WrapperNode.
We need some code cleanup between each step, because many classes
do a bit odd things (e.g. InputGraph::getGroupSize()). I'll straight
things up as I need to.
llvm-svn: 225313
ReaderErrorCategory was used only at one place. We now have a
DynamicErrorCategory for this kind of one-time error, so use it.
The calling function doesn't really care the type of an error, so
ReaderErrorCategory was actually dead code.
llvm-svn: 224245
These member functions returns either no_more_files error or a File object.
We could simply return a nullptr instead of a no_more_files.
This function will be removed soon as a part of InputGraph cleanup.
I had to do that step by step.
llvm-svn: 224208
The documentation of parseFile() said that "the resulting File
object may take ownership of the MemoryBuffer." So, whether or not
the ownership of a MemoryBuffer would be taken was not clear.
A FileNode (a subclass of InputElement, which is being deprecated)
keeps the ownership if a File doesn't take it.
This patch makes File always take the ownership of a buffer.
Buffers lifespan is not always the same as File instances.
Files are able to deallocate buffers after parsing the contents.
llvm-svn: 224113
This reverts commit r223330 because it broke Darwin and ELF
linkers in a way that we couldn't have caught with the existing
test cases.
llvm-svn: 223373
The aim of this patch is to reduce the excessive abstraction from
the InputGraph. We found that even a simple thing, such as sorting
input files (Mach-O) or adding a new file to the input file list
(PE/COFF), is nearly impossible with the InputGraph abstraction,
because it hides too much information behind it. As a result,
we invented complex interactions between components (e.g.
notifyProgress() mechanism) and tricky code to work around that
limitation. There were many occasions that we needed to write
awkward code.
This patch is a first step to make it cleaner. As a first step,
this removes Group class from the InputGraph. The grouping feature
is now directly handled by the Resolver. notifyProgress is removed
since we no longer need that. I could have cleaned it up even more,
but in order to keep the patch minimum, I focused on Group.
SimpleFileNode class, a container of File objects, is now limited
to have only one File. We shold have done this earlier.
We used to allow putting multiple File objects to FileNode.
Although SimpleFileNode usually has only one file, the Driver class
actually used that capability. I modified the Driver class a bit,
so that one FileNode is created for each input File.
We should now probably remove SimpleFileNode and directly store
File objects to the InputGraph in some way, because a container
that can contain only one object is useless. This is a TODO.
Mach-O input files are now sorted before they are passe to the
Resolver. DarwinInputGraph class is no longer needed, so removed.
PECOFF still has hacky code to add a new file to the input file list.
This will be cleaned up in another patch.
llvm-svn: 223330
RoundTripPasses should always be called in DEBUG mode if the environment
variable "LLD_RUN_ROUNDTRIP_TEST" is set.
Flavors should not be able to override this behavior.
llvm-svn: 223073
This would allow other flavor specific contexts to override the default value,
if they want to optionally run the round trip passes.
There is some information lost like the original file owner of the atom with
RoundTripPasses. The Gnu flavor needs this information inorder to implement
LinkerScript matching and for other diagnostic outputs such as Map files.
The flag also can be used to record information in the Atom if the information
to the Writer needs to be conveyed through References too.
llvm-svn: 222983
On darwin in final linked images, the __TEXT segment covers that start of the
file. That means in memory a process can see the mach_header (and load commands)
for every loaded image in a process. There are APIs that take and return the
mach_header addresses as a way to specify a particular loaded image.
For completeness, any code can get the address of the mach_header of the image
it is in by using &__dso_handle. In addition there are mach-o type specific
symbols like __mh_execute_header.
The linker needs to supply a definition for any of these symbols if used. But
the address the symbol it resolves to is not in any section. Instead it is the
address of the start of the __TEXT segment.
I needed to make a small change to SimpleFileNode to not override
resetNextIndex() because the Driver creates a SimpleFileNode to hold the
internal/implicit files that the context/writer can create. For some reason
SimpleFileNode overrode resetNextIndex() to do nothing instead of reseting
the index (which mach-o needs if the internal file is an archive).
llvm-svn: 221822
Darwin uses two-level-namespace lookup for symbols which means the static
linker records where each symbol must be found at runtime. Thus defining a
symbol in a dylib loaded earlier will not effect where symbols needed by
later dylibs will be found. Instead overriding is done through a section
of type S_INTERPOSING which contains tuples of <interposer, interposee>.
llvm-svn: 221421
The job of the CompactUnwind pass is to turn __compact_unwind data (and
__eh_frame) into the compressed final form in __unwind_info. After it's done,
the original atoms are no longer relevant and should be deleted (they cause
problems during actual execution, quite apart from the fact that they're not
needed).
llvm-svn: 221301
The darwin linker operates differently than the gnu linker with respect to
libraries. The darwin linker first links in all object files from the command
line, then to resolve any remaining undefines, it repeatedly iterates over
libraries on the command line until either all undefines are resolved or no
undefines were resolved in the last pass.
When Shankar made the InputGraph model, the plan for darwin was for the darwin
driver to place all libraries in a group at the end of the InputGraph. Thus
making the darwin model a subset of the gnu model. But it turns out that does
not work because the driver cannot tell if a file is an object or library until
it has been loaded, which happens later.
This solution is to subclass InputGraph for darwin and just iterate the graph
the way darwin linker needs.
llvm-svn: 220330
Previously, we would not check the target machine type and the module (object)
machine type. Add a check to ensure that we do not attempt to use an object
file with a different target architecture.
This change identified a couple of tests which were incorrectly mixing up
architecture types, using x86 input for a x64 target. Adjust the tests
appropriately. The renaming of the input and the architectures covers the
changes to the existing tests.
One significant change to the existing tests is that the newly added test input
for x64 uses the correct user label prefix for X64.
llvm-svn: 219093
Rather than a series of cascading ifs, use a switch statement to convert the
error code to a string. This has the benefit of allowing the compiler to inform
us if we ever add a new error code but fail to update the string representation.
Add in stringified versions for a couple of missing InputGraphErrors.
llvm-svn: 219089
The mergeByContent attribute on DefinedAtoms triggers the symbol table to
coalesce atoms with the exact same content. The problem is that atoms can also
have a required custom section. The coalescing should never change the custom
section of an atom.
The fix is to only consider to atoms to have the same content if their
sectionChoice() and customSectionName() attributes match.
llvm-svn: 218893
No functionality change. This removes a down-cast from LinkingContext to
MachOLinkingContext.
Also, remove const from LinkingContext::createImplicitFiles() to remove
the need for another const cast. Seems reasonable for createImplicitFiles()
to need to modify the context (MachOLinkingContext does).
llvm-svn: 218796
The darwin linker has the -demangle option which directs it to demangle C++
(and soon Swift) mangled symbol names. Long term we need some Diagnostics object
for formatting errors and warnings. But for now we have the Core linker just
writing messages to llvm::errs(). So, to enable demangling, I changed the
Resolver to call a LinkingContext method on the symbol name.
To make this more interesting, the demangling code is done via __cxa_demangle()
which is part of the C++ ABI, which is only supported on some platforms, so I
had to conditionalize the code with the config generated HAVE_CXXABI_H.
llvm-svn: 218718
This is a minimally useful pass to construct the __unwind_info section in a
final object from the various __compact_unwind inputs. Currently it doesn't
produce any compressed pages, only works for x86_64 and will fail if any
function ends up without __compact_unwind.
rdar://problem/18208653
llvm-svn: 218703
defined in a shared library.
Now LLD does not export a strong defined symbol if it coalesces away a
weak symbol defined in a shared library. This bug affects all ELF
architectures and leads to segfault:
% cat foo.c
extern int __attribute__((weak)) flag;
int foo() { return flag; }
% cat main.c
int flag = 1;
int foo();
int main() { return foo() == 1 ? 0 : -1; }
% clang -c -fPIC foo.c main.c
% lld -flavor gnu -target x86_64 -shared -o libfoo.so ... foo.o
% lld -flavor gnu -target x86_64 -o a.out ... main.o libfoo.so
% ./a.out
Segmentation fault
The problem is caused by the fact that we lose all information about
coalesced symbols after the `Resolver::resolve()` method is finished.
The patch solves the problem by overriding the
`LinkingContext::notifySymbolTableCoalesce()` method and saving names
of coalesced symbols. Later in the `buildDynamicSymbolTable()` routine
we use this information to export these symbols.
llvm-svn: 217363
This is the one interesting aspect from:
http://reviews.llvm.org/D4965
These hooks are useful for flavor specific processing, such as recording that
a DefinedAtom replaced a weak SharedLibraryAtom.
llvm-svn: 216122
insertElementAt(x, END) does the identical thing as addInputElement(x),
so the only reasonable use of insertElementAt is to call it with the
other possible argument, BEGIN. That means the second parameter of the
function is just redundant. This patch is to remove the second
parameter and rename the function accordingly.
llvm-svn: 213821
COFF supports a feature similar to ELF's section groups. This
patch implements it.
In ELF, section groups are identified by their names, and they are
treated somewhat differently from regular symbols. In COFF, the
feature is realized in a more straightforward way. A section can
have an annotation saying "if Nth section is linked, link this
section too."
I added a new reference type, kindAssociate. If a target atom is
coalesced away, the referring atom is removed by Resolver, so that
they are treated as a group.
Differential Revision: http://reviews.llvm.org/D4028
llvm-svn: 211106
This code was never being used and any use of it would look fairly strange.
For example, it would try to map a NativeReaderError::file_malformed to
std::errc::invalid_argument.
llvm-svn: 210913
isCoalescedAway(x) is faster than replacement(x) != x as the former
does not follow the replacement atom chain. Also it's easier to use.
llvm-svn: 210242
This provides support for the autoconfing & make build style.
The format, style and implementation follows that used within the llvm and clang projects.
TODO: implement out-of-source documentation builds.
llvm-svn: 210177
Previously section groups are doubly linked to their children.
That is, an atom representing a group has group-child references
to its group contents, and content atoms also have group-parent
references to the group atom. That relationship was invariant;
if X has a group-child edge to Y, Y must have a group-parent
edge to X.
However we were not using group-parent references at all. The
resolver only needs group-child edges.
This patch simplifies the section group by removing the unused
reverse edge. No functionality change intended.
Differential Revision: http://reviews.llvm.org/D3945
llvm-svn: 210066
Layout-before edges are no longer used for layout, but they are
still there for dead-stripping. If we would just remove them
from code, LLD would wrongly remove live atoms that were
referenced by layout-befores.
This patch fixes the issue. Before dead-stripping, it scans all
atoms to construct a reverse map for layout-after edges. Dead-
stripping pass uses the map to traverse the graph.
Differential Revision: http://reviews.llvm.org/D3986
llvm-svn: 210057
Reference::target() never returns a nullptr, so NULL check
is not needed and is more harmful than doing nothing.
No functionality change.
llvm-svn: 210008
This is a short-term fix to allow lld Readers to return error messages
with dynamic content.
The long term fix will be to enhance ErrorOr<> to work with errors other
than error_code. Or to change the interface to Readers to pass down a
diagnostics object through which all error messages are written.
llvm-svn: 209681
Alias symbols are SimpleDefinedAtoms and are platform neutral. They
don't have to belong ELF. This patch is to make it available to all
platforms. No functionality change intended.
Differential Revision: http://reviews.llvm.org/D3862
llvm-svn: 209475
In r205566, I made a change to Resolver so that Resolver revisit
only archive files in --start-group and --end-group pair. That's
not correct, as it also has to revisit DSO files.
This patch is to fix the issue.
Added a test to demonstrate the fix. I confirmed that it succeeded
before r205566, failed after r205566, and is ok with this patch.
Differential Revision: http://reviews.llvm.org/D3734
llvm-svn: 208797
Make it possible to add observers to an Input Graph, so that files
returned from an Input Graph can be examined before they are
passed to Resolver.
To implement some PE/COFF features we need to know all the symbols
that *can* be solved, including ones in archive files that are not
yet to be read.
Currently, Resolver only maintains a set of symbols that are
already read. It has no knowledge on symbols in skipped files in
an archive file.
There are many ways to implement that. I chose to apply the
observer pattern here because it seems most non-intrusive. We don't
want to mess up Resolver with architecture specific features.
Even in PE/COFF, the feature that needs this mechanism is minor.
So I chose not to modify Resolver, but add a hook to Input Graph.
Differential Revision: http://reviews.llvm.org/D3735
llvm-svn: 208753
Previously only the toplevel elements were expanded by expandElements().
Now we recursively call getReplacements() to expand input elements even
if they are in, say, in a group.
llvm-svn: 208144
Davide Italiano