This allows the combined LTO object to provide a definition with the same
name as a symbol that was internalized without causing a duplicate symbol
error. This normally happens during parallel codegen which externalizes
originally-internal symbols, for example.
In order to make this work, I needed to relax the undefined symbol error to
only report an error for symbols that are used in regular objects.
Differential Revision: http://reviews.llvm.org/D19954
llvm-svn: 268649
This patch implements a new design for the symbol table that stores
SymbolBodies within a memory region of the Symbol object. Symbols are mutated
by constructing SymbolBodies in place over existing SymbolBodies, rather
than by mutating pointers. As mentioned in the initial proposal [1], this
memory layout helps reduce the cache miss rate by improving memory locality.
Performance numbers:
old(s) new(s)
Without debug info:
chrome 7.178 6.432 (-11.5%)
LLVMgold.so 0.505 0.502 (-0.5%)
clang 0.954 0.827 (-15.4%)
llvm-as 0.052 0.045 (-15.5%)
With debug info:
scylla 5.695 5.613 (-1.5%)
clang 14.396 14.143 (-1.8%)
Performance counter results show that the fewer required indirections is
indeed the cause of the improved performance. For example, when linking
chrome, stalled cycles decreases from 14,556,444,002 to 12,959,238,310, and
instructions per cycle increases from 0.78 to 0.83. We are also executing
many fewer instructions (15,516,401,933 down to 15,002,434,310), probably
because we spend less time allocating SymbolBodies.
The new mechanism by which symbols are added to the symbol table is by calling
add* functions on the SymbolTable.
In this patch, I handle local symbols by storing them inside "unparented"
SymbolBodies. This is suboptimal, but if we do want to try to avoid allocating
these SymbolBodies, we can probably do that separately.
I also removed a few members from the SymbolBody class that were only being
used to pass information from the input file to the symbol table.
This patch implements the new design for the ELF linker only. I intend to
prepare a similar patch for the COFF linker.
[1] http://lists.llvm.org/pipermail/llvm-dev/2016-April/098832.html
Differential Revision: http://reviews.llvm.org/D19752
llvm-svn: 268178
Using multiple context used to be a really big memory saving because we
could free memory from each file while the linker proceeded with the
symbol resolution. We are getting lazier about reading data from the
bitcode, so I was curious if this was still a good tradeoff.
One thing that is a bit annoying is that we still have to copy the
symbol names. The problem is that the names are stored in the Module and
get freed when we move the module bits during linking.
Long term I think the solution is to add a symbol table to the bitcode.
That way IRObject file will not need to use a Module or a Context and we
can drop it while still keeping a StringRef to the names.
This patch is still be an interesting medium term improvement.
When linking llvm-as without debug info this patch is a small speedup:
master: 29.861877513 seconds
patch: 29.814533787 seconds
With debug info the numbers are
master: 34.765181469 seconds
patch: 34.563351584 seconds
The peak memory usage when linking llvm-as with debug info was
master: 599.10MB
patch: 600.13MB
llvm-svn: 267921
This patch is to remove -lto-no-discard-value-names flag and
instead to use -save-temps as we discussed in the post-commit
review thread for r267020.
Differential Revision: http://reviews.llvm.org/D19437
llvm-svn: 267230
These are properties of a symbol name, rather than a particular instance
of a symbol in an object file. We can simplify the code by collecting these
properties in Symbol.
The MustBeInDynSym flag has been renamed ExportDynamic, as its semantics
have been changed to be the same as those of --dynamic-list and
--export-dynamic-symbol, which do not cause hidden symbols to be exported.
Differential Revision: http://reviews.llvm.org/D19400
llvm-svn: 267183
Parallelism level can be chosen using the new --lto-jobs=K option
where K is the number of threads used for CodeGen. It currently
defaults to 1.
llvm-svn: 266484
Now MustBeInDynSym is only true if the symbol really must be in the
dynamic symbol table.
IsUsedInRegularObj is only true if the symbol is used in a .o or -u. Not
a .so or a .bc.
A benefit is that this is now done almost entirilly during symbol
resolution. The only exception is copy relocations because of aliases.
This includes a small fix in that protected symbols in .so don't force
executable symbols to be exported.
This also opens the way for implementing internalize for -shared.
llvm-svn: 265826
So, there are some cases when the IR Linker produces a broken
module (which doesn't pass the verifier) and we end up asserting
inside the verifier. I think it's always a bug producing a module
which does not pass the verifier but there are some cases in which
people can live with the broken module (e.g. if only DebugInfo
metadata are broken). The gold plugin has something similar.
This commit is motivated by a situation I found in the
wild. It seems that somebody else discovered it independently
and reported in PR24923.
llvm-svn: 265258
This fixes bootstrap of llvm-tblgen (with LTO) and PR27150.
Slightly longer explanation follows.
Emission of .init_array instead of .ctors is supported only on a
subset of the Target LLVM supports. Codegen needs to be conservative
and always emit .ctors unless instructed otherwise (based on target).
If the dynamic linker sees .init_array it completely ignores
what's inside .ctors and therefore some constructors are not called
(and this causes llvm-tblgen to crash on startup).
Teach LLD/LTO about the Codegen options so we end up always emitting
.init_array and avoid this issue.
In future, we might end up supporting mix of .ctors and .init_array
in different input files if this shows up as a real-world use case.
The way gold handles this case is mapping .ctors from input into
.init_array in output. There's also another caveat because
as far as I understand .ctors run in reverse order so when we do
the copy/mapping we need to reverse copy in the output if there's
more than one ctor. That's why I'd rather avoid this complicate logic
unless there's a real need.
An analogous reasoning holds for .dtors/.fini_array.
llvm-svn: 265085
IPO doesn't work very well across symbols referenced
by others TUs. The linker here tries to evaluate
which symbols are safe to internalize and switches
their linkage.
Differential Revision: http://reviews.llvm.org/D18415
llvm-svn: 264585
Ensure we keep the symbol we need to before it reaches
the Writer (and hit an assertion), changing its linkage
from linkonce_odr to weak. For a more detailed description
of the problem, see PR19901 where a similar problem was
fixed for the gold plugin. Thanks to Rafael for providing
a testcase.
llvm-svn: 264111
The code for LTO has been growing, so now is probably a good time to
move it to its own file. SymbolTable.cpp is for symbol table, and
because compiling bitcode files are semantically not a part of
symbol table, this is I think a good thing to do.
http://reviews.llvm.org/D18370
llvm-svn: 264091
Peter Collingbourne