Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.
See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html
Differential Revision: https://reviews.llvm.org/D108850
lld-link used to consistently print all /verbose output to stdout, and that was
an intentional decision: https://reviews.llvm.org/rG4bce7bcc88f3https://reviews.llvm.org/rGe6e206d4b4814 added message() and log(),
and back then `log()` morally was just `if (verbose) message(...)`
and message() wrote to stdout.
So that change moved most /verbose-induced writes to outs() to
log(). Except for the one in printDiscardedMessage(), since
the check for `verbose` for that one is in the caller, in
Writer::createSections():
if (config->verbose)
sc->printDiscardedMessage();
Later, https://reviews.llvm.org/D41033 changed log() to write to
stderr. That moved lld-link from writing all its /verbose output
to stdout to writing almost all of its /verbose output to stderr --
except for printDiscardedMessage() output.
This change moves printDiscardedMessage() to call log() as well,
so that all /verbose output once again consistently goes to the same
stream.
Differential Revision: https://reviews.llvm.org/D116667
Original commit description:
[LLD] Remove global state in lld/COFF
This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.
See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html for
context about removing globals from LLD.
I also haven't moved the `driver` or `config` variables yet.
Differential Revision: https://reviews.llvm.org/D109634
This reverts commit a2fd05ada9.
Original commits were b4fa71eed3
and e03c7e367a.
This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.
See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html for
context about removing globals from LLD.
I also haven't moved the `driver` or `config` variables yet.
Differential Revision: https://reviews.llvm.org/D109634
The COFF driver produces an ABSOLUTE relocation base for an ADDR32
relocation type and the system is 64 bits (machine=AMD64). The
relocation information won't be added in the output and could
produce an incorrect address access during run-time. This change
set checks if the relocation type is IMAGE_REL_AMD64_ADDR32 and
if so, adds the relocated symbol as IMAGE_REL_BASED_HIGHLOW base.
Differential Revision: https://reviews.llvm.org/D96619
This is a different approach from D98993 that should achieve most of the
same benefit. The two changes are:
1. Sort the list of associated child sections by section name
2. Do not consider associated sections to have children themselves
This fixes the main issue, which was that we sometimes considered an
.xdata section to have a child .pdata section. That lead to slow links
and larger binaries (less xdata folding).
Otherwise, this should be NFC: we go back to ignoring .debug/.gljmp and
other metadata sections rather than only looking at pdata/xdata. We
discovered that we do care about other associated sections, like ASan
global registration metadata.
This reverts commit bacf9cf2c5 and
reinstates commit 1a9bd5b813.
Reverting this commit did not appear to make the problem go away, so we
can go ahead and reland it.
This reverts commit 5b7aef6eb4 and relands
6529d7c5a4.
The ASan error was debugged and determined to be the fault of an invalid
object file input in our test suite, which was fixed by my last change.
LLD's project policy is that it assumes input objects are valid, so I
have added a comment about this assumption to the relocation bounds
check.
This is a pretty classic optimization. Instead of processing symbol
records and copying them to temporary storage, do a first pass to
measure how large the module symbol stream will be, and then copy the
data into place in the PDB file. This requires defering relocation until
much later, which accounts for most of the complexity in this patch.
This patch avoids copying the contents of all live .debug$S sections
into heap memory, which is worth about 20% of private memory usage when
making PDBs. However, this is not an unmitigated performance win,
because it can be faster to read dense, temporary, heap data than it is
to iterate symbol records in object file backed memory a second time.
Results on release chrome.dll:
peak mem: 5164.89MB -> 4072.19MB (-1,092.7MB, -21.2%)
wall-j1: 0m30.844s -> 0m32.094s (slightly slower)
wall-j3: 0m20.968s -> 0m20.312s (slightly faster)
wall-j8: 0m19.062s -> 0m17.672s (meaningfully faster)
I gathered similar numbers for a debug, component build of content.dll
in Chrome, and the performance impact of this change was in the noise.
The memory usage reduction was visible and similar.
Because of the new parallelism in the PDB commit phase, more cores makes
the new approach faster. I'm assuming that most C++ developer machines
these days are at least quad core, so I think this is a win.
Differential Revision: https://reviews.llvm.org/D94267
I noticed that std::error_code() does one-time initialization. Avoid
that overhead with Expected<T> and llvm::Error. Also, it is consistent
with the virtual interface and ELF, and generally cleaner.
Reviewed By: MaskRay
Differential Revision: https://reviews.llvm.org/D79643
This patch does the same thing as r365595 to other subdirectories,
which completes the naming style change for the entire lld directory.
With this, the naming style conversion is complete for lld.
Differential Revision: https://reviews.llvm.org/D64473
llvm-svn: 365730
lld/coff already deduplicated undefined symbols on a TU level: It would
group all references to a symbol from a single TU. This makes it so that
references from all TUs to a single symbol are grouped together.
Since lld/coff almost did what I thought it did already, the patch is
much smaller than the elf version. The only not local change is that
getSymbolLocations() now returns a vector<string> instead of a string,
so that the undefined symbol reporting code can know how many references
to a symbol exist in a given TU.
Fixes PR42260 for lld/coff.
Differential Revision: https://reviews.llvm.org/D63646
llvm-svn: 364285
An unrecognized signature (magic) at the beginning of a debug section
should not be a fatal error; it only means that the debug information
is in a format that is not supported by LLD. This can be due to it
being in CodeView versions 3 or earlier. These can occur in old import
libraries from legacy SDKs.
The test case was verified to work with MS link.exe.
Patch by Vladimir Panteleev!
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D63177
llvm-svn: 363212
Shaves another pointer off of SectionChunk, reducing the size from 96 to
88 bytes, down from 144 before I started working on this. Combined with
D62356, this reduced peak memory usage when linking chrome_child.dll
from 713MB to 675MB, or 5%.
Create NonSectionChunk to provide virtual dispatch to the rest of the
chunk types.
Reviewers: ruiu, aganea
Differential Revision: https://reviews.llvm.org/D62362
llvm-svn: 361667
Shaves another 8 bytes off of SectionChunk, the most commonly allocated
type in LLD.
These indices are only valid after we've assigned chunks to output
sections and removed empty sections, so do that in a new pass.
Reviewers: ruiu, aganea
Differential Revision: https://reviews.llvm.org/D62356
llvm-svn: 361657
This only needs to be done for MergeChunks, so just do that in a
separate pass in the Writer.
This is one small step towards eliminating the vtable in Chunk.
llvm-svn: 361573
The previous patch lost the call to PowerOf2Ceil, which causes LLD to
crash when handling common symbols with a non-power-of-2 size. I tweaked
the existing common.test to make the bsspad16 common symbol be 15 bytes
to add coverage for this case.
llvm-svn: 361426
Summary:
Valid section or chunk alignments are powers of 2 in the range [1,
8192]. These can be stored more canonically in log2 form to free up some
bits in Chunk. Combined with D61696, SectionChunk gets 8 bytes smaller.
Reviewers: ruiu, aganea
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D61698
llvm-svn: 361206
Change
std::error_code getSectionContents(DataRefImpl, StringRef &) const;
to
Expected<ArrayRef<uint8_t>> getSectionContents(DataRefImpl) const;
Many object formats use ArrayRef<uint8_t> as the underlying type, which
is generally better than StringRef to represent binary data, so change
the type to decrease the number of type conversions.
Reviewed By: ruiu, sbc100
Differential Revision: https://reviews.llvm.org/D61781
llvm-svn: 360648
Summary:
Prior to this change, every implementation of writeTo would add
OutputSectionOff to the output section buffer start before writing data.
Instead, do this math in the caller, so that it can be written once
instead of many times.
The output section offset is always equivalent to the difference between
the chunk RVA and the output section RVA, so we can replace the one
remaining usage of OutputSectionOff with that subtraction.
This doesn't change the size of SectionChunk because of alignment
requirements, but I will rearrange the fields in a follow-up change to
accomplish that.
Reviewers: ruiu, aganea
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D61696
llvm-svn: 360376
SectionChunk is one of the most frequently allocated data structures in
LLD, since there are about four per function when optimizations and
debug info are enabled (.text, .pdata, .xdata, .debug$S).
A PE COFF file cannot be larger than 2GB, so there is an inherent limit
on the length of the section name and the number of relocations.
Decompose the ArrayRef and StringRef into pointer and size, and put them
back together in the accessors for section name and relocation list.
I plan to gather complete performance numbers later by padding
SectionChunk with dead data and measuring performance after all the size
optimizations are done.
llvm-svn: 359923
Summary:
It currently receives an output parameter and returns
std::error_code. Expected<StringRef> fits for this purpose perfectly.
Differential Revision: https://reviews.llvm.org/D61421
llvm-svn: 359774
Summary:
Reorder the fields in both to use padding more efficiently, and add more
comments on the purpose of the fields.
Replace `std::vector<SectionChunk*> AssociativeChildren` with a
singly-linked list. This avoids the separate vector allocation to list
associative children, and shrinks the 3 pointers used for the typically
empty vector down to 1.
In the end, this reduces the sum of heap allocations used to link
browser_tests.exe with NO PDB by 13.10%, going from 2,248,728 KB to
1,954,071 KB of heap. These numbers exclude memory mapped files, which
are of course a significant factor in LLD's memory usage.
Reviewers: ruiu, mstorsjo, aganea
Subscribers: jdoerfert, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59797
llvm-svn: 357535
Summary:
This avoids allocating O(#relocs) of intermediate data for each section
when range extension thunks aren't needed for that section. This also
removes a std::vector from SectionChunk, which further reduces its size.
Instead, this change adds the range extension thunk symbols to the
object files that contain sections that need extension thunks. By adding
them to the symbol table of the parent object, that means they now have
a symbol table index. Then we can then modify the original relocation,
after copying it to read-write memory, to use the new symbol table
index.
This makes linking browser_tests.exe with no PDB 10.46% faster, moving
it from 11.364s to 10.288s averaged over five runs.
Reviewers: mstorsjo, ruiu
Subscribers: aganea, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59902
llvm-svn: 357200
On ARM64, this is normally necessary only after a module exceeds
128 MB in size (while the limit for thumb is 16 MB). For conditional
branches, the range limit is only 1 MB though (the same as for thumb),
and for the tbz instruction, the range is only 32 KB, which allows for
a test much smaller than the full 128 MB.
This fixes PR40467.
Differential Revision: https://reviews.llvm.org/D57575
llvm-svn: 352929
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
We initialize .text section with 0xcc (INT3 instruction), so we need to
explicitly write data even if it is zero if it can be in a .text section.
If you specify /merge:.rdata=.text, .rdata (which contains .idata) is put
to .text, so we need to do this.
Fixes https://bugs.llvm.org/show_bug.cgi?id=39826
Differential Revision: https://reviews.llvm.org/D55098
llvm-svn: 348000
Summary:
Reuse the "referenced by" note diagnostic code that we already use for
undefined symbols. In my case, it turned this:
lld-link: error: relocation against symbol in discarded section: .text
lld-link: error: relocation against symbol in discarded section: .text
...
Into this:
lld-link: error: relocation against symbol in discarded section: .text
>>> referenced by libANGLE.lib(CompilerGL.obj):(.SCOVP$M)
>>> referenced by libANGLE.lib(CompilerGL.obj):(.SCOVP$M)
...
lld-link: error: relocation against symbol in discarded section: .text
>>> referenced by obj/third_party/angle/libGLESv2/entry_points_egl_ext.obj:(.SCOVP$M)
>>> referenced by obj/third_party/angle/libGLESv2/entry_points_egl_ext.obj:(.SCOVP$M)
...
I think the new output is more useful.
Reviewers: ruiu, pcc
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D54240
llvm-svn: 346427
(patch by Benoit Rousseau)
This patch fixes a bug where the global variable initializers were sometimes not invoked in the correct order when it involved a C++ template instantiation.
Differential Revision: https://reviews.llvm.org/D52749
llvm-svn: 343847
Alexandre Ganea