First, do not reserve numSections in the Chunks array. In cases where
there are many non-prevailing sections, this will overallocate memory
which will not be used.
Second, free the memory for sparseChunks after initializeSymbols. After
that, it is never used.
This saves 50MB of 627MB for my use case without affecting performance.
This paves the way to doing more things in parallel, and allows us to
order type sources in dependency order. PDBs and PCH objects have to be
loaded before object files which use them.
This is a rebase of the unapplied remaining changes in
https://reviews.llvm.org/D59226. I found it very challenging to rebase
this across the LLD variable name style change. I recall there was a
tool for that, but I didn't take the time to use it.
Reviewers: aganea, akhuang
Subscribers: llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D79672
As we now have code that parses the dwarf info for variable locations,
we can use that instead of relying on the higher level Symbolizer library,
reducing the previous two different dwarf codepaths into one.
Differential Revision: https://reviews.llvm.org/D69198
llvm-svn: 375391
This fixes the second part of PR42407.
For files with dwarf debug info, it manually loads and iterates
.debug_info to find the declared location of variables, to allow
reporting them. (This matches the corresponding code in the ELF
linker.)
For functions, it uses the existing getFileLineDwarf which uses
LLVMSymbolizer for translating addresses to file lines.
In object files with codeview debug info, only the source location
of duplicate functions is printed. (And even there, only for the
first input file. The getFileLineCodeView function requires the
object file to be fully loaded and initialized to properly resolve
source locations, but duplicate symbols are reported at a stage when
the second object file isn't fully loaded yet.)
Differential Revision: https://reviews.llvm.org/D68975
llvm-svn: 375218
Summary:
This is a re-land of r370487 with a fix for the use-after-free bug
that rev contained.
This implements -start-lib and -end-lib flags for lld-link, analogous
to the similarly named options in ld.lld. Object files after
-start-lib are included in the link only when needed to resolve
undefined symbols. The -end-lib flag goes back to the normal behavior
of always including object files in the link. This mimics the
semantics of static libraries, but without needing to actually create
the archive file.
Reviewers: ruiu, smeenai, MaskRay
Reviewed By: ruiu, MaskRay
Subscribers: akhuang, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D66848
llvm-svn: 370816
Summary:
This implements -start-lib and -end-lib flags for lld-link, analogous
to the similarly named options in ld.lld. Object files after
-start-lib are included in the link only when needed to resolve
undefined symbols. The -end-lib flag goes back to the normal behavior
of always including object files in the link. This mimics the
semantics of static libraries, but without needing to actually create
the archive file.
Reviewers: ruiu, smeenai, MaskRay
Reviewed By: ruiu, MaskRay
Subscribers: akhuang, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D66848
llvm-svn: 370487
Extend WindowsResourceParser to support using a ResourceSectionRef for
loading resources from an object file.
Only allow merging resource object files in mingw mode; keep the
existing error on multiple resource objects in link mode.
If there only is one resource object file and no .res resources,
don't parse and recreate the .rsrc section, but just link it in without
inspecting it. This allows users to produce any .rsrc section (outside
of what the parser supports), just like before. (I don't have a specific
need for this, but it reduces the risk of this new feature.)
Separate out the .rsrc section chunks in InputFiles.cpp, and only include
them in the list of section chunks to link if we've determined that there
only was one single resource object. (We need to keep other chunks from
those object files, as they can legitimately contain other sections as
well, in addition to .rsrc section chunks.)
Differential Revision: https://reviews.llvm.org/D66824
llvm-svn: 370436
Also add test coverage for thin archives (which are the only way I could
come up with to test at least some of the diagnostic changes).
Differential Revision: https://reviews.llvm.org/D64927
llvm-svn: 366573
Summary:
Adds the following two options to lld-link:
-thinlto-prefix-replace: allows replacing a prefix in paths generated
for ThinLTO. This can be used to ensure index files and native object
files are stored in unique directories, allowing multiple distributed
ThinLTO links to proceed concurrently.
-thinlto-object-suffix-replace: allows replacing a suffix in object
file paths involved in ThinLTO. This allows minimized index files to
be used for the thin link while storing the paths to the full bitcode
files for subsequent steps (code generation and final linking).
Reviewers: ruiu, tejohnson, pcc, rnk
Subscribers: mehdi_amini, steven_wu, dexonsmith, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D64542
llvm-svn: 365807
This patch does the same thing as r365595 to other subdirectories,
which completes the naming style change for the entire lld directory.
With this, the naming style conversion is complete for lld.
Differential Revision: https://reviews.llvm.org/D64473
llvm-svn: 365730
Users are exepcted to pass all .res files to the linker, which then
merges all the resource in all .res files into a tree structure and then
converts the final tree structure to a .obj file with .rsrc$01 and
.rsrc$02 sections and then links that.
If the user instead passes several .obj files containing such resources,
the correct thing to do would be to have custom code to merge the trees
in the resource sections instead of doing normal section merging -- but
link.exe rejects if multiple resource obj files are passed in with
LNK4078, so let lld-link do that too instead of silently writing broken
.rsrc sections in that case.
The only real way to run into this is if users manually convert .res
files to .obj files by running cvtres and then handing the resulting
.obj files to lld-link instead, which in practice likely never happens.
(lld-link is slightly stricter than link.exe now: If link.exe is passed
one .obj file created by cvtres, and a .res file, for some reason it
just emits a warning instead of an error and outputs strange looking
data. lld-link now errors out on mixed input like this.)
One way users could accidentally run into this is the following
scenario: If a .res file is passed to lib.exe, then lib.exe calls
cvtres.exe on the .res file before putting it in the output .lib.
(llvm-lib currently doesn't do this.)
link.exe's /wholearchive seems to only add obj files referenced from the
static library index, but lld-link current really adds all files in the
archive. So if lld-link /wholearchive is used with .lib files produced
by lib.exe and .res files were among the files handed to lib.exe, we
previously silently produced invalid output, but now we error out.
link.exe's /wholearchive semantics on the other hand mean that it
wouldn't load the resource object files from the .lib file at all.
Since this scenario is probably still an unlikely corner case,
the difference in behavior here seems fine -- and lld-link might have to
change to use link.exe's /wholearchive semantics in the future anyways.
Vaguely related to PR42180.
Differential Revision: https://reviews.llvm.org/D63109
llvm-svn: 363078
Summary:
Archives can contain multiple members with the same name. This would
cause ThinLTO links to fail ("Expected at most one ThinLTO module per
bitcode file"). This change implements the same strategy we use in
the ELF linker: make the offset in the archive part of the module
name so that names are unique.
Reviewers: pcc, mehdi_amini, ruiu
Reviewed By: ruiu
Subscribers: eraman, steven_wu, dexonsmith, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D60549
llvm-svn: 358440
We introduce a new class hierarchy for debug types merging (in DebugTypes.h). The end-goal is to parallelize the type merging - please see the plan in D59226.
Previously, dependency discovery was done on the fly, much later, during the type merging loop. Unfortunately, parallelizing the type merging requires the dependencies to be merged in first, before any dependent ObjFile, thus this early discovery.
The overall intention for this path is to discover debug information dependencies at a much earlier stage, when processing input files. Currently, two types of dependency are supported: PDB type servers (when compiling with MSVC /Zi) and precompiled headers OBJs (when compiling with MSVC /Yc and /Yu). Once discovered, an explicit link is added into the dependent ObjFile, through the new debug types class hierarchy introduced in DebugTypes.h.
Differential Revision: https://reviews.llvm.org/D59053
llvm-svn: 357383
Summary:
This avoids allocating O(#relocs) of intermediate data for each section
when range extension thunks aren't needed for that section. This also
removes a std::vector from SectionChunk, which further reduces its size.
Instead, this change adds the range extension thunk symbols to the
object files that contain sections that need extension thunks. By adding
them to the symbol table of the parent object, that means they now have
a symbol table index. Then we can then modify the original relocation,
after copying it to read-write memory, to use the new symbol table
index.
This makes linking browser_tests.exe with no PDB 10.46% faster, moving
it from 11.364s to 10.288s averaged over five runs.
Reviewers: mstorsjo, ruiu
Subscribers: aganea, llvm-commits
Tags: #llvm
Differential Revision: https://reviews.llvm.org/D59902
llvm-svn: 357200
Take module DBI creation out of PDBLinker::addObjFile() into its own function.
This is groundwork towards parallelizable type merging, as proposed in D59226.
Differential Revision: https://reviews.llvm.org/D59261
llvm-svn: 356815
Turns out nobody understands what "conflicting comdat type" is supposed to
mean, so just emit a regular "duplicate symbol" error and move the comdat
selection information into /verbose output.
This also fixes a problem where the error output would depend on the order of
.obj files passed. Before this patch:
- If passed `one_only.obj discard.obj`, lld-link would only err "conflicting
comdat type"
- If passed `discard.obj one_only.obj`, lld-link would err "conflicting comdat
type" and then "duplicate symbol"
Now lld-link only errs "duplicate symbol" in both cases.
I considered adding a "Detail" parameter to reportDuplicate() that's printed in
parens at the end of the "duplicate symbol" diag if present, and then put the
comdat selection mismatch details there, but since users don't know what it's
supposed to mean decided against it. I also considered special-casing the
Detail message for one_only/discard mismatches, which in practice means
"function defined as inline in TU 1 but as out-of-line in TU 2", but I wasn't
sure how useful it is so I omitted that too.
Differential Revision: https://reviews.llvm.org/D58180
llvm-svn: 354006
LLD used to handle comdats as if the selection field was always set to
IMAGE_COMDAT_SELECT_ANY. This means for obj files produced by `cl /Gy`, LLD
would never report a duplicate symbol error.
This change:
- adds validation for the Selection field (should make no difference in
practice for compiler-generated obj inputs)
- rejects comdats that have different Selection fields in different obj files
(likewise). This is a bit more strict but also more self-consistent thank
link.exe (see comment in code)
- implements handling for all the selection kinds
In practice, compilers only generate comdats with
IMAGE_COMDAT_SELECT_NODUPLICATES (LLD now produces duplicate symbol errors for
these), IMAGE_COMDAT_SELECT_ANY (no behavior change), and
IMAGE_COMDAT_SELECT_LARGEST (for RTTI data; here LLD should no longer create
broken executables when linking some TUs with RTTI enabled and some with it
disabled – but see below).
The implementation of `IMAGE_COMDAT_SELECT_LARGEST` is incomplete: If one
SELECT_LARGEST comdat replaces an earlier one, the comdat symbol is replaced
correctly, but the old section stays loaded and if /opt:ref is disabled (via
/opt:noref or /debug) it's still written to the output. That's not ideal, but
better than the current treatment of just picking any one of those comdats. I
hope to fix this better later.
Fixes most of PR40094.
Differential Revision: https://reviews.llvm.org/D57324
llvm-svn: 352590
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
This change allows for link-time merging of debugging information from
Microsoft precompiled types OBJs compiled with cl.exe /Z7 /Yc and /Yu.
This fixes llvm.org/PR34278
Differential Revision: https://reviews.llvm.org/D45213
llvm-svn: 346154
MinGW configurations don't use associative comdats, as GNU ld doesn't
support that. Instead they produce normal comdats named .text$sym,
.xdata$sym and .pdata$sym.
GNU ld doesn't discard any comdats starting with .xdata or .pdata,
even if --gc-sections is used (while it does discard other unreferenced
comdats), regardless of what symbol name is used after the $ separator.
For LLD, treat any such comdat as implicitly associative to the base
symbol. This requires maintaining a map from symbol name to section
number, but that is only maintained when the MinGW flag has been
enabled.
Differential Revision: https://reviews.llvm.org/D49700
llvm-svn: 339058
Future symbol insertions can potentially change the type of these
symbols - keep pointers to the base class to reflect this, and
use dynamic casts to inspect them before using as the subclass
type.
This fixes crashes that were possible before, by touching these
symbols that now are populated as e.g. a DefinedRegular, via
the old pointers with DefinedImportThunk type.
Differential Revision: https://reviews.llvm.org/D48953
llvm-svn: 336652
In COFF, duplicate string literals are merged by placing them in a
comdat whose leader symbol name contains a specific prefix followed
by the hash and partial contents of the string literal. This gives
us an easy way to identify sections containing string literals in
the linker: check for leader symbol names with the given prefix.
Any sections that are identified in this way as containing string
literals may be tail merged. We do so using the StringTableBuilder
class, which is also used to tail merge string literals in the ELF
linker. Tail merging is enabled only if ICF is enabled, as this
provides a signal as to whether the user cares about binary size.
Differential Revision: https://reviews.llvm.org/D44504
llvm-svn: 327668
Summary:
This protects calls to longjmp from transferring control to arbitrary
program points. Instead, longjmp calls are limited to the set of
registered setjmp return addresses.
This also implements /guard:nolongjmp to allow users to link in object
files that call setjmp that weren't compiled with /guard:cf. In this
case, the linker will approximate the set of address taken functions,
but it will leave longjmp unprotected.
I used the following program to test, compiling it with different -guard
flags:
$ cl -c t.c -guard:cf
$ lld-link t.obj -guard:cf
#include <setjmp.h>
#include <stdio.h>
jmp_buf buf;
void g() {
printf("before longjmp\n");
fflush(stdout);
longjmp(buf, 1);
}
void f() {
if (setjmp(buf)) {
printf("setjmp returned non-zero\n");
return;
}
g();
}
int main() {
f();
printf("hello world\n");
}
In particular, the program aborts when the code is compiled *without*
-guard:cf and linked with -guard:cf. That indicates that longjmps are
protected.
Reviewers: ruiu, inglorion, amccarth
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D43217
llvm-svn: 325047
Summary:
This patch adds some initial support for Windows control flow guard. At
the end of the day, the linker needs to synthesize a table of RVAs very
similar to the structured exception handler table (/safeseh).
Both /safeseh and /guard:cf take sections of symbol table indices
(.sxdata and .gfids$y) and turn them into RVA tables referenced by the
load config struct in the CRT through special symbols.
Reviewers: ruiu, amccarth
Subscribers: llvm-commits
Differential Revision: https://reviews.llvm.org/D42592
llvm-svn: 324306
Instead of building intermediate sets of exception handlers for each
object file, just create one for the final output file.
Differential Revision: https://reviews.llvm.org/D40581
llvm-svn: 319244
If /debug was not specified, readSection will return a null
pointer for debug sections. If the debug section is associative with
another section, we need to make sure that the section returned from
readSection is not a null pointer before adding it as an associative
section.
Differential Revision: https://reviews.llvm.org/D40533
llvm-svn: 319133
With this change, instead of creating a SectionChunk for each section
in the object file, we only create them when we encounter a prevailing
comdat section.
Also change how symbol resolution occurs between comdat symbols. Now
only the comdat leader participates in comdat resolution, and not any
other external associated symbols. This is more in line with how COFF
semantics are defined, and should allow for a more straightforward
implementation of non-ANY comdat types.
On my machine, this change reduces our runtime linking a release
build of chrome_child.dll with /nopdb from 5.65s to 4.54s (median of
50 runs).
Differential Revision: https://reviews.llvm.org/D40238
llvm-svn: 319090
Now that we have only SymbolBody as the symbol class. So, "SymbolBody"
is a bit strange name now. This is a mechanical change generated by
perl -i -pe s/SymbolBody/Symbol/g $(git grep -l SymbolBody lld/ELF lld/COFF)
nd clang-format-diff.
Differential Revision: https://reviews.llvm.org/D39459
llvm-svn: 317370
Zequan Wu