Commit Graph

1517 Commits

Author SHA1 Message Date
Nico Weber 5730d11c2b [lld-link] Consistently print all /verbose output to stderr
lld-link used to consistently print all /verbose output to stdout, and that was
an intentional decision: https://reviews.llvm.org/rG4bce7bcc88f3

https://reviews.llvm.org/rGe6e206d4b4814 added message() and log(),
and back then `log()` morally was just `if (verbose) message(...)`
and message() wrote to stdout.

So that change moved most /verbose-induced writes to outs() to
log(). Except for the one in printDiscardedMessage(), since
the check for `verbose` for that one is in the caller, in
Writer::createSections():

    if (config->verbose)
      sc->printDiscardedMessage();

Later, https://reviews.llvm.org/D41033 changed log() to write to
stderr. That moved lld-link from writing all its /verbose output
to stdout to writing almost all of its /verbose output to stderr --
except for printDiscardedMessage() output.

This change moves printDiscardedMessage() to call log() as well,
so that all /verbose output once again consistently goes to the same
stream.

Differential Revision: https://reviews.llvm.org/D116667
2022-01-05 11:52:04 -05:00
Fangrui Song 292395329c [lld-link] Remove unneeded lto::InputFile::create after D116434 2022-01-04 19:38:32 -08:00
Luís Ferreira 10e40a4ea3 [lld] Add support for other demanglers other than Itanium
LLVM core library supports demangling other mangled symbols other than itanium,
such as D and Rust. LLD should use those demanglers in order to output pretty
demangled symbols on error messages.

Reviewed By: MaskRay, #lld-macho

Differential Revision: https://reviews.llvm.org/D116279
2022-01-05 03:25:41 +00:00
Fangrui Song d496abbe2a [lld-link] Replace LazyObjFile with lazy ObjFile/BitcodeFile
Similar to ELF 3a5fb57393.

* previously when a LazyObjFile was extracted, a new ObjFile/BitcodeFile was created; now the file is reused, just with `lazy` cleared
* avoid the confusing transfer of `symbols` from LazyObjFile to the new file
* simpler code, smaller executable (5200+ bytes smaller on x86-64)
* make eager parsing feasible (for parallel section/symbol table initialization)

Reviewed By: aganea, rnk

Differential Revision: https://reviews.llvm.org/D116434
2022-01-04 15:11:44 -08:00
Luís Ferreira 8792cd75d0 Revert "[lld] Add support for other demanglers other than Itanium"
This reverts commit e60d6dfd5a.

clang-ppc64le-rhel buildbot failed (https://lab.llvm.org/buildbot#builders/57/builds/13424):

    tools/lld/MachO/CMakeFiles/lldMachO.dir/Symbols.cpp.o: In function `lld::demangle(llvm::StringRef, bool)':
    Symbols.cpp:(.text._ZN3lld8demangleEN4llvm9StringRefEb[_ZN3lld8demangleEN4llvm9StringRefEb]+0x90): undefined reference to `llvm::demangle(std::string const&)'
2021-12-30 18:04:21 +00:00
Luís Ferreira e60d6dfd5a [lld] Add support for other demanglers other than Itanium
LLVM core library supports demangling other mangled symbols other than itanium,
such as D and Rust. LLD should use those demanglers in order to output pretty
demangled symbols on error messages.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D116279
2021-12-30 17:52:38 +00:00
Kazu Hirata 62e48ed10f Use isa instead of dyn_cast (NFC) 2021-12-24 21:22:27 -08:00
Noah Shutty fb6b103daa [lld] Replace Symbolize.h with DIContext.h in lld's COFF lib
lld only needs DIContext.h which it gets through Symbolize.h -> SymbolizableModule.h -> DIContext.h. This replaces it with a direct include of DIContext.h to avoid any confusion and pulling in unnecessary headers.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D115659
2021-12-13 22:16:41 +00:00
Chris Davis e4eb6216c2 Enable pdbpagesize to allow support for PDB file sizes > 4GB
Enable the pdbpagesize flag to allow linking of PDB files > 4GB.
Also includes a couple small fixes to change to uint64_t to support the
larger file sizes.  I updated the max file size check in MSFBuilder.cpp
to take into account the page size.

Differential Revision: https://reviews.llvm.org/D115051
2021-12-06 18:22:08 -05:00
Martin Storsjö d703b92296 [LLD] [COFF] Omit section symbols and IMAGE_SYM_CLASS_LABEL from the PE symbol table
The section symbols aren't of much practical use when looking at
a linked image. This shrinks one observed mingw style unstripped
binary by 14%.

IMAGE_SYM_CLASS_LABEL is in spirit the same as a temporary assembler
label that isn't emitted on the object file level at all.

Differential Revision: https://reviews.llvm.org/D113866
2021-11-23 10:17:04 +02:00
Martin Storsjö 7c15da6761 [LLD] [COFF] Interpret the immediate in ARM64 adr/adrp relocations as signed 21 bit
This matches how MS link.exe interprets this relocation.

Differential Revision: https://reviews.llvm.org/D114347
2021-11-23 10:13:01 +02:00
Duncan P. N. Exon Smith 9a2b54af22 lld: const-qualify iterations through VarStreamArray, NFC
No functionality change here; just unblocking a patch to LLVM.
2021-11-12 14:29:49 -08:00
Noah Shutty d788c44f5c [Support] Improve Caching conformance with Support library behavior
This diff makes several amendments to the local file caching mechanism
which was migrated from ThinLTO to Support in
rGe678c51177102845c93529d457b020f969125373 in response to follow-up
discussion on that commit.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D113080
2021-11-04 13:00:44 -07:00
Nico Weber f964ca896f [lld/coff] Add parsing for /pdbpagesize: flag
It's not used for anything yet, but we now accept `/pdbpagesize:4096`
(the default behavior) and we give arguably more useful diagnostics
for other values.

It's plumbed through to the MSF layer, so just uncommenting out
the bit in DriverUtils.cpp that rejects args other than 4096 is enough
to try other values.

Differential Revision: https://reviews.llvm.org/D112871
2021-10-31 18:36:23 -04:00
Sam Clegg 4da38c14d0 [lld] Rename addCombinedLTOObjects to match ELF driver. NFC
This function was renamed in https://reviews.llvm.org/D62291.
The new name seems more accurate and also its good to maintain
some consistency between these methods in the different drivers.

Differential Revision: https://reviews.llvm.org/D112719
2021-10-28 11:46:19 -07:00
Kazu Hirata 4ba9d9c84f Use StringRef::contains (NFC) 2021-10-23 20:41:46 -07:00
Noah Shutty e678c51177 [Support][ThinLTO] Move ThinLTO caching to LLVM Support library
We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LLVM.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D111371
2021-10-18 18:57:25 -07:00
Petr Hosek 8e46e34d24 Revert "[Support][ThinLTO] Move ThinLTO caching to LLVM Support library"
This reverts commit 92b8cc52bb since
it broke the gold plugin.
2021-10-18 12:24:05 -07:00
Noah Shutty 92b8cc52bb [Support][ThinLTO] Move ThinLTO caching to LLVM Support library
We would like to move ThinLTO’s battle-tested file caching mechanism to
the LLVM Support library so that we can use it elsewhere in LLVM.

Patch By: noajshu

Differential Revision: https://reviews.llvm.org/D111371
2021-10-18 12:08:49 -07:00
Kazu Hirata 8568ca789e Use llvm::erase_if (NFC) 2021-10-18 09:33:42 -07:00
Nico Weber f09dce564e [lld] fix typos to cycle bots 2021-10-12 17:03:39 -04:00
Nico Weber f3091831f4 [lld] Use checkError more
No behavior change.
2021-10-04 11:46:16 -04:00
Amy Huang 6f7483b1ec Reland "[LLD] Remove global state in lld/COFF" after fixing asan and msan test failures
Original commit description:

  [LLD] Remove global state in lld/COFF

  This patch removes globals from the lldCOFF library, by moving globals
  into a context class (COFFLinkingContext) and passing it around wherever
  it's needed.

  See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html for
  context about removing globals from LLD.

  I also haven't moved the `driver` or `config` variables yet.

  Differential Revision: https://reviews.llvm.org/D109634

This reverts commit a2fd05ada9.

Original commits were b4fa71eed3
and e03c7e367a.
2021-09-17 17:18:42 -07:00
Amy Huang a2fd05ada9 Temporarily revert "[LLD] Remove global state in lld/COFF" and "[lld] Add test to
check for timer output"

Seems to be causing a number of asan test failures.

This reverts commit b4fa71eed3
and e03c7e367a.
2021-09-16 11:58:11 -07:00
Amy Huang e03c7e367a [lld] Add test to check for timer output
This test checks that timers are working and printing as expected.

I also seem to have changed the order of the timers in my globals refactoring
patch, so I fixed it here.

Differential Revision: https://reviews.llvm.org/D109904
2021-09-16 11:36:46 -07:00
Amy Huang b4fa71eed3 [LLD] Remove global state in lld/COFF
This patch removes globals from the lldCOFF library, by moving globals
into a context class (COFFLinkingContext) and passing it around wherever
it's needed.

See https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html for
context about removing globals from LLD.

I also haven't moved the `driver` or `config` variables yet.

Differential Revision: https://reviews.llvm.org/D109634
2021-09-16 11:00:23 -07:00
Alexandre Ganea 7f0664f193 [LLD][COFF] Clean paths in PDB even when /pdbsourcepath is omitted
Differential Revision: https://reviews.llvm.org/D109030
2021-08-31 19:05:10 -04:00
Nico Weber 34ac7a7ac1 [lld/COFF] Ignore /LTCG, /LTCG:, /LTCGOUT:, /ILK: flags
We currently complain "could not open /LTCG: no such file or directory",
which isn't very useful.  We could emit a warning when we see this flag, but
just ignoring it seems fine.

Final missing part of PR38799.

Differential Revision: https://reviews.llvm.org/D108799
2021-08-27 09:13:30 -04:00
Nico Weber 66dc44f703 [lld/COFF] Use P_priv more
P_priv does the same as the old QF further down. Standardize on P_priv.

No behavior change.

Differential Revision: https://reviews.llvm.org/D108798
2021-08-27 08:48:05 -04:00
Nico Weber 400a1de3ac [lld/COFF] Improve handling of the /manifestdependency: flag
If multiple /manifestdependency: flags are passed, they are
naively deduped, but after that each of them should have an
effect, instead of just the last one.

Also, /manifestdependency: flags are allowed in .drectve sections
(from `#pragma comment(linker, ...`). To make the interaction between
/manifestdependency: flags enabling manifest by default but
/manifest:no overriding this work, add an explict ManifestKind::Default
state to represent no explicit /manifest flag being passed.
To make /manifestdependency: flags from input file .drectve sections
work with /manifest:embed, delay embedded manifest emission until
after input files have been read.

Differential Revision: https://reviews.llvm.org/D108628
2021-08-25 14:36:32 -04:00
Yolanda Chen 8fa16cc628 [LTO][lld] Add lto-pgo-warn-mismatch option
When enable CSPGO for ThinLTO, there are profile cfg mismatch warnings that will cause lld-link errors (with /WX)
due to source changes (e.g. `#if` code runs for profile generation but not for profile use)
To disable it we have to use an internal "/mllvm:-no-pgo-warn-mismatch" option.
In contrast clang uses option ”-Wno-backend-plugin“ to avoid such warnings and gcc has an explicit "-Wno-coverage-mismatch" option.

Add "lto-pgo-warn-mismatch" option to lld COFF/ELF to help turn on/off the profile mismatch warnings explicitly when build with ThinLTO and CSPGO.

Differential Revision: https://reviews.llvm.org/D104431
2021-08-11 09:45:55 -07:00
Wang, Pengfei 6c4809825d Revert "[lld] Add lto-pgo-warn-mismatch option"
This reverts commit 0cfb00a1c9.
2021-08-11 16:25:42 +08:00
Yolanda Chen 0cfb00a1c9 [lld] Add lto-pgo-warn-mismatch option
When enable CSPGO for ThinLTO, there are profile cfg mismatch warnings that will cause lld-link errors (with /WX).
To disable it we have to use an internal "/mllvm:-no-pgo-warn-mismatch" option.
In contrast clang uses option ”-Wno-backend-plugin“ to avoid such warnings and gcc has an explicit "-Wno-coverage-mismatch" option.

Add this "lto-pgo-warn-mismatch" option to lld to help turn on/off the profile mismatch warnings explicitly when build with ThinLTO and CSPGO.

Reviewed By: tejohnson

Differential Revision: https://reviews.llvm.org/D104431
2021-08-11 14:43:26 +08:00
Martin Storsjö 9dbc4b09af [LLD] [COFF] Make -export-all-symbols work as intended for EXEs
If some symbols are marked with dllexport, we still want to export
all symbols if -export-all-symbols is specified. Previously, this
only worked as it should for DLL output, not for EXE.

This should fix downstream bug
https://github.com/msys2/MINGW-packages/issues/9163.

Differential Revision: https://reviews.llvm.org/D106245
2021-07-22 23:34:03 +03:00
Martin Storsjö e0e09481ee [LLD] [COFF] Add a couple "MinGW only" comments re linking against DLLs. NFC.
This was requested in the post-commit review of D104530.
2021-07-20 23:57:24 +03:00
Fangrui Song db5e078690 [LTO] Add SelectionKind to IRSymtab and use it in ld.lld/LLVMgold
In PGO, a C++ external linkage function `foo` has a private counter
`__profc_foo` and a private `__profd_foo` in a `comdat nodeduplicate`.

A `__attribute__((weak))` function `foo` has a weak hidden counter `__profc_foo`
and a private `__profd_foo` in a `comdat nodeduplicate`.

In `ld.lld a.o b.o`, say a.o defines an external linkage `foo` and b.o
defines a weak `foo`. Currently we treat `comdat nodeduplicate` as `comdat any`,
ld.lld will incorrectly consider `b.o:__profc_foo` non-prevailing.  In the worst
case when `b.o:__profd_foo` is retained and `b.o:__profc_foo` isn't, there will
be dangling reference causing an `undefined hidden symbol` error.

Add SelectionKind to `Comdat` in IRSymtab and let linkers ignore nodeduplicate comdat.

Differential Revision: https://reviews.llvm.org/D106228
2021-07-20 13:22:00 -07:00
David Blaikie 1def2579e1 PR51018: Remove explicit conversions from SmallString to StringRef to future-proof against C++23
C++23 will make these conversions ambiguous - so fix them to make the
codebase forward-compatible with C++23 (& a follow-up change I've made
will make this ambiguous/invalid even in <C++23 so we don't regress
this & it generally improves the code anyway)
2021-07-08 13:37:57 -07:00
Jeremy Drake 7a7da69fbe [LLD] [COFF] Avoid thread exhaustion on 32-bit Windows host
LLD on 32-bit Windows would frequently fail on large projects with
an exception "thread constructor failed: Exec format error".  The stack
trace pointed to this usage of std::async, and looking at the
implementation in libc++ it seems using std::async with
std::launch::async results in the immediate creation of a new thread
for every call.  This could result in a potentially unbounded number
of threads, depending on the number of input files.  This seems to
be hitting some limit in 32-bit Windows host.

I took the easy route, and only use threads on 64-bit Windows, not all
Windows as before.  I was thinking a more proper solution might
involve using a thread pool rather than blindly spawning any number
of new threads, but that may have other unforeseen consequences.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D105506
2021-07-07 22:00:18 +03:00
Martin Storsjö ce211c505b [LLD] [COFF] Fix up missing stdcall decorations in MinGW mode
If linking directly against a DLL without an import library, the
DLL export symbols might not contain stdcall decorations.

If we have an undefined symbol with decoration, and we happen to have
a matching undecorated symbol (which either is lazy and can be loaded,
or already defined), then alias it against that instead.

This matches what's done in reverse, when we have a def file
declaring to export a symbol without decoration, but we only have
a defined decorated symbol. In that case we do a fuzzy match
(SymbolTable::findMangle). This case is more straightforward; if we
have a decorated undefined symbol, just strip the decoration and look
for the corresponding undecorated symbol name.

Add warnings and options for either silencing the warning or disabling
the whole feature, corresponding to how ld.bfd does it.

(This feature works for any symbol decoration mismatch, not only when
linking against a DLL directly; ld.bfd also tolerates it anywhere,
and also fixes up mismatches in the other direction, like
SymbolTable::findMangle, for any symbol, not only exports. But in
practice, at least for lld, it would primarily end up used for linking
against DLLs.)

Differential Revision: https://reviews.llvm.org/D104532
2021-07-02 09:49:14 +03:00
Martin Storsjö a9ff1ce1b9 [LLD] [COFF] Support linking directly against DLLs in MinGW mode
GNU ld.bfd supports linking directly against DLLs without using an
import library, and some projects have picked up on this habit.
(There's no one single unsurmountable issue with using import
libraries, but this is a regularly surfacing missing feature.)

As long as one is linking by name (instead of by ordinal), the DLL
export table contains most of the information needed. (One can
inspect what section a symbol points at, to see if it's a function
or data symbol. The practical implementation of this loops over all
sections for each symbol, but as long as they're not very many, that
should hopefully be tolerable performance wise.)

One exception where the information in the DLL isn't entirely enough
is on i386 with stdcall functions; depending on how they're done,
the exported function name can be a plain undecorated name, while
the import library would contain the full decorated symbol name. This
issue is addressed separately in a different patch.

This is implemented mimicing the structure of a regular import library,
with one InputFile corresponding to the static archive that just adds
lazy symbols, which then are fetched when they are needed. When such
a symbol is fetched, we synthesize a coff_import_header structure
in memory and create a regular ImportFile out of it.

The implementation could be even smaller by just creating ImportFiles
for every symbol available immediately, but that would have the
drawback of actually ending up importing all symbols unless running
with GC enabled (and mingw mode defaults to having it disabled for
historical reasons).

Differential Revision: https://reviews.llvm.org/D104530
2021-07-02 09:49:13 +03:00
Martin Storsjö d07f43641f [LLD] [COFF] Fix handling of LTO comdats with nontrivial selection types after 728cc0075e
Commit 728cc0075e made comdat symbols
from LTO objects be treated as any regular comdat symbol. This works
great for symbols that actually are IMAGE_COMDAT_SELECT_ANY, but
if the symbols have a less trivial selection type that require comparing
either the section chunk size or contents, we can't check that before
actually doing the LTO compilation.

Therefore bring back one aspect of handling from before; that comdat
resolution with a leader from an LTO symbol is essentially skipped,
like it was before 728cc0075e.

Differential Revision: https://reviews.llvm.org/D104605
2021-06-25 09:39:56 +03:00
Fangrui Song f1e2d5851b [OptTable] Rename PrintHelp to printHelp
To be consistent with other member functions and match the coding standard.
2021-06-24 14:47:03 -07:00
Martin Storsjö 3c6f8ca7c9 [lld] Rename StringRef _lower() method calls to _insensitive() 2021-06-25 00:22:01 +03:00
Reid Kleckner 8d84751ac4 Revert "[LLD] [COFF] Avoid doing repeated fuzzy symbol lookup for each iteration. NFC."
This reverts commit e1adf90826.

This appears to affect the way that C++ mangled symbols appear in the
import library when using a .def file that names a C++ free function
with no name decoration. I will follow up with a reduced test case
shortly.
2021-06-22 11:35:14 -07:00
Fangrui Song 50225112b5 [lld-link] Fix -Wunused-but-set-variable in -DLLVM_ENABLE_ASSERTIONS=off build. NFC 2021-06-20 11:35:02 -07:00
Martin Storsjö e1adf90826 [LLD] [COFF] Avoid doing repeated fuzzy symbol lookup for each iteration. NFC.
This is run every time around in the main linker loop. Once a match
has been found, stop trying to rematch such a symbol.

Not sure if this has any actual measurable performance impact though
(SymbolTable::findMangle() iterates over the whole symbol table for
each call and does fuzzy matching on top of that) but this makes the
code more reassuring to read at least. (This is in practice run for def
files listing undecorated stdcall functions to be exported.)

Differential Revision: https://reviews.llvm.org/D104529
2021-06-19 22:32:37 +03:00
Martin Storsjö ceee35e3e4 [LLD] [COFF] Remove a stray duplicate comment. NFC.
The following class isn't part of the export table; there's a
second correctly placed comment about the things that actually
belong to the export table.
2021-06-17 13:02:35 +03:00
Martin Storsjö 728cc0075e [LLD] [COFF] Fix autoexport from LTO objects with comdat symbols
Make sure that comdat symbols also have a non-null dummy
SectionChunk associated.

This requires moving around an existing FIXME regarding comdats in
LTO.

Differential Revision: https://reviews.llvm.org/D103012
2021-06-03 15:14:49 +03:00
Alexandre Ganea 2b9b9652ce [LLD][COFF] Reduce the maximum size of the GHASH table
Before this patch, the maximum size of the GHASH table was 2^31 buckets. However we were storing the bucket index into a TypeIndex which has an input limit of (2^31)-4095 indices, see this link. Any value above that limit will improperly set the TypeIndex's high bit, which is interpreted as DecoratedItemIdMask. This used to cause bad indices on extraction when calling TypeIndex::toArrayIndex().

Differential Revision: https://reviews.llvm.org/D103297
2021-05-28 09:46:17 -04:00
Reid Kleckner 99f023656b [PDB] Fix ubsan complaint about memcpy from null pointer 2021-05-27 19:49:09 -07:00