Commit Graph

1571 Commits

Author SHA1 Message Date
Jan Ole Hüser 4e5a59a383 [LLD][COFF] Fix writing a map file when range extension thunks are inserted
Bug: An assertion fails:

    Assertion failed: isa<To>(Val) && "cast<Ty>() argument of incompatible type!",
    file C:\Users\<user>\prog\llvm\llvm-git-lld-bug\llvm\include\llvm/Support/Casting.h, line 578

Bug is triggered, if

    - a map file is requested with /MAP, and
    - Architecture is ARMv7, Thumb, and
    - a relative jump (branch instruction) is greater than 16 MiB (2^24)

The reason for the Bug is:

    - a Thunk is created for the jump
    - a Symbol for the Thunk is created
        - of type `DefinedSynthetic`
        - in file `Writer.cpp`
        - in function `getThunk`
    - the Symbol has no name
    - when creating the map file, the name of the Symbol is queried
    - the function `Symbol::computeName` of the base class `Symbol`
      casts the `this` pointer to type `DefinedCOFF` (a derived type),
      but the acutal type is `DefinedSynthetic`
    - The in the llvm::cast an assertion fails

Changes:

- Modify regression test to trigger this bug
- Give the symbol pointing to the thunk a name, to fix the bug
- Add assertion, that only DefinedCOFF symbols are allowed to have an
  empty name, when the constructor of the base class Symbol is executed

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D133201
2022-09-07 09:26:41 +03:00
Kazu Hirata 89f1433225 Use llvm::lower_bound (NFC) 2022-09-03 11:17:37 -07:00
Alvin Wong 12d865415f [COFF] Use the more accurate GuardFlags definition everywhere
This also modifies llvm-readobj to be more future-proof when printing
the guard FIDs table by calculating the entry size correctly according
to MS docs.

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D132924
2022-08-31 15:11:34 +03:00
Martin Storsjö af39e6f6fc [LLD] [COFF] Fix export directives in object files from -includeoptional
When an object file contains an export directive, we normally do some
amount of deferred processing of them at the end of the linking
process. The -includeoptional option was handled after this, and
any object files (defining new exports) weren't handled.

Move the handling of the -includeoptional into the same late loop
which does the fixups for e.g. export directives.

Ideally, this would also be done for object files that are pulled
in by the wrap options, and for mingw autoimports, but those changes
require more modifications, to make them safe for potentially
being executed multiple times.

This fixes https://github.com/llvm/llvm-project/issues/57243.

Differential Revision: https://reviews.llvm.org/D132361
2022-08-25 11:56:11 +03:00
Kazu Hirata 258531b7ac Remove redundant initialization of Optional (NFC) 2022-08-20 21:18:28 -07:00
Martin Storsjö 5d513ef6cf [LLD] [COFF] Add support for a new, mingw specific embedded directive -exclude-symbols:
This is an entirely new embedded directive - extending the GNU ld
command line option --exclude-symbols to be usable in embedded
directives too.

(GNU ld.bfd also got support for the same new directive, currently in
the latest git version, after the 2.39 branch.)

This works as an inverse to the regular embedded dllexport directives,
for cases when autoexport of all eligible symbols is performed.

Differential Revision: https://reviews.llvm.org/D130120
2022-08-11 11:59:48 +03:00
Martin Storsjö d1da6469f9 [LLD] [MinGW] Implement the --exclude-symbols option
This adds support for the existing GNU ld command line option, which
allows excluding individual symbols from autoexport (when linking a
DLL and no symbols are marked explicitly as dllexported).

Differential Revision: https://reviews.llvm.org/D130118
2022-08-11 11:59:47 +03:00
Fangrui Song dfe2a3f3eb [COFF] Change a llvm::StringSet<> to llvm::DenseSet<StringRef>. NFC
The former stores strings and is therefore more expensive.
2022-08-09 17:50:42 -07:00
Pengxuan Zheng c951edb7b2 [LLD][COFF] Identify /GL object files which are inside libraries
With D26647, we can already identify input object files compiled by cl.exe with
/GL. It seems to be helpful to do the same and print an error message for those
object files compiled with /GL but are inside libraries/archives too.

Reviewed By: rnk, thieta

Differential Revision: https://reviews.llvm.org/D131458
2022-08-09 12:32:35 -07:00
Nico Weber aa1abd7684 [lld/win] Use C++17 structured bindings
No behavior change.

Differential Revision: https://reviews.llvm.org/D131403
2022-08-08 11:51:38 -04:00
Nico Weber 7c26641d9d [lld/win] Use C++17 nested namespace syntax in most places
Like D131354, but for COFF.

No behavior change.

Differential Revision: https://reviews.llvm.org/D131405
2022-08-08 11:48:12 -04:00
Tobias Hieta 576375a2d6
[LLD][COFF] Ignore DEBUG_S_XFGHASH_TYPE/VIRTUAL
These are new debug types that ships with the latest
Windows SDK and would warn and finally fail lld-link.

The symbols seems to be related to Microsoft's XFG
which is their version of CFG. We can't handle any of
this yet, so for now we can just ignore these types
so that lld doesn't fail with a new version of Windows
SDK.

Fixes: #56285

Reviewed By: rnk

Differential Revision: https://reviews.llvm.org/D129378
2022-08-08 15:53:52 +02:00
Fangrui Song ec04e45c03 [lld] LLVM_FALLTHROUGH => [[fallthrough]]. NFC
With C++17 there is no Clang pedantic warning or MSVC C5051.
2022-08-07 00:02:35 +00:00
Kazu Hirata 873888c179 Use is_sorted (NFC) 2022-07-29 21:18:42 -07:00
Kazu Hirata c730f9a164 Convert for_each to range-based for loops (NFC) 2022-07-23 12:17:27 -07:00
Dmitri Gribenko aba43035bd Use llvm::sort instead of std::sort where possible
llvm::sort is beneficial even when we use the iterator-based overload,
since it can optionally shuffle the elements (to detect
non-determinism). However llvm::sort is not usable everywhere, for
example, in compiler-rt.

Reviewed By: nhaehnle

Differential Revision: https://reviews.llvm.org/D130406
2022-07-23 15:19:05 +02:00
Martin Storsjö 801971e5b4 [LLD] [COFF] Improve the error message for too many exported symbols
Print the actual number of symbols that would have been exported
too, which helps assessing the situation.

Differential Revision: https://reviews.llvm.org/D130117
2022-07-20 16:58:29 +03:00
Alex Brachet fd9962e75d [COFF] Add vfsoverlay flag
This patch adds a new flag vfsoverlay similar to clang’s
ivfsoverlay flag. This is helpful when compiling on case
sensitive file systems when cross compiling to Windows.
Particularly when compiling third party code containing
\#pragma comment(“linker”, “/defaultlib:...”) which
can’t be easily changed.

Differential Revision: https://reviews.llvm.org/D125800
2022-07-11 21:31:01 +00:00
Pengxuan Zheng b5e49cdea9 [LLD][COFF] Ignore /kernel flag
There exists some description of the flag from Microsoft, but not sure if
there's more to it. We ignore the flag for now until we find out more about it.

https://docs.microsoft.com/en-us/cpp/build/reference/kernel-create-kernel-mode-binary?view=msvc-170

Reviewed By: thieta, hans

Differential Revision: https://reviews.llvm.org/D128238
2022-07-01 10:03:02 -07:00
Martin Storsjö 4d2eda2bb3 Revert "[LLD] [COFF] Use StringTableBuilder to optimize the string table"
This reverts commit 9ffeaaa0ea.

This fixes debugging large executables with lldb and gdb.

When StringTableBuilder is used, the string offsets for any string
can point anywhere in the string table - while previously, all strings
were inserted in order (without deduplication and tail merging).

For symbols, there's no complications in encoding the string offset;
the offset is encoded as a raw 32 bit binary number in half of the
symbol name field.

For sections, the string table offset is written as
"/<decimaloffset>", but if the decimal offset would be larger than
7 digits, it's instead written as "//<base64offset>". Tools that
operate on object files can handle the base64 offset format, but
apparently neither lldb nor gdb expect that syntax when locating the
debug information section. Prior to the reverted commit, all long
section names were located at the start of the string table, so
their offset never exceeded the range for the decimal syntax.

Just reverting this change for now, as the actual benefit from it
was fairly modest.

Longer term, lld could write all long section names unoptimized
at the start of the string table, followed by all the strings for
symbol names, with deduplication and tail merging. And lldb and
gdb could be fixed to handle sections with the base64 offset syntax.

This fixes https://github.com/mstorsjo/llvm-mingw/issues/289.
2022-06-21 13:25:08 +03:00
Kazu Hirata ed8fceaa09 Don't use Optional::getValue (NFC) 2022-06-20 23:35:53 -07:00
Pengxuan Zheng dec1614791 [LLD][COFF] Ignore /pdbcompress flag
Microsoft does not seem to document the flag. Ignoring it for now is probably
better than getting an unknown flag error.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D128231
2022-06-20 16:48:39 -07:00
Kazu Hirata 5413bf1bac Don't use Optional::hasValue (NFC) 2022-06-20 11:33:56 -07:00
Nico Weber 7effcbda49 Rename parallelForEachN to just parallelFor
Patch created by running:

  rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: https://reviews.llvm.org/D128140
2022-06-19 17:49:00 -04:00
Pengxuan Zheng 9db61c3fe1 [LLD][COFF] Convert file name to lowercase when inserting it into visitedLibs
It seems to be a bug in `LinkerDriver::findFile`, the file name is not converted
to lowercase when being inserted into `visitedLibs`. This is the only exception
in the file and all other places always convert file names to lowercase when
inserting them into `visitedLibs` (or `visitedFiles`).

Reviewed By: thieta, hans

Differential Revision: https://reviews.llvm.org/D127709
2022-06-15 09:39:35 -07:00
Eli Friedman 13fc178173 Force GHashCell to be 8-byte-aligned.
Otherwise, with recent versions of libstdc++, clang can't tell that the
atomic operations are properly aligned, and generates calls to
libatomic.  (Actually, because of the use of reinterpret_cast, it wasn't
guaranteed to be aligned, but I think it ended up being aligned in
practice.)

Fixes https://github.com/llvm/llvm-project/issues/54790 , the part where
LLVM failed to build.

Differential Revision: https://reviews.llvm.org/D123872
2022-04-18 08:46:03 -07:00
Tobias Hieta 837d16fb4c [NFC] Simplify /noimplib argument logic 2022-04-13 16:40:30 +02:00
Tobias Hieta 2af4385477 [LLD][COFF] Add support for /noimplib
Mostly for compatibility reasons with link.exe this flag
makes sure we don't write a implib - not even when /implib
is also passed, that's how link.exe works.

Differential Revision: https://reviews.llvm.org/D123591
2022-04-13 16:40:29 +02:00
Tobias Hieta eb4eef9ec4 [LLD][COFF] Add support for /noimplib
Mostly for compatibility reasons with link.exe this flag
makes sure we don't write a implib - not even when /implib
is also passed, that's how link.exe works.

Reviewed By: mstorsjo

Differential Revision: https://reviews.llvm.org/D123591
2022-04-13 10:32:44 +02:00
Nico Weber 75196b99fb [llvm-lib] Add /WX, warn by default on empty inputs, add opt-out
lib.exe by default exits successfully without writing an output
file when no inputs are passed. llvm-lib has the same behavior,
for compatibility.

This behavior interacts poorly with build systems: If a static
library target had no inputs, llvm-lib would not produce an output
file, causing ninja (or make, or a similar system) to successfully
run that step, but then re-run it on the next build.

After this patch, llvm-lib emits a warning in this case, that with
/WX can be turned into an error. That way, ninja (or make, or...)
will mark the initial build as failed.

People who don't like the warning can use /ignore:emptyoutput to
suppress it.

The warning also points out the existing flag /llvmlibempty which
forces creation of an empty .lib file (this is an extension to lib.exe).

Differential Revision: https://reviews.llvm.org/D123517
2022-04-11 13:15:30 -04:00
Nikita Popov b8f50abd04 [lld] Remove support for legacy pass manager
This removes options for performing LTO with the legacy pass
manager in LLD. Options that explicitly enable the new pass manager
are retained as no-ops.

Differential Revision: https://reviews.llvm.org/D123219
2022-04-07 10:17:31 +02:00
Tobias Hieta 0dfa8a019d [LLD][COFF] Fix TypeServerSource matcher with more than one collision
Follow-up from 98bc304e9f - while that
commit fixed when you had two PDBs colliding on the same Guid it didn't
fix the case where you had more than two PDBs using the same Guid.

This commit fixes that and also tests much more carefully that all
the types are correct no matter the order.

Reviewed By: aganea, saudi

Differential Revision: https://reviews.llvm.org/D123185
2022-04-07 09:33:46 +02:00
Nikita Popov ed4e6e0398 [cmake] Remove LLVM_ENABLE_NEW_PASS_MANAGER cmake option
Or rather, error out if it is set to something other than ON. This
removes the ability to enable the legacy pass manager by default,
but does not remove the ability to explicitly enable it through
various flags like -flegacy-pass-manager or -enable-new-pm=0.

I checked, and our test suite definitely doesn't pass with
LLVM_ENABLE_NEW_PASS_MANAGER=OFF anymore.

Differential Revision: https://reviews.llvm.org/D123126
2022-04-06 09:52:21 +02:00
Tobias Hieta 98bc304e9f [lld][COFF] Fix TypeServerSource lookup on GUID collisions
Microsoft shipped a bunch of PDB files with broken/invalid GUIDs
which lead lld to use 0xFF as the key for these files in an internal
cache. When multiple files have this key it will lead to collisions
and confused symbol lookup.

Several approaches to fix this was considered. Including making the key
the path to the PDB file, but this requires some filesystem operations
in order to normalize the file path.

Since this only happens with malformatted PDB files and we haven't
seen this before they malformatted files where shipped with visual
studio we probably shouldn't optimize for this use-case.

Instead we now just don't insert files with Guid == 0xFF into the
cache map and warn if we get collisions so similar problems can be
found in the future instead of being silent.

Discussion about the root issue and the approach to this fix can be found on Github: https://github.com/llvm/llvm-project/issues/54487

Reviewed By: aganea

Differential Revision: https://reviews.llvm.org/D122372
2022-04-02 10:09:07 +02:00
Fangrui Song c37accf0a2 [Option] Avoid using the default argument for the 3-argument hasFlag. NFC
The default argument true is error-prone: I think many would think the
default is false.
2022-03-26 00:57:06 -07:00
serge-sans-paille f06d487dd6 Cleanup includes: WindowsDriver & WindowsManifest
Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D121330
2022-03-10 17:19:06 +01:00
Martin Storsjö 4c3b74b7f5 [LLD] [COFF] Order .debug_* sections at the end, to avoid leaving gaps if stripped
So far, we sort all discardable sections at the end, with only some
extra logic to make sure that the .reloc section is at the start
of that group of sections. But if there are other discardable
sections, other than .reloc, they must also be ordered before
.debug_* sections, to avoid leaving gaps if the executable is
stripped.

(Stripping executables doesn't remove all discardable sections,
only the ones named .debug_*).

Rust binaries seem to include a .rmeta section, which is marked
discardable. This fixes stripping such binaries if built with
dwarf debug info included.

This fixes issues observed in MSYS2 in
https://github.com/msys2/MINGW-packages/pull/10555.

Differential Revision: https://reviews.llvm.org/D120805
2022-03-03 10:08:51 +02:00
Martin Storsjö 9ffeaaa0ea [LLD] [COFF] Use StringTableBuilder to optimize the string table
This does tail merging (and deduplication) of the strings.

On a statically linked clang.exe, this shrinks the ~17 MB string
table by around 0.5 MB. This adds ~160 ms to the linking time
which originally was around 950 ms.

For cases where `-debug:symtab` or `-debug:dwarf` isn't set, the
string table is only used for long section names, where this
shouldn't make any difference at all.

Differential Revision: https://reviews.llvm.org/D120677
2022-03-01 18:44:03 +02:00
Martin Storsjö 9dd2d50984 [LLD] [COFF] Use the new encodeSectionName() helper for long section names
The previous code used an unbounded sprintf, which in theory can
overflow, writing either the null terminator or the last digits
into the next struct member.

In practice, in LLD, all long section names are written sequentially
first at the start of the string table, followed by all the long
symbol names. Due to this, even if the total string table would
end up large, the long section names have fairly short offsets,
which is why this hasn't been an issue in practice.

I don't think it's worth trying to write a test that produces an
executable with enough long section names to make the section names
themselves exceed 10^6 bytes, which is currently necessary to trigger
faults with the previous form.

Differential Revision: https://reviews.llvm.org/D120676
2022-03-01 11:33:02 +02:00
serge-sans-paille eb4c860811 Cleanup llvm/DebugInfo/PDB headers
accumulated preprocessed size:
before: 1065515095
after: 1065629059

Discourse thread: https://discourse.llvm.org/t/include-what-you-use-include-cleanup
Differential Revision: https://reviews.llvm.org/D120195
2022-02-23 10:31:34 +01:00
Nico Weber 746bd89000 fix comment typo to cycle bots 2022-02-22 16:25:51 -05:00
Peter Kasting c5fb05f663 Reland: Make lld-link work in a non-MSVC shell, add /winsysroot:
This relands 73e585e44d (and 0574b5fc65), with a fix for
the failing test (by using Optional<StringRef>s instead of
making StringRef::empty() mean absence of value).

Differential Revision: https://reviews.llvm.org/D118070
2022-02-16 09:22:39 -05:00
Douglas Yung 437d4e01fe Revert "try to fix windows build after 73e585e44d" and
Revert "Reland "[lld/coff] Make lld-link work in a non-MSVC shell, add /winsysroot:""

This reverts commit 0574b5fc65 and 73e585e44d.

This change is causing the test Driver/cl-options.c to fail on Windows buildbots.
https://lab.llvm.org/staging/#/builders/204/builds/1343
2022-02-11 23:47:53 -08:00
Nico Weber 73e585e44d Reland "[lld/coff] Make lld-link work in a non-MSVC shell, add /winsysroot:"
This relands commit b3b2538df1, except that the new files in Support
are instead in a new library WindowsDriver.
2022-02-11 17:07:33 -05:00
Adrian Prantl baac665adf Revert "[lld/coff] Make lld-link work in a non-MSVC shell, add /winsysroot:"
This reverts commit b3b2538df1,
it introduced a cycklic module depenency that broke the -DLLVM_ENABLE_MODULES=1 build.
2022-02-11 13:07:23 -08:00
Peter Kasting b3b2538df1 [lld/coff] Make lld-link work in a non-MSVC shell, add /winsysroot:
Makes lld-link work in a non-MSVC shell by autodetecting MSVC toolchain. Also
adds support for /winsysroot and a few other switches.

All this is done by refactoring to share code with clang-cl's existing support
for the same.

Differential Revision: https://reviews.llvm.org/D118070
2022-02-11 13:55:18 -05:00
Benjamin Kramer f15014ff54 Revert "Rename llvm::array_lengthof into llvm::size to match std::size from C++17"
This reverts commit ef82063207.

- It conflicts with the existing llvm::size in STLExtras, which will now
  never be called.
- Calling it without llvm:: breaks C++17 compat
2022-01-26 16:55:53 +01:00
serge-sans-paille ef82063207 Rename llvm::array_lengthof into llvm::size to match std::size from C++17
As a conquence move llvm::array_lengthof from STLExtras.h to
STLForwardCompat.h (which is included by STLExtras.h so no build
breakage expected).
2022-01-26 16:17:45 +01:00
Alexandre Ganea 83d59e05b2 Re-land [LLD] Remove global state in lldCommon
Move all variables at file-scope or function-static-scope into a hosting structure (lld::CommonLinkerContext) that lives at lldMain()-scope. Drivers will inherit from this structure and add their own global state, in the same way as for the existing COFFLinkerContext.

See discussion in https://lists.llvm.org/pipermail/llvm-dev/2021-June/151184.html

The previous land f860fe3622 caused issues in https://lab.llvm.org/buildbot/#/builders/123/builds/8383, fixed by 22ee510dac.

Differential Revision: https://reviews.llvm.org/D108850
2022-01-20 14:53:26 -05:00
Ayke van Laethem d649faff9c
[LLD][COFF] Support GNU style == aliases
D46245 added support for this in llvm-libtool, but while lld-link can
also create .lib files from .def files it didn't support aliases.

I compared the Inputs/library.def test against the output from
llvm-libtool and it matches, except for the fact that lld-link reorders
functions for some reason.

I have also verified that this fixes a bug I was running into while
trying to compile .def files to .lib files in MinGW-w64 (using lld-link
instead of llvm-libtool).

Differential Revision: https://reviews.llvm.org/D113365
2022-01-19 14:22:13 +01:00