Commit Graph

13152 Commits

Author SHA1 Message Date
Jez Ng 1e1a3f67ee [lld-macho] Ensure reads from nlist_64 structs are aligned when necessary
My test refactoring in D80217 seems to have caused yaml2obj to emit
unaligned nlist_64 structs, causing ASAN'd lld to be unhappy. I don't
think this is an issue with yaml2obj though -- llvm-mc also seems to
emit unaligned nlist_64s. This diff makes lld able to safely do aligned
reads under ASAN builds while hopefully creating no overhead for regular
builds on architectures that support unaligned reads.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D80414
2020-06-02 13:19:38 -07:00
Jez Ng f04d1c3b90 [lld-macho] Move all tests for erroneous inputs under invalid/
For consistency.

The no-id-dylib test was originally referencing the Inputs/ folder via a
relative path. Instead of updating that path, I decided to make the test
self-contained.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D80217
2020-06-02 13:19:38 -07:00
Jez Ng a04c133564 [lld-macho] Set __PAGEZERO size to 4GB
That's what ld64 uses for 64-bit targets. I figured it's best to make
this change sooner rather than later since a bunch of our tests are
relying on hardcoded addresses that depend on this value.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80177
2020-06-02 13:19:38 -07:00
Jez Ng df2a5778c3 [lld-macho] Error on encountering undefined symbols
... instead of silently emitting a reference to the zero address.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80169
2020-06-02 13:19:38 -07:00
Jez Ng 6f6d91867d [lld-macho] Add some relocation validation logic
I considered making a `Target::validate()` method, but I wasn't sure how
I felt about the overhead of doing yet another switch-dispatch on the
relocation type, so I put the validation in `relocateOne` instead...
might be a bit of a micro-optimization, but `relocateOne` does assume
certain things about the relocations it gets, and this error handling
makes that explicit, so it's not a totally unreasonable code
organization.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80049
2020-06-02 13:19:38 -07:00
Igor Kudrin 4933ab2ccb [DebugInfo] Report the format of call frame information entries [6/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Simon Atanasyan b00f0d4238 [mips] Support 64-bit relative relocations
MIPS 64-bit ABI does not provide special PC-relative relocation like
R_MIPS_PC32 in 32-bit case. But we can use a "chain of relocation"
defined by N64 ABIs. In that case one relocation record might contain up
to three relocations which applied sequentially. Width of a final relocation
mask applied to the result of relocation depends on the last relocation
in the chain. In case of 64-bit PC-relative relocation we need the following
chain: `R_MIPS_PC32 | R_MIPS_64`. The first relocation calculates an
offset, but does not truncate the result. The second relocation just
apply calculated result as a 64-bit value.

The 64-bit PC-relative relocation might be useful in generation of
`.eh_frame` sections to escape passing `-Wl,-z,notext` flags to linker.

Differential Revision: https://reviews.llvm.org/D80390
2020-06-02 11:44:11 +03:00
Sriraman Tallam e0bca46b08 Options for Basic Block Sections, enabled in D68063 and D73674.
This patch adds clang options:
-fbasic-block-sections={all,<filename>,labels,none} and
-funique-basic-block-section-names.
LLVM Support for basic block sections is already enabled.

+ -fbasic-block-sections={all, <file>, labels, none} : Enables/Disables basic
block sections for all or a subset of basic blocks. "labels" only enables
basic block symbols.
+ -funique-basic-block-section-names: Enables unique section names for
basic block sections, disabled by default.

Differential Revision: https://reviews.llvm.org/D68049
2020-06-02 00:23:32 -07:00
Fangrui Song a6ae333a0c [ELF] --wrap: don't error `undefined reference to __real_foo` (--no-allow-shlib-undefined) if foo is a wrapped definition
This is a regression after D51283.

Also, export `foo` if `__real_foo` is referenced by a shared object.
2020-06-01 23:00:51 -07:00
Reid Kleckner 11d1aa0bcc [COFF] Free some memory used for chunks
First, do not reserve numSections in the Chunks array. In cases where
there are many non-prevailing sections, this will overallocate memory
which will not be used.

Second, free the memory for sparseChunks after initializeSymbols. After
that, it is never used.

This saves 50MB of 627MB for my use case without affecting performance.
2020-06-01 18:51:47 -07:00
Reid Kleckner 45fd3e4688 [PDB] Share code to relocate .debug$[SF] sections, NFC
Sink relocateDebugChunk near the only call site.
2020-06-01 13:16:57 -07:00
Reid Kleckner 8f0a660030 [PDB] Use inlinee file checksum offsets directly
The inlinees section contains references to the file checksum table. The
file checksum table in the PDB must have the same layout as the file
checksum table in the object file, so all the existing file id
references should stay valid.

Previously, we would do this:
  for all inlined functions:
    - lookup filename from checksum and string table
    - make that filename absolute
    - look up the new file id for that filename up in the new checksum
      table

This lead to pdbMakeAbsolute and remove_dots ending up in the hot path.
We should only need to absolutify the source path once, not once every
time we process an inline function from that source file.

This speeds up linking chrome PGO stage 1 net_unittests.exe from 9.203s
to 8.500s (-7.6%). Looking just at time to process symbol records, it
goes from ~2000ms to ~1300ms, which is consistent with the overall
speedup of about 700ms. This will be less noticeable in debug builds,
which have fewer inlined functions records.
2020-06-01 12:28:32 -07:00
Fangrui Song 751f18e7d4 [ELF] Refine --export-dynamic-symbol semantics to be compatible GNU ld 2.35
GNU ld from binutils 2.35 onwards will likely support
--export-dynamic-symbol but with different semantics.
https://sourceware.org/pipermail/binutils/2020-May/111302.html

Differences:

1. -export-dynamic-symbol is not supported
2. --export-dynamic-symbol takes a glob argument
3. --export-dynamic-symbol can suppress binding the references to the definition within the shared object if (-Bsymbolic or -Bsymbolic-functions)
4. --export-dynamic-symbol does not imply -u

I don't think the first three points can affect any user.
For the fourth point, Not implying -u can lead to some archive members unfetched.
Add -u foo to restore the previous behavior.

Exact semantics:

* -no-pie or -pie: matched non-local defined symbols will be added to the dynamic symbol table.
* -shared: matched non-local STV_DEFAULT symbols will not be bound to definitions within the shared object
  even if they would otherwise be due to -Bsymbolic, -Bsymbolic-functions, or --dynamic-list.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80487
2020-06-01 11:30:03 -07:00
Fangrui Song ee9a251caf [ELF] Set DF_1_PIE for -pie
DF_1_PIE originated from Solaris (https://docs.oracle.com/cd/E36784_01/html/E36857/chapter6-42444.html ).
GNU ld since
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=5fe2850dd96483f176858fd75c098313d5b20bc2
sets the flag on non-Solaris platforms.

It can help distinguish PIE from ET_DYN.
eu-classify from elfutils uses this to recognize PIE (https://sourceware.org/git/?p=elfutils.git;a=commit;h=3f489b5c7c78df6d52f8982f79c36e9a220e8951 )

glibc uses this flag to reject dlopen'ing a PIE (https://sourceware.org/bugzilla/show_bug.cgi?id=24323 )

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80872
2020-06-01 10:19:41 -07:00
Fangrui Song ce1fadca60 [ELF][docs] Update supported targets
PowerPC, PowerPC64 and x86-32 have production quality.
Mention Hexagon, RISC-V and SPARC V9.
2020-05-30 19:33:50 -07:00
Fangrui Song 881c5eef98 [ELF] Add -z rel and -z rela
LLD supports both REL and RELA for static relocations, but emits either
of REL and RELA for dynamic relocations. The relocation entry format is
specified by each psABI.

musl ld.so supports both REL and RELA. For such ld.so implementations,
REL (.rel.dyn .rel.plt) has size benefits even if the psABI chooses RELA:
sizeof(Elf64_Rel)=16 < sizeof(Elf64_Rela)=24.

* COPY, GLOB_DAT and J[U]MP_SLOT always have 0 addend. A ld.so
  implementation does not need to read the implicit addend.
  REL is strictly better.
* A RELATIVE has a non-zero addend. Such relocations can be packed
  compactly with the RELR relocation entry format, which is out of scope
  of this patch.
* For other dynamic relocation types (e.g. symbolic relocation R_X86_64_64),
  a ld.so implementation needs to read the implicit addend. REL may have
  minor performance impact, because reading implicit addends forces
  random access reads instead of being able to blast out a bunch of
  writes while chasing the relocation array.

This patch adds -z rel and -z rela to change the relocation entry format
for dynamic relocations. I have tested that a -z rel produced x86-64
executable works with musl ld.so

-z rela may be useful for debugging purposes on processors whose psABIs
specify REL as the canonical format: addends can be easily read by a tool.

Reviewed By: grimar, mcgrathr

Differential Revision: https://reviews.llvm.org/D80496
2020-05-29 14:22:03 -07:00
Sam Clegg 81443ac1bc [WebAssembly] Add placeholders for R_WASM_TABLE_INDEX_REL_SLEB relocations
Previously in the object format we punted on this and simply wrote
zeros (and didn't include the function in the elem segment).  With
this change we write a meaningful value which is the segment
relative table index of the associated function.

This matches the that wasm-ld produces in `-r` mode.  This inconsistency
between the output the MC object writer and the wasm-ld object
writer could cause warnings to be emitted when reading back in the
output of `wasm-ld -r`.  See:
https://github.com/emscripten-core/emscripten/issues/11217

This only applies to this one relocation type which is only generated
when compiling in PIC mode.

Differential Revision: https://reviews.llvm.org/D80774
2020-05-29 10:57:26 -07:00
Sam Clegg fd1c894a4a [lld][WebAssembly] Convert some lld tests to assembly
When we originally wrote these tests we didn't have a stable and
fleshed out assembly format.  Now we do so we should prefer that
over llvm ir for lld tests to avoid including more part of llvm
than necessary in order to run the test.

This change converts just 30 out of about 130 test files. More to
come when I have some more time.

Differential Revision: https://reviews.llvm.org/D80361
2020-05-28 16:52:01 -07:00
Hiroshi Yamauchi a7fa35a629 [ThinLTO] Compute the basic block count across modules.
Summary:
Count the per-module number of basic blocks when the module summary is computed
and sum them up during Thin LTO indexing.

This is used to estimate the working set size under the partial sample PGO.

This is split off of D79831.

Reviewers: davidxl, espindola

Subscribers: emaste, inglorion, hiraditya, MaskRay, steven_wu, dexonsmith, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80403
2020-05-28 10:33:05 -07:00
Simon Pilgrim f6417f5db8 FileOutputBuffer.h - remove unused includes. NFC.
Move dependent includes down to source files where necessary.
2020-05-28 14:38:12 +01:00
Fangrui Song 1224e619d9 [ELF][test] Fix wrap-no-real.s after D51283
Give %t3.so a DT_SONAME so that the DT_NEEDED entry in a dependent executable has a fixed length.
2020-05-27 17:43:33 -07:00
Rui Ueyama 54d2896852 [ELF] --wrap: Drop __real_ symbol from the symbol table
In D34993, we discussed and concluded that we should drop `__real_
symbol from the symbol table, but I did the opposite in D50569.
This patch is to drop `__real_` symbol.

MaskRay's note: omitting `__real_` is important if it is undefined:
otherwise a subsequent link may error due to the undefined `__real_` in .dynsym

Differential Revision: https://reviews.llvm.org/D51283
2020-05-27 16:58:00 -07:00
Fangrui Song b8a3c618d6 [ELF] Allow misaligned SHT_GNU_verneed
Bazel created interface shared objects (.ifso) may be misaligned.  We use
llvm::support::detail::packed_endian_specific_integral under the hood
which allows reading of misaligned values, so there is not a need to
diagnose (in LLD we don't intend to support sophisticated parsing for
SHT_GNU_*).
2020-05-26 11:18:19 -07:00
Fangrui Song bae7cf6746 [ELF][PPC64] Synthesize _savegpr[01]_{14..31} and _restgpr[01]_{14..31}
In the 64-bit ELF V2 API Specification: Power Architecture, 2.3.3.1. GPR
Save and Restore Functions defines some special functions which may be
referenced by GCC produced assembly (LLVM does not reference them).

With GCC -Os, when the number of call-saved registers exceeds a certain
threshold, GCC generates `_savegpr0_* _restgpr0_*` calls and expects the
linker to define them. See
https://sourceware.org/pipermail/binutils/2002-February/017444.html and
https://sourceware.org/pipermail/binutils/2004-August/036765.html . This
is weird because libgcc.a would be the natural place. However, the linker
generation approach has the advantage that the linker can generate
multiple copies to avoid long branch thunks. We don't consider the
advantage significant enough to complicate our trunk implementation, so
we take a simple approach.

* Check whether `_savegpr0_{14..31}` are used
* If yes, define needed symbols and add an InputSection with the code sequence.

`_savegpr1_*` `_restgpr0_*` and `_restgpr1_*` are similar.

Reviewed By: sfertile

Differential Revision: https://reviews.llvm.org/D79977
2020-05-26 09:35:41 -07:00
Kang Zhang 86e3abc9e6 [PowerPC] Add some InstAlias definitions
Summary:
This patch add the InstAlias definitions for below instructions.

ADDI ADDIS ADDI8 ADDIS8
RLWINM8
ISEL ISEL8
OR OR_rec ORI ORI8 XORI8
CNTLZW8 CNTLZW8_rec
TEND TSR
RFEBB
NOR NOR_rec
MTCRF
SUBF SUBF_rec SUBFC SUBFC_rec
RLDICL_32_64
TW

Reviewed By: steven.zhang

Differential Revision: https://reviews.llvm.org/D77559
2020-05-24 14:05:28 +00:00
Tobias Hieta f794808bb9 [LLD/MinGW]: Expose --thinlto-cache-dir
Differential Revision: https://reviews.llvm.org/D80438
2020-05-24 12:30:56 +03:00
Fangrui Song e32f04cdc9 [ELF] Parse SHT_GNU_verneed and respect versioned undefined symbols in shared objects
An undefined symbol in a shared object can be versioned, like `f@v1`.
We currently insert `f` as an Undefined into the symbol table, but we
should insert `f@v1` instead.

The string `v1` is inferred from SHT_GNU_versym and SHT_GNU_verneed.
This patch implements the functionality.

Failing to do this can cause two issues:

* If a versioned symbol referenced by a shared object is defined in the
  executable, we will fail to export it.
* If a versioned symbol referenced by a shared object in another object
  file, --no-allow-shlib-undefined may spuriously report an
  "undefined reference to " error. See https://bugs.llvm.org/show_bug.cgi?id=44842
  (Linking -lfftw3 -lm on Arch Linux can cause
  `undefined reference to __log_finite`)

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D80059
2020-05-23 09:55:48 -07:00
Thomas Lively d851fce4cb [lld][WebAssembly] Do not emit initialization for .bss segments
Summary:
This patch fixes a bug where initialization code for .bss segments was
emitted in the memory initialization function even though the .bss
segments were discounted in the datacount section and omitted in the
data section. This was producing invalid binaries due to out-of-bounds
segment indices on the memory.init and data.drop instructions that
were trying to operate on the nonexistent .bss segments.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80354
2020-05-21 11:33:25 -07:00
Reid Kleckner 3508c1d8fb [LLD] Make scoped timers thread safe
Summary:
This is a pre-requisite to parallelizing PDB symbol and type merging.
Currently this timer usage would not be thread safe.

Reviewers: aganea, MaskRay

Subscribers: jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80298
2020-05-20 16:16:08 -07:00
Jez Ng ce0d8beebc [lld-macho][re-land] Support X86_64_RELOC_UNSIGNED
This reverts commit db8559eee4.
2020-05-19 12:31:55 -07:00
Jez Ng 4eb6f4854e [lld-macho][re-land] Support .subsections_via_symbols
Summary:
This diff restores and builds upon @pcc and @ruiu's initial work on
subsections.

The .subsections_via_symbols directive indicates we can split each
section along symbol boundaries, unless those symbols have been marked
with `.alt_entry`.

We exercise this functionality in our tests by using order files that
rearrange those symbols.

Depends on D79668.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: thakis, llvm-commits, pcc, ruiu

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79926
2020-05-19 12:31:54 -07:00
Fangrui Song 6467649974 [ELF] Make --trace-symbol track preempted shared definitions
Note, we still name a preempted SharedSymbol "shared definition",
instead of "reference" as printed by GNU ld. This difference should not matter.

```
// GNU ld
ld.bfd: t: definition of f@v1
ld.bfd: t.so: reference to f@v1
```

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80143
2020-05-19 08:56:35 -07:00
Jez Ng 70fbbcdd34 Revert "[lld-macho] Support .subsections_via_symbols"
Due to build breakage mentioned in https://reviews.llvm.org/D79926.

This reverts commit e270b2f172.
2020-05-19 08:30:02 -07:00
Jez Ng db8559eee4 Revert "[lld-macho] Support X86_64_RELOC_UNSIGNED"
This reverts commit 1f820e3559.
2020-05-19 08:30:02 -07:00
Jez Ng 1f820e3559 [lld-macho] Support X86_64_RELOC_UNSIGNED
Note that it's only used for non-pc-relative contexts.

Reviewed By: MaskRay, smeenai

Differential Revision: https://reviews.llvm.org/D80048
2020-05-19 07:46:57 -07:00
Jez Ng e270b2f172 [lld-macho] Support .subsections_via_symbols
This diff restores and builds upon @pcc and @ruiu's initial work on
subsections.

The .subsections_via_symbols directive indicates we can split each
section along symbol boundaries, unless those symbols have been marked
with `.alt_entry`.

We exercise this functionality in our tests by using order files that
rearrange those symbols.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D79926
2020-05-19 07:46:57 -07:00
Jez Ng 55e9eb416e [lld-macho] Support -order_file
The order file indicates how input sections should be sorted within each
output section, based on the symbols contained within those sections.

This diff sets the stage for implementing and testing
`.subsections_via_symbols`, where we will break up InputSections by each
symbol and sort them more granularly.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D79668
2020-05-19 07:46:57 -07:00
Greg McGary 1aadd6ce61 [lld] Remove unused lld/test/Driver/Inputs/**/libtest.a
Under `lld/test/Driver/Inputs/`, all instances of `libtest.a` are
unreferenced. FYI, all of these are empty archives, and the files
contain only a magic number.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D80182
2020-05-19 07:19:10 -07:00
Hongtao Yu 90af55d8a9 [LLD][ELF] Use offset in thin archives to disambiguate thinLTO members
This is fixing a thinLTO module collision issue for thin archives. The problem is that we always use a zero offset to name members in a thin archive and that causes the following build error:

    ld.lld: error: Expected at most one ThinLTO module per bitcode file

which happens to a thin archive that has two members with the same object file name (whose paths will be ignored by thinLTO driver)

The fix here is to use real member offset instead as is done for non-thin archives.

Differential Revision: https://reviews.llvm.org/D79880
2020-05-15 12:02:08 -07:00
Fangrui Song e36223c85c [ELF] Enforce two dashes for Flag options not supported by GNU ld (i.e. no compatibility burden)
Announced on https://lists.llvm.org/pipermail/llvm-dev/2020-May/141416.html

Similar to D79371, but for `multiclass B` (convenience helper for defining --foo and --no-foo)

Some changed options are also used by gold, but I haven't seen their
one-dash use cases outside of lld's testsuite.
2020-05-15 11:07:25 -07:00
Fangrui Song 07837b8f49 [ELF] Use namespace qualifiers (lld:: or elf::) instead of `namespace lld { namespace elf {`
Similar to D74882. This reverts much code from commit
bd8cfe65f5 (D68323) and fixes some
problems before D68323.

Sorry for the churn but D68323 was a mistake. Namespace qualifiers avoid
bugs where the definition does not match the declaration from the
header. See
https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions (D74515)

Differential Revision: https://reviews.llvm.org/D79982
2020-05-15 08:49:53 -07:00
Georgii Rymar 969c63a2ec [LLD][ELF] - Fix section-alignment.test after yaml2obj change.
yaml2obj changed in rG7ccae2cece72. Previous version of the
yaml produced a different error and a 500mb binary.
2020-05-15 12:00:37 +03:00
Russell Gallop 85bb9b71b7 [ELF] Update release notes and man page for LLD time-trace
Differential Revision: https://reviews.llvm.org/D79780
2020-05-15 08:35:58 +01:00
Kellie Medlin 2b920ae78c [lld] Add archive file support to Mach-O backend
With this change, basic archive files can be linked together. Input
section discovery has been refactored into a function since archive
files lazily resolve their symbols / the object files containing those
symbols.

Reviewed By: int3, smeenai

Differential Revision: https://reviews.llvm.org/D78342
2020-05-14 12:58:35 -07:00
Reid Kleckner 3774bcf9f8 [COFF] Fix var names cVStrTab->cvStrTab sXDataChunks->sxDataChunks
NFC
2020-05-14 11:23:07 -07:00
Reid Kleckner f5a79713b6 [LLD] Fix precomp-link.test
I had a stale directory at %t, so the test passed locally, but not
remotely. Skip the %t directory altogether.
2020-05-14 10:32:50 -07:00
Reid Kleckner 54a335a2f6 [COFF] Move type merging to TpiSource::mergeDebugT virtual method
This paves the way to doing more things in parallel, and allows us to
order type sources in dependency order. PDBs and PCH objects have to be
loaded before object files which use them.

This is a rebase of the unapplied remaining changes in
https://reviews.llvm.org/D59226. I found it very challenging to rebase
this across the LLD variable name style change. I recall there was a
tool for that, but I didn't take the time to use it.

Reviewers: aganea, akhuang

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79672
2020-05-14 09:47:00 -07:00
Matt Arsenault 2b2a961309 lld: Let find_package(LLD) work
Install a cmake config file. Copied exactly from how clang exports.

I also wasn't sure whether the canonical capitalization is "lld" or
"LLD". The project() is still calling this lld, but most places seemed
to capitalize it.
2020-05-14 09:59:53 -04:00
Martin Storsjö b0969be4df [LLD] [COFF] Try to fix test errors from 7f0e6c31c2 on windows
Just skip trying to match for the path separator explicitly (instead
of making it match either a forward or backwards slash), simplifying
the test a little.
2020-05-14 13:47:34 +03:00
Martin Storsjö 7f0e6c31c2 [LLD] [COFF] Add options for disabling auto import and runtime pseudo relocs
Allow disabling either the full auto import feature, or just
forbidding the cases that require runtime fixups.

As long as all auto imported variables are referenced from separate
.refptr$<name> sections, we can alias them on top of the IAT entries
and don't actually need any runtime fixups via pseudo relocations.
LLVM generates references to variables in .refptr stubs, if it
isn't known that the variable for sure is defined in the same object
module. Runtime pseudo relocs are needed if the addresses of auto
imported variables are used in constant initializers though.

Fixing up runtime pseudo relocations requires the use of
VirtualProtect (which is disallowed in WinStore/UWP apps) or
VirtualProtectFromApp. To allow any risk of ambiguity, allow
rejecting cases that would require this at the linker stage.

This adds support for the --disable-runtime-pseudo-reloc and
--disable-auto-import options in the MinGW driver (matching GNU ld.bfd)
with corresponding lld private options in the COFF driver.

Differential Revision: https://reviews.llvm.org/D78923
2020-05-14 13:05:14 +03:00
Sam Clegg 064e9907ba [lld][WebAssembly] Fix for --relocatable and signature mismatches
This is a followup to https://reviews.llvm.org/D78779.

When signatures mismatch we create set of variant symbols.  Some of
the fields in these symbols were not be initialized correct.
Specifically we were seeing isUsedInRegularObj not being set correctly,
leading to the symbol not getting included in the symbol table
and a crash writing relections in --reloctable mode.

There is larger refactor due here, but this is a minimal change the
fixes the bug at hand.

Differential Revision: https://reviews.llvm.org/D79756
2020-05-13 10:27:09 -07:00
Nico Weber 759bae956a [lld-macho] Ignore -platform_version and -syslibroot flags.
clang passes these flags; this makes it easier to try `clang -v`
output with `ld -flavor darwinnew`.

Differential Revision: https://reviews.llvm.org/D79797
2020-05-12 19:17:01 -04:00
Jez Ng 87b6fd3e02 [lld-macho] Add support for creating and reading reexported dylibs
This unblocks the linking of real programs, since many core system
functions are only available as sub-libraries of libSystem.

Differential Revision: https://reviews.llvm.org/D79228
2020-05-12 07:52:03 -07:00
Jez Ng c8c39185f3 [lld-macho] Re-add dylink-lazy test
This reverts commit eb81de2de4003e3045fdf743e093f77e37aee9bf; the
test commands just needed to be run under llvm-lit.
2020-05-12 07:52:03 -07:00
Reid Kleckner 6da5672962 [LLD] Rename iDTable -> idTable, NFC
The variable renaming change did not handle this variable well.
2020-05-12 06:37:39 -07:00
Eric Christopher eb81de2de4 Temporarily Revert "[lld-macho] Re-add dylink-lazy test" as it
appears to be still failing.

This reverts commit 723c46e645.
2020-05-11 19:47:21 -07:00
Eric Christopher 020022e12e Fix auto -> auto * clang tidy. 2020-05-11 15:50:52 -07:00
stevewan 44f7b40249 Add vendor macro to "lld"
Summary:
Add the vendor macro to "lld" for extended version output support,
such that it's able to print additional version info. This is
consistent with the Clang and LLVM version printer, and the
additional version message can be provided via PACKAGE_VENDOR.

Reviewers: hubert.reinterpretcast, kbarton, cebowleratibm, rzurob, ruiu

Reviewed By: hubert.reinterpretcast

Subscribers: emaste, mgorny, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79026
2020-05-11 18:35:11 -04:00
Jez Ng 723c46e645 [lld-macho] Re-add dylink-lazy test
The initial attempt didn't work on Windows; apparently Powershell has a
different syntax for running commands sequentially and concatenating
their outputs. So I've created two temporary files instead.

Differential Revision: https://reviews.llvm.org/D79697
2020-05-11 12:39:21 -07:00
Peter Smith 0ae7990b60 [ELF][ARM] Support /DISCARD/ of subset of .ARM.exidx sections
Both the .ARM.exidx and .eh_frame sections have a custom SyntheticSection
that acts as a container for the InputSections. The InputSections are added
to the SyntheticSection prior to /DISCARD/ which limits the affect a
/DISCARD/ can have to the whole SyntheticSection. In the majority of cases
this is sufficient as it is not common to discard subsets of the
InputSections. The Linux kernel has one of these scripts which has something
like:
/DISCARD/ : { *(.ARM.exidx.exit.text) *(.ARM.extab.exit.text) ... }
The .ARM.exidx.exit.text are not discarded because the InputSection has been
transferred to the Synthetic Section. The *(.ARM.extab.exit.text) sections
have not so they are discarded. When we come to write out the .ARM.exidx
sections the dangling references from .ARM.exidx.exit.text to
.ARM.extab.exit.text currently cause relocation out of range errors, but
could as easily cause a fatal error message if we check for dangling
references at relocation time.

This patch attempts to respect the /DISCARD/ command by running it on the
.ARM.exidx InputSections stored in the SyntheticSection.

The .eh_frame is in theory affected by this problem, but I don't think that
there is a dangling reference problem that can happen with these sections.

Fixes remaining part of pr44824

Differential Revision: https://reviews.llvm.org/D79687
2020-05-11 14:27:13 +01:00
Martin Storsjö ed0a57f753 [LLD] [COFF] Fix def file exporting of symbols containing periods
This fixes an accidental breakage of exporting symbols using def
files, when the symbol name contains a period, since commit
0ca06f7950, mixing up a symbol name containing a period with
the case of exporting a symbol as a forward to another dll.

Differential Revision: https://reviews.llvm.org/D79619
2020-05-10 23:30:14 +03:00
Jez Ng 0dcd7c34f1 [lld-macho] Delete dylink-lazy test for now
Summary: It's breaking Windows builds
2020-05-10 02:22:33 -07:00
Jez Ng 8a5e45aa90 [lld-macho] Try to unbreak dylink-lazy test on Windows 2020-05-10 00:23:32 -07:00
Jez Ng 198b0c57df [lld-macho] Support pc-relative section relocations
Summary: So far we've only supported symbol relocations.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79211
2020-05-09 20:56:23 -07:00
Jez Ng 7bbdbacd00 [lld-macho] Use export trie instead of symtab when linking against dylibs
Summary:
This allows us to link against stripped dylibs. Moreover, it's simply
more correct: The symbol table includes symbols that the dylib uses but
doesn't export.

This temporarily regresses our ability to do lazy symbol binding because
dyld_stub_binder isn't in libSystem's export trie. Rather, it is in one
of the sub-libraries libSystem re-exports. (This doesn't affect our
tests since we are mocking out dyld_stub_binder there.) A follow-up diff
will address this by adding support for sub-libraries.

Depends on D79114.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: mgorny, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79226
2020-05-09 20:56:22 -07:00
Jez Ng 5d3feefa0d [lld-macho] Dylib symbols should always replace undefined symbols
Summary:
Otherwise we get undefined symbol errors depending on the order of
arguments on the command line.

Depends on D78270.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79114
2020-05-09 20:56:22 -07:00
Jez Ng b3e2fc931d [lld-macho] Support calls to functions in dylibs
Summary:
This diff implements lazy symbol binding -- very similar to the PLT
mechanism in ELF.

ELF's .plt section is broken up into two sections in Mach-O:
StubsSection and StubHelperSection. Calls to functions in dylibs will
end up calling into StubsSection, which contains indirect jumps to
addresses stored in the LazyPointerSection (the counterpart to ELF's
.plt.got).

Initially, the LazyPointerSection contains addresses that point into one
of the entry points in the middle of the StubHelperSection. The code in
StubHelperSection will push on the stack an offset into the
LazyBindingSection. The push is followed by a jump to the beginning of
the StubHelperSection (similar to PLT0), which then calls into
dyld_stub_binder. dyld_stub_binder is a non-lazily bound symbol, so this
call looks it up in the GOT.

The stub binder will look up the bind opcodes in the LazyBindingSection
at the given offset. The bind opcodes will tell the binder to update the
address in the LazyPointerSection to point to the symbol, so that
subsequent calls don't have to redo the symbol resolution. The binder
will then jump to the resolved symbol.

Depends on D78269.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78270
2020-05-09 20:56:22 -07:00
Jez Ng db157d2733 [lld-macho] Follow-up to D77893
Summary:
1. Don't have isHidden() depend on isNeeded(). Whether a section is
  hidden is orthogonal from whether it is needed: hidden sections will
  never have a header regardless of whether they have a body. (I know we
  override this method with return false for synthetic sections, but
  regardless I think it's confusing to write it this way for non-synthetic
  sections.)

2. Don't call writeTo() on unneeded sections. D78270 assumes that this
  is true when implementing the stub helper section.

3. Filter out the unneeded sections early on to avoid having to deal
   with them in multiple places.

4. Remove assumption in test that the referenced file has no other symbols.
  (We should create separate input files for future tests to avoid such
  issues.)

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79460
2020-05-09 20:56:22 -07:00
Reid Kleckner 77ecf90c52 [COFF] Migrate COFFObjectFile to Expected<T>
I noticed that std::error_code() does one-time initialization. Avoid
that overhead with Expected<T> and llvm::Error. Also, it is consistent
with the virtual interface and ELF, and generally cleaner.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D79643
2020-05-08 14:01:39 -07:00
Thomas Lively a1ae9566ea [WebAssembly] Disallow 'shared-mem' rather than 'atomics'
Summary:
The WebAssembly backend automatically lowers atomic operations and TLS
to nonatomic operations and non-TLS data when either are present and
the atomics or bulk-memory features are not present, respectively. The
resulting object is no longer thread-safe, so the linker has to be
told not to allow it to be linked into a module with shared
memory. This was previously done by disallowing the 'atomics' feature,
which prevented any objct with its atomic operations or TLS removed
from being linked with any object containing atomics or TLS, and
therefore preventing it from being linked into a module with shared
memory since shared memory requires atomics.

However, as of https://github.com/WebAssembly/threads/issues/144, the
validation rules are relaxed to allow atomic operations to validate
with unshared memories, which makes it perfectly safe to link an
object with stripped atomics and TLS with another object that still
contains TLS and atomics as long as the resulting module has an
unshared memory. To allow this kind of link, this patch disallows a
pseudo-feature 'shared-mem' rather than 'atomics' to communicate to
the linker that the object is not thread-safe. This means that the
'atomics' feature is available to accurately reflect whether or not an
object has atomics enabled.

As a drive-by tweak, this change also requires that bulk-memory be
enabled in addition to atomics in order to use shared memory. This is
because initializing shared memories requires bulk-memory operations.

Reviewers: aheejin, sbc100

Subscribers: dschuff, jgravelle-google, hiraditya, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79542
2020-05-08 13:52:39 -07:00
Wei Mi 538208f6c0 [lld] Add a new output section ".text.unknown" for funtions with unknown hotness
For sampleFDO, because the optimized build uses profile generated from previous
release, often we couldn't tell a function without profile was truely cold or
just newly created so we had to treat them conservatively and put them in .text
section instead of .text.unlikely. The result was when we persue the best
performance by locking .text.hot and .text in memory, we wasted a lot of memory
to keep cold functions inside. This problem has been largely solved for regular
sampleFDO using profile-symbol-list (https://reviews.llvm.org/D66374), but for
the case when we use partial profile, we still waste a lot of memory because
of it.

In https://reviews.llvm.org/D62540, we propose to save functions with unknown
hotness information in a special section called ".text.unknown", so that
compiler will treat those functions as luck-warm, but runtime can choose not
to mlock the special section in memory or use other strategy to save memory.
That will solve most of the memory problem even if we use a partial profile.

The patch adds the support in lld for the special section.For sampleFDO,
because the optimized build uses profile generated from previous release,
often we couldn't tell a function without profile was truely cold or just
newly created so we had to treat them conservatively and put them in .text
section instead of .text.unlikely. The result was when we persue the best
performance by locking .text.hot and .text in memory, we wasted a lot of
memory to keep cold functions inside. This problem has been largely solved
for regular sampleFDO using profile-symbol-list
(https://reviews.llvm.org/D66374), but for the case when we use partial
profile, we still waste a lot of memory because of it.

In https://reviews.llvm.org/D62540, we propose to save functions with unknown
hotness information in a special section called ".text.unknown", so that
compiler will treat those functions as luck-warm, but runtime can choose not
to mlock the special section in memory or use other strategy to save memory.
That will solve most of the memory problem even if we use a partial profile.

The patch adds the support in lld for the special section.

Differential Revision: https://reviews.llvm.org/D79590
2020-05-08 11:14:48 -07:00
Reid Kleckner 3b3e28a07c [PDB] Optimize public symbol processing
Reduces time to link PGO instrumented net_unittets.exe by 11% (9.766s ->
8.672s, best of three). Reduces peak memory by 65.7MB (2142.71MB ->
2076.95MB).

Use a more compact struct, BulkPublic, for faster sorting. Sort in
parallel. Construct the hash buckets in parallel. Try to use one vector
to hold all the publics instead of copying them from one to another.
Allocate all the memory needed to serialize publics up front, and then
serialize them in place in parallel.

Reviewed By: aganea, hans

Differential Revision: https://reviews.llvm.org/D79467
2020-05-08 10:23:27 -07:00
Fangrui Song e20a215992 [ELF] Add convenience TableGen classes to enforce two dashes for options not supported by GNU ld
Announced on https://lists.llvm.org/pipermail/llvm-dev/2020-May/141416.html

For many options, we have to support either one or two dash to be
compatible with GNU ld. For newer and lld specific options, we can enforce strict double dashes.

Affected options:

* --thinlto-*
* --lto-*
* --shuffle-sections=

This patch does not change `-plugin-opt=*` because clang driver passes
`-plugin-opt=*` and I don't intend to cause churn.

In 2000, GNU ld tried something similar with --omagic
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e4897a3288f37d5f69e8acd256a6e83e607fe8d8

Reviewed By: tejohnson, psmith

Differential Revision: https://reviews.llvm.org/D79371
2020-05-08 07:37:06 -07:00
Reid Kleckner d71c3c425c [COFF] Dump string table size for COFF file headers
I couldn't find this info in any other dumper, so it might as well be
here.
2020-05-06 15:48:36 -07:00
Sam Clegg f03b6e785b [lld][WebAssembly] Honor --allow-undefined for data symbols too
This was originally the way this worked before before
https://reviews.llvm.org/D60882.

In retrospect it seems inconsistent that `--allow-undefined` doesn't
work for all symbols.  See:
https://groups.google.com/g/emscripten-discuss/c/HSRgQiIq1gI/m/Kt9oFWHiAwAJ

I'm also planning a followup change which implement the full
`--unresolved-symbols=..` flags supported by ELF linkers (both ld and
ld.lld) since it seems more standard.

Differential Revision: https://reviews.llvm.org/D79247
2020-05-06 12:39:29 -07:00
Alexandre Ganea 6adc45d3fd [LLD][COFF] Move debug info for thread-local variables into PDB global stream
Before this patch, the debug record S_GTHREAD32 which represents global thread_local symbols, was emitted by LLD into the respective module stream. This makes Visual Studio unable to display thread_local symbols in the debugger.

After this patch, S_GTHREAD32 is moved into the globals stream. This matches MSVC behavior.

Differential Revision: https://reviews.llvm.org/D79005
2020-05-06 15:23:58 -04:00
Reid Kleckner 932f0276ea [Support] Move LLD's parallel algorithm wrappers to support
Essentially takes the lld/Common/Threads.h wrappers and moves them to
the llvm/Support/Paralle.h algorithm header.

The changes are:
- Remove policy parameter, since all clients use `par`.
- Rename the methods to `parallelSort` etc to match LLVM style, since
  they are no longer C++17 pstl compatible.
- Move algorithms from llvm::parallel:: to llvm::, since they have
  "parallel" in the name and are no longer overloads of the regular
  algorithms.
- Add range overloads
- Use the sequential algorithm directly when 1 thread is requested
  (skips task grouping)
- Fix the index type of parallelForEachN to size_t. Nobody in LLVM was
  using any other parameter, and it made overload resolution hard for
  for_each_n(par, 0, foo.size(), ...) because 0 is int, not size_t.

Remove Threads.h and update LLD for that.

This is a prerequisite for parallel public symbol processing in the PDB
library, which is in LLVM.

Reviewed By: MaskRay, aganea

Differential Revision: https://reviews.llvm.org/D79390
2020-05-05 15:21:05 -07:00
Sid Manning 0e6536fd97 [Hexagon] Add R_HEX_GD_PLT_B22/32_PCREL relocations
Extended versions of GD_PLT_B22_PCREL. These surface when -mlong-calls
is used.

Differential Revision: https://reviews.llvm.org/D79191
2020-05-05 11:47:51 -05:00
Peter Smith 48aebfc908 [ELF][ARM] Do not create .ARM.exidx sections for out of range inputs
A linker will create .ARM.exidx sections for InputSections that don't
have them. This can cause a relocation out of range error If the
InputSection happens to be extremely far away from the other sections.
This is often the case for the vector table on older ARM CPUs as the only
two places that the table can be placed is 0 or 0xffff0000. We fix this
by removing InputSections that need a linker generated .ARM.exidx
section if that would cause an error.

Differential Revision: https://reviews.llvm.org/D79289
2020-05-05 09:59:45 +01:00
Martin Storsjö 5a1c30177f [LLD] [COFF] Fix a typo in an assert message. NFC. 2020-05-05 11:46:50 +03:00
Zakk Chen ad5fad0ac5 [LTO] Suppress emission of empty combined module by default
Summary:
That unless the user requested an output object (--lto-obj-path), the an
unused empty combined module is not emitted.

This changed is helpful for some target (ex. RISCV-V) which encoded the
ABI info in IR module flags (target-abi). Empty unused module has no ABI
info so the linker would get the linking error during merging
incompatible ABIs.

Reviewers: tejohnson, espindola, MaskRay

Subscribers: emaste, inglorion, arichardson, hiraditya, simoncook, MaskRay, steven_wu, dexonsmith, PkmX, dang, lenary, s.egerton, luismarques, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78988
2020-05-04 18:31:09 -07:00
Reid Kleckner 2868ee5b32 [PDB] Use the global BumpPtrAllocator
Profiling shows that time is spent destroying the allocator member of
PDBLinker, and that is unneeded.
2020-05-04 16:15:36 -07:00
Fangrui Song 6939fe6e08 [lld-macho] Support X86_64_RELOC_SIGNED_{1,2,4}
We currently only support extern relocations.
`X86_64_RELOC_SIGNED_{1,2,4}` are like X86_64_RELOC_SIGNED, but with the
implicit addend fixed to 1, 2, and 4, respectively.
See the comment in `lib/Target/X86/MCTargetDesc/X86MachObjectWriter.cpp RecordX86_64Relocation`.

Reviewed By: int3

Differential Revision: https://reviews.llvm.org/D79311
2020-05-04 15:15:35 -07:00
Fangrui Song c49f83b6e9 [ELF] Don't advance sh_offset for an empty section whose PT_LOAD is removed (due to p_memsz=0)
removeEmptyPTLoad() removes empty (p_memsz=0) PT_LOAD segments.  In
assignFileOffsets(), setFileOffset() unnecessarily advances file offsets
for containing empty sections.

This is exposed by arm Linux kernel's multi_v5_defconfig
(see https://bugs.llvm.org/show_bug.cgi?id=45632)

```
ld.lld (max-page-size=65536):
  [34] .init.data        PROGBITS        c0c24000 c34000 0128ac 00  WA  0   0 4096
  [35] .text_itcm        PROGBITS        fffe0000 c50000 000000 00  WA  0   0  1
  [36] .data_dtcm        PROGBITS        fffe8000 c58000 000000 00  WA  0   0  1
  [37] .data             PROGBITS        c0c38000 c58000 0647a0 00  WA  0   0 32

arm-linux-gnueabi-ld (max-page-size=65536):
  [23] .init.data        PROGBITS        c0c12000 c22000 0128ac 00  WA  0   0 4096
  [24] .text_itcm        PROGBITS        fffe0000 ca2558 000000 00   W  0   0  1
  [25] .data_dtcm        PROGBITS        fffe8000 ca2558 000000 00   W  0   0  1
  [26] .data             PROGBITS        c0c26000 c36000 0647a0 00  WA  0   0 32
```

This patch clears OutputSection::ptLoad if ptLoad is removed by
removeEmptyPTLoad(). Conceptually this removes "dangling" references.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D79254
2020-05-04 08:07:34 -07:00
Reid Kleckner fce5457a14 [COFF] Avoid allocating temporary vectors during ICF
Heap profiling with ETW shows that LLD performs 4,053,721 heap
allocations over its lifetime, and ~800,000 of them come from
assocEquals. These vectors are created just to do a comparison, so fuse
the comparison into the loop and avoid the allocation.

ICF is overall a small portion of the time spent linking, and I did not
measure overall throughput improvements from this change above the noise
threshold. However, these show up in the heap profiler, and the work is
done, so we might as well land it if the code is clear enough.

Reviewed By: hans

Differential Revision: https://reviews.llvm.org/D79297
2020-05-04 07:01:14 -07:00
Peter Smith 3834385f27 [ELF] Move SHF_LINK_ORDER till OutputSection addresses are known
Sections with the SHF_LINK_ORDER flag must be ordered in the same relative
order as the Sections they have a link to. When using a linker script an
arbitrary expression may be used for the virtual address of the
OutputSection. In some cases the virtual address does not monotonically
increase as the OutputSection index increases, so if we base the ordering
of the SHF_LINK_ORDER sections on the index then we can get the order
wrong. We fix this by moving SHF_LINK_ORDER resolution till after we have
created OutputSection virtual addresses.

Differential Revision: https://reviews.llvm.org/D79286
2020-05-04 14:25:25 +01:00
Reid Kleckner 9b7f6146bd [COFF] Paritally inline Symbol::getName, NFC 2020-05-03 07:58:05 -07:00
Reid Kleckner 1e5793345b Re-land "[PDB] Avoid calling discoverTypeIndices for a known record kind"
Fixed bad usage of slice API causing assertion failures.

Reverts 810c8e9b49
Reinstates bd7ea8641e
2020-05-02 18:39:33 -07:00
Nico Weber 810c8e9b49 Revert "[PDB] Avoid calling discoverTypeIndices for a known record kind"
This reverts commit bd7ea8641e.
Breaks check-lld everywhere.
2020-05-02 21:06:06 -04:00
Reid Kleckner bd7ea8641e [PDB] Avoid calling discoverTypeIndices for a known record kind
This particular overload allocates memory, and we do this for every
S_[GL]PROC32_ID record. Instead, hardcode the offset of the typeindex
that we are looking for in the LF_[MEM]FUNC_ID record. We already
assumed that looking up the item index already found a record of this
kind.
2020-05-02 15:51:08 -07:00
Reid Kleckner 3542384ae9 [COFF] Use a global option table to avoid reconstructing it
Otherwise an ArgumentParser is constructed for every directive section,
and that involves copying the entire table of options into a vector.
There is no need for this, just have one option table.
2020-05-02 15:04:19 -07:00
Thomas Preud'homme d735c7048c [test] Fix lld's ELF/linkerscript/thunk-gen-mips.s
Summary:
Lld test ELF/linkerscript/thunk-gen-mips.s was accidentally disabled due
to the use of wrong FileCheck directives. As a result the test seems to
have bitrotted as it fails to pass if fixing the directive. To ease
updates to the test in case of change of the __start address the checks
have been changed to use numeric variables to express all the addresses
based on the __start address.

Reviewed By: atanasyan

Differential Revision: https://reviews.llvm.org/D79270
2020-05-02 22:49:23 +01:00
Reid Kleckner 270d3faf6e [COFF] Add and use a zero-copy tokenizer for .drectve
This generalizes the main Windows command line tokenizer to be able to
produce StringRef substrings as well as freshly copied C strings. The
implementation is still shared with the normal tokenizer, which is
important, because we have unit tests for that.

.drective sections can be very long. They can potentially list up to
every symbol in the object file by name. It is worth avoiding these
string copies.

This saves a lot of memory when linking chrome.dll with PGO
instrumentation:

             BEFORE      AFTER      % IMP
peak memory: 6657.76MB   4983.54MB  -25%
real:        4m30.875s   2m26.250s  -46%

The time improvement may not be real, my machine was noisy while running
this, but that the peak memory usage improvement should be real.

This change may also help apps that heavily use dllexport annotations,
because those also use linker directives in object files. Apps that do
not use many directives are unlikely to be affected.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D79262
2020-05-02 10:47:02 -07:00
Kellie Medlin 6cb073133c [lld] Merge Mach-O input sections
Summary: Similar to other formats, input sections in the MachO
implementation are now grouped under output sections. This is primarily
a refactor, although there's some new logic (like resolving the output
section's flags based on its inputs).

Differential Revision: https://reviews.llvm.org/D77893
2020-05-01 16:57:18 -07:00
Fangrui Song 5c86b08a6f [ELF][test] Improve tests
Prepare for the upcomong change that removes unneeded sh_offset
advancement for empty sections whose PT_LOAD are removed.
2020-05-01 11:27:51 -07:00
Sam Clegg 0a6c4d8d2e [WebAssmebly] Add support for defined wasm globals in MC and lld
This change add support for defined wasm globals in the .s format,
the MC layer, and wasm-ld

Currently there is no support custom initialization and all wasm
globals are initialized to zero.

Fixes: PR45742

Differential Revision: https://reviews.llvm.org/D79137
2020-04-30 12:43:15 -07:00
Thomas Preud'homme 9ecddde321 [test] Fix ELF/linkerscript/input-archive.s w/ @ in path
Lld test ELF/linkerscript/input-archive.s fails when path contain a @
because is not accepted in unquoted token in linker scripts which leads
to the path being broken in 2 around the @. This commit quotes the path
used in the linker script created by this and similar testcases allowing
the test to pass even in the presence of an @ sign in the path.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D79103
2020-04-30 20:14:22 +01:00
Fangrui Song b257d3c8a8 [ELF][PPC64] Suppress toc-indirect to toc-relative relaxation if R_PPC64_TOC16_LO is seen
The current implementation assumes that R_PPC64_TOC16_HA is always followed
by R_PPC64_TOC16_LO_DS. This can break with R_PPC64_TOC16_LO:

  // Load the address of the TOC entry, instead of the value stored at that address
  addis 3, 2, .LC0@tloc@ha  # R_PPC64_TOC16_HA
  addi  3, 3, .LC0@tloc@l   # R_PPC64_TOC16_LO
  blr

which is used by boringssl's util/fipstools/delocate/delocate.go
https://github.com/google/boringssl/blob/master/crypto/fipsmodule/FIPS.md has some documentation.
In short, this tool converts an assembly file to avoid any potential relocations.
The distance to an input .toc is not a constant after linking, so it cannot use an `addis;ld` pair.
Instead, it jumps to a stub which loads the TOC entry address with `addis;addi`.

This patch checks the presence of R_PPC64_TOC16_LO and suppresses
toc-indirect to toc-relative relaxation if R_PPC64_TOC16_LO is seen.
This approach is conservative and loses some relaxation opportunities but is easy to implement.

  addis 3, 2, .LC0@toc@ha  # no relaxation
  addi  3, 3, .LC0@toc@l   # no relaxation
  li    9, 0
  addis 4, 2, .LC0@toc@ha  # can relax but suppressed
  ld    4, .LC0@toc@l(4)   # can relax but suppressed

Also note that interleaved R_PPC64_TOC16_HA and R_PPC64_TOC16_LO_DS is
possible and this patch accounts for that.

  addis 3, 2, .LC1@toc@ha  # can relax
  addis 4, 2, .LC2@toc@ha  # can relax
  ld    3, .LC1@toc@l(3)   # can relax
  ld    4, .LC2@toc@l(4)   # can relax

Reviewed By: #powerpc, sfertile

Differential Revision: https://reviews.llvm.org/D78431
2020-04-30 09:16:51 -07:00
Fangrui Song b912b887d8 [ELF] Add --print-archive-stats=
gold has an option --print-symbol-counts= which prints:

  // For each archive
  archive $archive $members $fetched_members
  // For each object file
  symbols $object $defined_symbols $used_defined_symbols

In most cases, `$defined_symbols = $used_defined_symbols` unless weak
symbols are present. Strangely `$used_defined_symbols` includes symbols defined relative to --gc-sections discarded sections.
The `symbols` lines do not appear to be useful.

`archive` lines are useful: `$fetched_members=0` lines correspond to
unused archives. The information can be used to trim dependencies.

This patch implements --print-archive-stats= which prints the number of
members and the number of fetched members for each archive.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D78983
2020-04-29 18:04:37 -07:00
Fangrui Song e96d7b5e9e [ELF] Add --rosegment to complement --no-rosegment
This option can cancel --no-rosegment and it just seems right to have
a corresponding positive option for a --no-* negative option.

Anecdotally, gold had --rosegment but did not have --no-rosegment.
I added --no-rosegment (https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=9a6c68caa9543e09b064b7ac7c2b658f277bc19c) for binutils>=2.35
2020-04-29 18:00:00 -07:00
Jez Ng e82c5e17b5 [lld-macho] Support X86_64_RELOC_BRANCH
Relatively straightforward diff, to set the stage for calling functions
in dylibs.

Differential Revision: https://reviews.llvm.org/D78269
2020-04-29 15:45:01 -07:00
Jez Ng df92377823 [lld-macho] Have Symbol::getVA() return a non-relative virtual address
Currently, getVA() returns a virtual address with the assumption that
the ImageBase is zero. As I understand, this is what lld-ELF is doing.
However, under our current design, it seems like an awkward setup --
I'm finding that I have to add and subtract ImageBase in several places
to make things work out.

As such, I think it's simpler to have getVA() return a non-relative VA,
but I'm not sure if I'm missing something. Would love to hear more from
folks familiar with lld-ELF.

Differential Revision: https://reviews.llvm.org/D78168
2020-04-29 15:44:50 -07:00
Jez Ng 918948db4d [lld-macho] Support reading of universal binaries
Differential Revision: https://reviews.llvm.org/D77006
2020-04-29 15:44:44 -07:00
Jez Ng 89285a1a97 [lld-macho] Disable colors in errors when not printing to a pty
This makes for better tests (and is just the right thing to do)

Differential Revision: https://reviews.llvm.org/D79069
2020-04-29 15:44:35 -07:00
Jez Ng 9854edd817 [lld-macho] Implement basic export trie
Build the trie by performing a three-way radix quicksort: We start by
sorting the strings by their first characters, then sort the strings
with the same first characters by their second characters, and so on
recursively. Each time the prefixes diverge, we add a node to the trie.
Thanks to @ruiu for the idea.

I used llvm-mc's radix quicksort implementation as a starting point. The
trie offset fixpoint code was taken from
MachONormalizedFileBinaryWriter.cpp.

Differential Revision: https://reviews.llvm.org/D76977
2020-04-29 15:44:27 -07:00
Fangrui Song 1ccde53342 [ELF] --gdb-index: support .debug_loclists
--gdb-index currently crashes when reading a translation unit with
DWARF v5 .debug_loclists . Call stack:

```
SyntheticSections.cpp GdbIndexSection::create
SyntheticSections.cpp readAddressAreas
DWARFUnit.cpp DWARFUnit::tryExtractDIEsIfNeeded
DWARFListTable.cpp DWARFListTableHeader::extract
...
DWARFDataExtractor.cpp DWARFDataExtractor::getRelocatedValue
lld/ELF/DWARF.cpp LLDDwarfObj<ELFT>::find (sec.sec is nullptr)
...

```

This patch adds support for .debug_loclists to make `DWARFUnit::tryExtractDIEsIfNeeded` happy.
Building --gdb-index does not need .debug_loclists

Reviewed By: dblaikie, grimar

Differential Revision: https://reviews.llvm.org/D79061
2020-04-29 15:04:13 -07:00
Fangrui Song 53ff95254d Reland D78837 [lld] Remove special cases from default ld driver mode.
Drops the behavior from rL217112.

Use the Gnu driver mode by default for all platforms when ld is
invoked. Other names for the program (such as link or ld64) continue
working as before.

Reviewed By: MaskRay, srhines, smeenai, ruiu

Differential Revision: https://reviews.llvm.org/D78837
2020-04-29 14:45:44 -07:00
Dan Albert 0a78e42b1f Revert "[lld] Remove special cases from default ld driver mode."
This reverts commit da093c388f.

Broke a test on Darwin. Will fix the test and resubmit.
2020-04-29 14:14:51 -07:00
Dan Albert da093c388f [lld] Remove special cases from default ld driver mode.
Summary:
Use the Gnu driver mode by default for all platforms when ld is
invoked. Other names for the program (such as link or ld64) continue
working as before.

Reviewers: MaskRay, int3, srhines, smeenai, ruiu

Reviewed By: MaskRay, srhines, smeenai, ruiu

Subscribers: smeenai, srhines, nickdesaulniers, llvm-commits

Tags: #lld, #llvm

Differential Revision: https://reviews.llvm.org/D78837
2020-04-29 12:28:29 -07:00
Sean Fertile f9106e85c4 Revert "[ELF][PPC64] Don't perform toc-indirect to toc-relative relax... "
This reverts commit 03ffe58605.

Full tile of reverted commit is:
[ELF][PPC64] Don't perform toc-indirect to toc-relative relaxation for
R_PPC64_TOC16_HA not followed by R_PPC64_TOC16_LO_DS

Breaks the multistage lld PowerPC buildbot.
2020-04-29 10:30:35 -04:00
Saleem Abdulrasool 216833b32b Revert "Temporarily revert "build: use `find_package(Python3)` if available""
This reverts commit 35edd704e0.

Revert the revert and extend the patch further to account for the use of
the `PYTHONINTERP_FOUND`.
2020-04-29 01:38:08 +00:00
Jez Ng 62b8f32f76 [lld-macho][reland] Add support for emitting dylibs with a single symbol
This got reverted due to UBSAN errors in a diff lower in the stack,
which is being fixed in https://reviews.llvm.org/D79050. This diff is
otherwise identical to the original https://reviews.llvm.org/D76908
(which was committed in 9598778bd1 and reverted in b52bc2653b).

Differential Revision: https://reviews.llvm.org/D79051
2020-04-28 17:08:32 -07:00
Jez Ng 4f0cccdd7a [lld-macho][reland] Add basic symbol table output
This diff implements basic support for writing a symbol table.

Attributes are loosely supported for extern symbols and not at all for
other types.

Initial version by Kellie Medlin <kelliem@fb.com>

Originally committed in a3d95a50ee and reverted in fbae153ca5 due to
UBSAN erroring over unaligned writes. That has been fixed in the
current diff with the following changes:

```
diff --git a/lld/MachO/SyntheticSections.cpp b/lld/MachO/SyntheticSections.cpp
--- a/lld/MachO/SyntheticSections.cpp
+++ b/lld/MachO/SyntheticSections.cpp
@@ -133,6 +133,9 @@ SymtabSection::SymtabSection(StringTableSection &stringTableSection)
     : stringTableSection(stringTableSection) {
   segname = segment_names::linkEdit;
   name = section_names::symbolTable;
+  // TODO: When we introduce the SyntheticSections superclass, we should make
+  // all synthetic sections aligned to WordSize by default.
+  align = WordSize;
 }

 size_t SymtabSection::getSize() const {
diff --git a/lld/MachO/Writer.cpp b/lld/MachO/Writer.cpp
--- a/lld/MachO/Writer.cpp
+++ b/lld/MachO/Writer.cpp
@@ -371,6 +371,7 @@ void Writer::assignAddresses(OutputSegment *seg) {
     ArrayRef<InputSection *> sections = p.second;
     for (InputSection *isec : sections) {
       addr = alignTo(addr, isec->align);
+      // We must align the file offsets too to avoid misaligned writes of
+      // structs.
+      fileOff = alignTo(fileOff, isec->align);
       isec->addr = addr;
       addr += isec->getSize();
       fileOff += isec->getFileSize();
@@ -396,6 +397,7 @@ void Writer::writeSections() {
     uint64_t fileOff = seg->fileOff;
     for (auto &sect : seg->getSections()) {
       for (InputSection *isec : sect.second) {
+        fileOff = alignTo(fileOff, isec->align);
         isec->writeTo(buf + fileOff);
         fileOff += isec->getFileSize();
       }
```

I don't think it's easy to write a test for alignment (that doesn't
involve brittly hard-coding file offsets), so there isn't one... but
UBSAN builds pass now.

Differential Revision: https://reviews.llvm.org/D79050
2020-04-28 17:07:06 -07:00
Eric Christopher 35edd704e0 Temporarily revert "build: use `find_package(Python3)` if available"
as it seems to be causing multiple people problems with running tests
and building.

This reverts commit c4c3883b00.
2020-04-28 16:41:22 -07:00
Fangrui Song 03ffe58605 [ELF][PPC64] Don't perform toc-indirect to toc-relative relaxation for R_PPC64_TOC16_HA not followed by R_PPC64_TOC16_LO_DS
The current implementation assumes that R_PPC64_TOC16_HA is always followed
by R_PPC64_TOC16_LO_DS. This can break with:

// Load the address of the TOC entry, instead of the value stored at that address
  addis 3, 2, .LC0@tloc@ha  # R_PPC64_TOC16_HA
  addi  3, 3, .LC0@tloc@l   # R_PPC64_TOC16_LO
  blr

which is used by boringssl's util/fipstools/delocate/delocate.go
https://github.com/google/boringssl/blob/master/crypto/fipsmodule/FIPS.md has some documentation.
In short, this tool converts an assembly file to avoid any potential relocations.
The distance to an input .toc is not a constant after linking, so the assembly cannot use an `addis;ld` pair.
Instead, delocate changes the code to jump to a stub (`addis;addi`) which loads the TOC entry address.

Reviewed By: sfertile

Differential Revision: https://reviews.llvm.org/D78431
2020-04-28 12:13:27 -07:00
Shoaib Meenai fbae153ca5 Revert "[lld-macho] Add basic symbol table output"
This reverts commit a3d95a50ee.

Reverting due to UBSan failures:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/40817/steps/check-lld%20ubsan/logs/stdio
2020-04-28 11:34:03 -07:00
Shoaib Meenai b52bc2653b Revert "[lld-macho] Add support for emitting dylibs with a single symbol"
This reverts commit 9598778bd1.

Reverting due to UBSan failures:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/40817/steps/check-lld%20ubsan/logs/stdio
2020-04-28 11:34:03 -07:00
Shoaib Meenai af40bff32d [MachO] Fix UB in memcpy
UBSan complains about a memcpy with a null pointer, so just skip the
memcpy call if the data is empty.
2020-04-28 11:33:54 -07:00
Reid Kleckner 01b5f52140 [COFF] Add a fastpath for /INCLUDE: in .drective sections
This speeds up linking chrome.dll with PGO instrumentation by 13%
(154271ms -> 134033ms).

LLVM's Option library is very slow. In particular, it allocates at least
one large-ish heap object (Arg) for every argument. When PGO
instrumentation is enabled, all the __profd_* symbols are added to the
@llvm.used list, which compiles down to these /INCLUDE: directives. This
means we have O(#symbols) directives to parse in the section, so we end
up allocating an Arg for every function symbol in the object file. This
is unnecessary.

To address the issue and speed up the link, extend the fast path that we
already have for /EXPORT:, which has similar scaling issues.

I promise that I took a hard look at optimizing the Option library, but
its data structures are very general and would need a lot of cleanup. We
have accumulated lots of optional features (option groups, aliases,
multiple values) over the years, and these are now properties of every
parsed argument, when the vast majority of arguments do not use these
features.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D78845
2020-04-28 10:35:57 -07:00
Fangrui Song d9786b566b [ELF] Clear lazyObjFiles in lld:🧝:link after D46034 2020-04-28 09:54:20 -07:00
Saleem Abdulrasool c4c3883b00 build: use `find_package(Python3)` if available
This is primarily motivated by the desire to move from Python2 to
Python3.  `PYTHON_EXECUTABLE` is ambiguous.  This explicitly identifies
the python interpreter in use.  Since the LLVM build seems to be able to
completed successfully with python3, use that across the build.  The old
path aliases `PYTHON_EXECUTABLE` to be treated as Python3.
2020-04-28 09:24:27 -07:00
Igor Kudrin 9f65f5acca [LLD][ELF] Eliminate symbols of merged .ARM.exidx sections.
GNU tools generate mapping symbols "$d" for .ARM.exidx sections. The
symbols are added to the symbol table much earlier than the merging
takes place, and after that, they become dangling. Before the patch,
LLD output those symbols as SHN_ABS with the value of 0. The patch
removes such symbols from the symbol table.

Differential Revision: https://reviews.llvm.org/D78820
2020-04-28 18:58:40 +07:00
Jez Ng 9598778bd1 [lld-macho] Add support for emitting dylibs with a single symbol
Summary:
Add logic for emitting the correct set of load commands and segments
when `-dylib` is passed.

I haven't gotten to implementing a real export trie yet, so we can only
emit a single symbol, but it's enough to replace the YAML test files
introduced in D76252.

Differential Revision: https://reviews.llvm.org/D76908
2020-04-27 13:33:46 -07:00
Jez Ng a3d95a50ee [lld-macho] Add basic symbol table output
This diff implements basic support for writing a symbol table.

- Attributes are loosely supported for extern symbols and not at all for
  other types

Immediate future work will involve implementing section merging.

Initial version by Kellie Medlin <kelliem@fb.com>

Differential Revision: https://reviews.llvm.org/D76742
2020-04-27 13:33:15 -07:00
Jez Ng 6f63216c3d [lld-macho] Extend SyntheticSections to cover all segment load commands
Previously, the special segments `__PAGEZERO` and `__LINKEDIT` were
implemented as special LoadCommands. This diff implements them using
special sections instead which have an `isHidden()` attribute. We do not
emit section headers for hidden sections, but we use their addresses and
file offsets to determine that of their containing segments. In addition
to allowing us to share more segment-related code, this refactor is also
important for the next step of emitting dylibs:

1) dylibs don't have segments like __PAGEZERO, so we need an easy way of
   omitting them w/o messing up segment indices
2) Unlike the kernel, which is happy to run an executable with
   out-of-order segments, dyld requires dylibs to have their segment
   load commands arranged in increasing address order. The refactor
   makes it easier to implement sorting of sections and segments.

Differential Revision: https://reviews.llvm.org/D76839
2020-04-27 12:58:12 -07:00
Nico Weber 76f74d15fe remove temporary logging again now that things are figured out 2020-04-27 14:01:37 -04:00
Hongtao Yu 964ef8eecc [lld] Support --lto-emit-asm and --plugin-opt=emit-asm
Summary: The switch --plugin-opt=emit-asm can be used with the gold linker to dump the final assembly code generated by LTO in a user-friendly way. Unfortunately it doesn't work with lld. I'm hooking it up with lld. With that switch, lld emits assembly code into the output file (specified by -o) and if there are multiple input files, each of their assembly code will be emitted into a separate file named by suffixing the output file name with a unique number, respectively. The linking then stops after generating those assembly files.

Reviewers: espindola, wenlei, tejohnson, MaskRay, grimar

Reviewed By: tejohnson, MaskRay, grimar

Subscribers: pcc, emaste, inglorion, arichardson, hiraditya, MaskRay, steven_wu, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77231
2020-04-27 11:00:46 -07:00
Simon Pilgrim 0847cfa334 [lld][macho] Fix implicit dependency on DenseMap.h include
It was depending on CachedHashString.h providing the include.
2020-04-27 14:05:29 +01:00
Sam Clegg 4b8e2d8e81 [lld][WebAssembly] Fix crash on function signature mismatch with --relocatable
These stub new function were not being added to the symbol table
which in turn meant that we were crashing when trying to output
relocations against them.

Differential Revision: https://reviews.llvm.org/D78779
2020-04-25 10:26:11 -07:00
Igor Kudrin 66e4eb9c1b [LLD][ELF] Implement --discard-* for cases when -r or --emit-relocs are used.
When discarding local symbols with --discard-all or --discard-locals,
the ones which are used in relocations should be preserved. LLD used
the simplest approach and just ignored those switches when -r or
--emit-relocs was specified.

The patch implements handling the --discard-* switches for the cases
when relocations are kept by identifying used local symbols and allowing
removing only unused ones. This makes the behavior of LLD compatible
with GNU linkers.

Differential Revision: https://reviews.llvm.org/D77807
2020-04-25 18:59:41 +07:00
Peter Smith 3b1622d63a [LLD][ELF][ARM] recommit Fix ARM Exidx order for non monotonic section order
Fixed error detected by msan. The size field of the .ARM.exidx synthetic
section needs to be initialized to at least estimation level before
calling assignAddresses as that will use the size field.

This was previously reverted in 1ca16fc4f5.

Differential Revision: https://reviews.llvm.org/D78422
2020-04-24 13:47:28 +01:00
Peter Smith 1ca16fc4f5 Revert "[LLD][ELF][ARM] Fix ARM Exidx order for non monotonic section order"
This reverts commit f969c2aa65.

There are some msan buildbot failures sanitzer-x86_64-linux-fast that
I need to investigate.

Differential Revision: https://reviews.llvm.org/D78422
2020-04-23 16:58:50 +01:00
Peter Smith f969c2aa65 [LLD][ELF][ARM] Fix ARM Exidx order for non monotonic section order
The contents of the .ARM.exidx section must be ordered by SHF_LINK_ORDER
rules. We don't need to know the precise address for this order, but we
do need to know the relative order of sections. We have been using the
sectionIndex for this purpose, this works when the OutputSection order
has a monotonically increasing virtual address, but it is possible to
write a linker script with non-monotonically increasing virtual address.
For these cases we need to evaluate the base address of the OutputSection
so that we can order the .ARM.exidx sections properly.

This change moves the finalisation of .ARM.exidx till after the first
call to AssignAddresses. This permits us to sort on virtual address which
is linker script safe. It also permits a fix for part of pr44824 where
we generate .ARM.exidx section for the vector table when that table is so
far away it is out of range of the .ARM.exidx section. This fix will come
in a follow up patch.

Differential Revision: https://reviews.llvm.org/D78422
2020-04-23 15:46:44 +01:00
Sergej Jaskiewicz 2899103108 [TimeProfiler] Emit clock synchronization point
Time profiler emits relative timestamps for events (the number of
microseconds passed since the start of the current process).

This patch allows combining events from different processes while
preserving their relative timing by emitting a new attribute
"beginningOfTime". This attribute contains the system time that
corresponds to the zero timestamp of the time profiler.

This has at least two use cases:

- Build systems can use this to merge time traces from multiple compiler
  invocations and generate statistics for the whole build. Tools like
  ClangBuildAnalyzer could also leverage this feature.

- Compilers that use LLVM as their backend by invoking llc/opt in
  a child process. If such a compiler supports generating time traces
  of its own events, it could merge those events with LLVM-specific
  events received from llc/opt, and produce a more complete time trace.

A proof-of-concept script that merges multiple logs that
contain a synchronization point into one log:
https://github.com/broadwaylamb/merge_trace_events

Differential Revision: https://reviews.llvm.org/D78030
2020-04-23 01:09:31 +03:00
Sergej Jaskiewicz a5bf02815d [TimeProfiler] Emit real process ID and thread names
Differential Revision: https://reviews.llvm.org/D78027
2020-04-23 00:12:51 +03:00
Fangrui Song c384ca3c6a [ELF] For relative paths in INPUT() and GROUP(), search the directory of the current linker script before searching other paths
For a relative path in INPUT() or GROUP(), this patch changes the search order by adding the directory of the current linker script.
The new search order (consistent with GNU ld >= 2.35 regarding the new test `test/ELF/input-relative.s`):

1. the directory of the current linker script (GNU ld from Binutils 2.35 onwards; https://sourceware.org/bugzilla/show_bug.cgi?id=25806)
2. the current working directory
3. library paths (-L)

This behavior makes it convenient to replace a .so or .a with a linker script with additional input. For example, glibc

```
% cat /usr/lib/x86_64-linux-gnu/libm.a
/* GNU ld script
*/
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /usr/lib/x86_64-linux-gnu/libm-2.29.a /usr/lib/x86_64-linux-gnu/libmvec.a )
```

could be simplified as `GROUP(libm-2.29.a libmvec.a)`.

Another example is to make libc++.a a linker script:
```
INPUT(libc++.a.1 libc++abi.a)
```

Note, -l is not affected.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D77779
2020-04-22 12:34:20 -07:00
Peter Smith 673612a035 [LLD][ELF][ARM] Replace adr, ldr with .inst .reloc in test [NFC]
After D78301 MC no longer emits a relocation for this case. Change to use
.inst and .reloc to synthesize the same instruction and relocation. One
more test case I missed.
2020-04-22 12:55:26 +01:00
Fangrui Song 01d2a01e79 [ELF] Fix a null pointer dereference when relocating a Local-Exec TLS relocation for a lazy symbol
If there is no SHF_TLS section, there will be no PT_TLS and Out::tlsPhdr may be a nullptr.
If the symbol referenced by an R_TLS is lazy, we should treat the symbol as undefined.

Also reorganize tls-in-archive.s and tls-weak-undef.s . They do not test what they intended to test.
2020-04-21 15:39:31 -07:00
Jez Ng 060efd24c7 [lld-macho] Add basic support for linking against dylibs
This diff implements:

* dylib loading (much of which is being restored from @pcc and @ruiu's
  original work)
* The GOT_LOAD relocation, which allows us to load non-lazy dylib
  symbols
* Basic bind opcode emission, which tells `dyld` how to populate the GOT

Differential Revision: https://reviews.llvm.org/D76252
2020-04-21 13:43:19 -07:00
Fangrui Song 497c76e96d [ELF] Keep local symbols when both --emit-relocs and --discard-all are specified
This fixes a bug as exposed by D77807.

Add tests for {--emit-relocs,-r} x {--discard-locals,--discard-all}. They add coverage for previously undertested cases:

* STT_SECTION associated to GCed sections (`gc`)
* STT_SECTION associated to retained sections (`text`)
* STT_SECTION associated to non-SHF_ALLOC sections (`.comment`)
* STB_LOCAL in GCed sections (`unused_gc`)

Reviewed By: grimar, ikudrin

Differential Revision: https://reviews.llvm.org/D78389
2020-04-21 08:28:12 -07:00
Fangrui Song 58207d6fe1 [ELF] Fix "TLS attribute mismatch" false positives for STT_NOTYPE undefined symbols
D13550 added the diagnostic to address/work around a crash.
The rule was refined by D19836 (test/ELF/tls-archive.s) to exclude Lazy symbols.

https://bugs.llvm.org/show_bug.cgi?id=45598 reported another case where the current logic has a false positive:

Bitcode does not record undefined module-level inline assembly symbols
(`IRSymtab.cpp:Builder::addSymbol`). Such an undefined symbol does not
have the FB_tls bit and lld will not consider it STT_TLS. When the symbol is
later replaced by a STT_TLS Defined, lld will error "TLS attribute mismatch".

This patch fixes this false positive by allowing a STT_NOTYPE undefined
symbol to be replaced by a STT_TLS.

Considered alternative:

Moving the diagnostics to scanRelocs() can improve the diagnostics (PR36049)
but that requires a fair amount of refactoring. We will need more
RelExpr members. It requires more thoughts whether it is worthwhile.

See `test/ELF/tls-mismatch.s` for behavior differences. We will fail to
diagnose a likely runtime bug (STT_NOTYPE non-TLS relocation referencing
a TLS definition). This is probably acceptable because compiler
generated code sets symbol types properly.

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D78438
2020-04-21 07:56:35 -07:00
Fangrui Song 6e1fe78569 [ELF][test] Reorganize "TLS attribute mismatch" tests 2020-04-21 07:56:35 -07:00
Eli Friedman 875e26d343 Fix lld testcases I missed in D78413 (9b9454a) 2020-04-20 12:48:57 -07:00
Fangrui Song 232578804a [ELF] Add --warn-backrefs-exclude=<glob>
D77522 changed --warn-backrefs to not warn for linking sandwich
problems (-ldef1 -lref -ldef2). This removed lots of false positives.

However, glibc still has some problems. libc.a defines some symbols
which are normally in libm.a and libpthread.a, e.g. __isnanl/raise.

For a linking order `-lm -lpthread -lc`, I have seen:

```
// different resolutions: GNU ld/gold select libc.a(s_isnan.o) as the definition
backward reference detected: __isnanl in libc.a(printf_fp.o) refers to libm.a(m_isnanl.o)

// different resolutions: GNU ld/gold select libc.a(raise.o) as the definition
backward reference detected: raise in libc.a(abort.o) refers to libpthread.a(pt-raise.o)
```

To facilitate deployment of --warn-backrefs, add --warn-backrefs-exclude= so that
certain known issues (which may be impractical to fix) can be whitelisted.

Deliberate choices:

* Not a comma-separated list (`--warn-backrefs-exclude=liba.a,libb.a`).
  -Wl, splits the argument at commas, so we cannot use commas.
  --export-dynamic-symbol is similar.
* Not in the style of `--warn-backrefs='*' --warn-backrefs=-liba.a`.
  We just need exclusion, not inclusion. For easier build system
  integration, we should avoid order dependency. With the current
  scheme, we enable --warn-backrefs, and indivial libraries can add
  --warn-backrefs-exclude=<glob> to their LDFLAGS.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D77512
2020-04-20 07:52:15 -07:00
Peter Smith 61bccda9d9 [LLD][ELF][ARM] Convert ADR/LDR to .inst .reloc
After D78301 MC no longer emits a relocation for this case. Change to use
.inst and .reloc to synthesize the same instruction and relocation.
2020-04-20 13:59:05 +01:00
Nico Weber 2f2667b4ef add more temporary logging for a bot-only failure 2020-04-18 20:22:52 -04:00
Tobias Hieta 87383e408d [ELF][ARM] Increase default max-page-size from 4096 to 6536
See http://lists.llvm.org/pipermail/llvm-dev/2020-April/140549.html

For the record, GNU ld changed to 64k max page size in 2014
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=7572ca8989ead4c3425a1500bc241eaaeffa2c89
"[RFC] ld/ARM: Increase maximum page size to 64kB"

Android driver forced 4k page size in AArch64 (D55029) and ARM (D77746).

A binary linked with max-page-size=4096 does not run on a system with a
higher page size configured. There are some systems out there that do
this and it leads to the binary getting `Killed!` by the kernel.

In the non-linker-script cases, when linked with -z noseparate-code
(default), the max-page-size increase should not cause any size
difference. There may be some VMA usage differences, though.

Reviewed By: psmith, MaskRay

Differential Revision: https://reviews.llvm.org/D77330
2020-04-18 08:19:45 -07:00
Nico Weber c7c3b877c8 add temporary logging to help diagnose a bot-only failure 2020-04-17 22:06:37 -04:00
Reid Kleckner 91a6bfed61 [COFF] Assign unique identifiers to ObjFiles from LTO
Use the unique filenames that are used when /lldsavetemps is passed.
After this change, module names for LTO blobs in PDBs will be unique.
Visual Studio and probably other debuggers expect module names to be
unique.

Revert some changes from 1e0b158db (2017) that are no longer necessary
after removing MSVC LTO support.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D78221
2020-04-17 17:15:12 -07:00
LemonBoy aff950e95d [ELF] Support a few more SPARCv9 relocations
Implemented a bunch of relocations found in binaries with medium/large code model and the Local-Exec TLS model. The binaries link and run fine in Qemu.
In addition, the emulation `elf64_sparc` is now recognized.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D77672
2020-04-17 08:12:15 -07:00