Commit Graph

6308 Commits

Author SHA1 Message Date
Ben Dunbobbin f42f599d37 [LLD][ELF][Windows] Allow LLD to overwrite existing output files that are in use
On Windows co-operative programs can be expected to open LLD's
output in FILE_SHARE_DELETE mode. This allows us to delete the
file (by moving it to a temporary filename and then deleting
it) so that we can link another output file that overwrites
the existing file, even if the current file is in use.

A similar strategy is documented here:
https://boostgsoc13.github.io/boost.afio/doc/html/afio/FAQ/deleting_open_files.html

Differential Revision: https://reviews.llvm.org/D82567
2020-07-03 13:08:56 +01:00
Jez Ng 53eb7fda51 [lld-macho] Support binding dysyms to any section
Previously, we only supported binding dysyms to the GOT. This
diff adds support for binding them to any arbitrary section. C++
programs appear to use this, I believe for vtables and type_info.

This diff also makes our bind opcode encoding a bit smarter -- we now
encode just the differences between bindings, which will make things
more compact.

I was initially concerned about the performance overhead of iterating
over these relocations, but it turns out that the number of such
relocations is small. A quick analysis of my llvm-project build
directory showed that < 1.3% out of ~7M relocations are RELOC_UNSIGNED
bindings to symbols (including both dynamic and static symbols).

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D83103
2020-07-02 21:21:01 -07:00
Nico Weber 425fb21e03 ld64.lld: Make janky support for tbd files actually work sometimes
Also fix a bug in the test input that made the test miss this issue.
2020-07-02 15:31:21 -04:00
James Henderson 5012ddd5af [ELF][test] Add some additional .eh_frame/.eh_frame_hdr testing
This patch adds a few extra cases to the existing testing for eh_frame
and eh_frame_hdr behaviour in LLD. They all come from a private
testsuite we are trying to migrate to lit.

Reviewed by: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D82852
2020-07-02 15:51:33 +01:00
Craig Topper 0aad82943a [X86] Enable multibyte NOPs in 64-bit mode for padding/alignment.
The default CPU used by llvm-mc doesn't have the NOPL feature, but
if we know we're compiling in 64-bit mode we should be able to
use nopl.
2020-07-01 23:59:01 -07:00
Nico Weber ba5087f130 ld64.lld: Add janky support for tbd files
With this, a simple hello world links against libSystem.tbd and the
old ld64.lld linker kind of works again with newer SDKs.

The motivation here is to have an arm64 cross linker that's good
enough to be able to run simple configure link checks on non-mac
systems for generating config.h files. Once -flavor darwinnew can
link arm64, we'll switch to that.
2020-07-02 00:05:52 -04:00
Jez Ng 7996a1ef70 [lld-macho] Make sure ZeroFill sections are at the end of their segments
Summary:
ld64 does this, and references an internal rdar:// number as an explanation. No
idea what that rdar issue is, but in practice, it seems that not putting a BSS
section at the end can cause subsequent sections in the same segment to be
overwritten with zeroes.

Reviewers: #lld-macho

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81888
2020-07-01 19:39:29 -07:00
Douglas Yung 6f13299c02 Revert "Update lto.ll test after 3367e9da enabled multibyte NOPs in 64-bit mode."
This reverts commit 79f6a814ab.

Didn't notice that someone had reverted the commit that caused the problem.
2020-07-01 16:34:57 -07:00
Douglas Yung 79f6a814ab Update lto.ll test after 3367e9da enabled multibyte NOPs in 64-bit mode.
This should fix the PS4 linux build bot:
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/70409
2020-07-01 16:25:25 -07:00
Fangrui Song e6ad78fe05 [ELF] Don't resolve a relocation in .debug_line referencing an ICF folded symbol to the tombstone value
After D81784, we resolve a relocation in .debug_* referencing an ICF folded
section symbol to a tombstone value.

Doing this for .debug_line has a problem (https://reviews.llvm.org/D81784#2116925 ):
.debug_line may describe folded lines as having addresses UINT64_MAX or
some wraparound small addresses.

```
int foo(int x) {
  return x; // line 2
}

int bar(int x) {
  return x; // line 6
}
```

```
Address            Line   Column File   ISA Discriminator Flags
------------------ ------ ------ ------ --- ------------- -------------
0x00000000002016c0      1      0      1   0             0  is_stmt
0x00000000002016c7      2      9      1   0             0  is_stmt
prologue_end
0x00000000002016ca      2      2      1   0             0
0x00000000002016cc      2      2      1   0             0  end_sequence
// UINT64_MAX and wraparound small addresses
0xffffffffffffffff      5      0      1   0             0  is_stmt
0x0000000000000006      6      9      1   0             0  is_stmt
prologue_end
0x0000000000000009      6      2      1   0             0
0x000000000000000b      6      2      1   0             0  end_sequence
0x00000000002016d0      9      0      1   0             0  is_stmt
0x00000000002016df     10      6      1   0             0  is_stmt prologue_end
0x00000000002016e6     11     11      1   0             0  is_stmt
...
```

These entries can confuse debuggers:

gdb before 2020-07-01 (binutils-gdb a8caed5d7faa639a1e6769eba551d15d8ddd9510 "Recognize -1 as a tombstone value in .debug_line")
(can't continue due to a breakpoint in an invalid region of memory):
```
Warning:
Cannot insert breakpoint 1.
Cannot access memory at address 0x6
```
lldb (breakpoint has no effect):
```
(lldb) b 6
Breakpoint 1: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
```

This patch special cases .debug_line to not use the tombstone value,
restoring the previous behavior: .debug_line will have entries with the
same addresses (ICF) but different line numbers. A breakpoint on line 2
or 6 will trigger on both functions.

Reviewed By: dblaikie, jhenderson

Differential Revision: https://reviews.llvm.org/D82828
2020-07-01 13:38:16 -07:00
Fangrui Song f5e0ae240f [ELF][test] Fix ppc32-gnu-ifunc.s and remove XFAIL: * 2020-06-30 22:05:51 -07:00
Sam Clegg 73e575a88e [lld][WebAssembly] Give better error message on bad archive member
Include the archive name as well as the member name when an error
is encountered parsing bitcode archives.

Differential Revision: https://reviews.llvm.org/D82884
2020-06-30 21:36:36 -07:00
Fangrui Song d94526bb5f [ELF] --warn-backrefs: check that D79300 fixed an issue due to `mb = {}`
D79300 forgot to change `getBuffer().empty()` in LazyObjFile::parse to
`fetched`. This caused incorrect iterating after the current LazyObjFile was
fetched. This issue is benign and can just cause loss of "undefined symbols"
and "backward reference" diagnostics.

Before D79300 `mb = {}` caused --warn-backrefs-exclude to be useless for
a fetched LazyObjFile.

Add two test cases.
2020-06-26 20:31:47 -07:00
Fangrui Song 4542c18ef2 [ELF] -r: don't parse @ (symbol versioning) for .symver inline asm in bitcode
Fixes PR46420
Similar to D43307 for non-LTO.

Module-level inline assembly can use .symver to create a symbol with `@` in the name.
For relocatable output, @ should be retained in the symbol name. `@ver` should
not be parsed and dropped.

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D82433
2020-06-24 08:22:22 -07:00
Stefan Pintilie 8131ef5d63 [LLD][PowerPC] Add support for R_PPC64_GOT_PCREL34
Add support for the 34bit relocation R_PPC64_GOT_PCREL34 for
PC Relative in LLD.

Reviewers: sfertile, MaskRay

Differential Revision: https://reviews.llvm.org/D81948
2020-06-24 07:40:35 -05:00
Leonard Chan 723b5a1785 [lld][ELF][AArch64] Handle R_AARCH64_PLT32 relocation
This is the followup to D77647 which implements handling for the new
R_AARCH64_PLT32 relocation type in lld. This relocation would benefit the
PIC-friendly vtables feature described in D72959.

Differential Revision: https://reviews.llvm.org/D81184
2020-06-23 16:10:07 -07:00
Petr Hosek fffd05d525 [ELF] Add -z start-stop-visibility= to set __start_/__stop_ symbol visibility
This matches the equivalent flag implemented in GNU linkers, see
https://sourceware.org/pipermail/binutils/2020-June/111685.html for
the associated discussion.

Differential Revision: https://reviews.llvm.org/D55682
2020-06-23 15:59:59 -07:00
Stefan Pintilie 3a55a2a97f [LLD][PowerPC] Add support for R_PPC64_PCREL34
Add support for the 34bit relocation R_PPC64_PCREL34 for PC Relative in LLD.
2020-06-23 14:59:19 -05:00
Fangrui Song e618ccbf43 [ELF] Resolve relocations in .debug_* referencing (discarded symbols or ICF folded section symbols) to tombstone values
See D59553, https://lists.llvm.org/pipermail/llvm-dev/2020-May/141885.html and
https://sourceware.org/pipermail/binutils/2020-May/111357.html for
extensive discussions on a tombstone value.
See http://www.dwarfstd.org/ShowIssue.php?issue=200609.1
(Reserve an address value for "not present") for a DWARF enhancement proposal.

We resolve such relocations to a tombstone value to indicate that the address is invalid.
This solves several problems (the normal behavior is to resolve the relocation to the addend):

* For an empty function in a collected section, a pair of (0,0) can
  terminate .debug_loc and .debug_ranges (as of binutils 2.34, GNU ld
  resolves such a relocation to 1 to avoid the .debug_ranges issue)
* If DW_AT_high_pc is sufficiently large, the address range can collide
  with a regular code range of low address (https://bugs.llvm.org/show_bug.cgi?id=41124 )
* If a text section is folded into another by ICF, we may leave entries
  in multiple CUs claiming ownership of the same range of code, which can
  confuse consumers.
* Debug information associated with COMDAT sections can have problems
  similar to ICF, but is more complex - thus not addressed by this patch.

For pre-DWARF-v5 .debug_loc and .debug_ranges, a pair of 0 can terminate
entries (invalidating subsequent ranges).
-1 is a reserved value with special meaning (base address selection entry) which can't be used either.
Use -2 instead.

For all other .debug_*, use UINT32_MAX for 32-bit targets and UINT64_MAX
for 64-bit targets. In the code, we intentionally use
`uint64_t tombstone = UINT64_MAX` for 32-bit targets as well: this matches
SignExtend64 as used in `relocateAlloc`. (Actually UINT32_MAX does not work for R_386_32)

Note 0, we only special case `target->symbolicRel` (R_X86_64_64, R_AARCH64_ABS64, R_PPC64_ADDR64), not
short-range absolute relocations (e.g. R_X86_64_32). Only forms like DW_FORM_addr need to be special cased.
They can hold an arbitrary address (must be 64-bit on a 64-bit target). (In theory,
producers can make use of small code model to emit 32-bit relocations. This doesn't seem to be leveraged.)

Note 1, we have to ignore the addend, because we don't want to resolve
DW_AT_low_pc (which may have a non-zero addend) to -1+addend (wrap
around to a low address):

  __attribute__((section(".text.x"))) void f1() { }
  __attribute__((section(".text.x"))) void f2() { } // DW_AT_low_pc has a non-zero addend

Note 2, if the prevailing copy does not have debugging information while
a non-prevailing copy has (partial debug build), we don't do extra work
to attach debugging information to the prevailing definition.  (clang
has a lot of debug info optimizations that are on-by-default that assume
the whole program is built with debug info).

  clang -c -ffunction-sections a.cc    # prevailing copy has no debug info
  clang -c -ffunction-sections -g b.cc

Reviewed By: dblaikie, avl, jhenderson

Differential Revision: https://reviews.llvm.org/D81784
2020-06-23 11:48:46 -07:00
Sam Clegg 79aad89d8d [WebAssembly] Add support for externalref to MC and wasm-ld
This allows code for handling externref values to be processed by the
assembler and linker.

Differential Revision: https://reviews.llvm.org/D81977
2020-06-22 15:57:24 -07:00
Peter Collingbourne bd7defeb94 llvm-nm: Implement --special-syms.
Differential Revision: https://reviews.llvm.org/D82251
2020-06-22 13:05:47 -07:00
Jez Ng 3646ee503d [lld-macho] Refactor segment/section creation, sorting, and merging
Summary:
There were a few issues with the previous setup:

1. The section sorting comparator used a declarative map of section names to
  determine the correct order, but it turns out we need to match on more than
  just names -- in particular, an upcoming diff will sort based on whether the
  S_ZERO_FILL flag is set. This diff changes the sorter to a more imperative but
  flexible form.

2. We were sorting OutputSections stored in a MapVector, which left the
  MapVector in an inconsistent state -- the wrong keys map to the wrong values!
  In practice, we weren't doing key lookups (only container iteration) after the
  sort, so this was fine, but it was still a dubious state of affairs. This diff
  copies the OutputSections to a vector before sorting them.

3. We were adding unneeded OutputSections to OutputSegments and then filtering
  them out later, which meant that we had to remember whether an OutputSegment
  was in a pre- or post-filtered state. This diff only adds the sections to the
  segments if they are needed.

In addition to those major changes, two minor ones worth noting:

1. I renamed all OutputSection variable names to `osec`, to parallel `isec`.
  Previously we were using some inconsistent combination of `osec`, `os`, and
  `section`.

2. I added a check (and a test) for InputSections with names that clashed with
  those of our synthetic OutputSections.

Reviewers: #lld-macho

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81887
2020-06-21 17:13:59 -07:00
Fangrui Song 8ffb2097cc [ELF] Refine LMA offset propagation rule in D76995
If neither AT(lma) nor AT>lma_region is specified,
D76995 keeps `lmaOffset` (LMA - VMA) if the previous section is in the
default LMA region.

This patch additionally checks that the two sections are in the same
memory region.

Add a test case derived from https://bugs.llvm.org/show_bug.cgi?id=45313

  .mdata : AT(0xfb01000) { *(.data); } > TCM
  // It is odd to make .bss inherit lmaOffset, because the two sections
  // are in different memory regions.
  .bss : { *(.bss) } > DDR

With this patch, section VMA/LMA match GNU ld. Note, GNU ld supports
out-of-order (w.r.t sh_offset) sections and places .text and .bss in the
same PT_LOAD. We don't have that behavior.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D81986
2020-06-19 09:11:33 -07:00
Fangrui Song c4d13f72a6 [ELF] Refactor ObjFile<ELFT>::initializeSymbols to enforce the invariant: InputFile::symbols has non null entry
Fixes PR46348.

ObjFile<ELFT>::initializeSymbols contains two symbol iteration loops:

```
for each symbol
  if non-inheriting && non-local
    fill in this->symbols[i]

for each symbol
  if local
    fill in this->symbols[i]
  else
    symbol resolution
```

Symbol resolution can trigger a duplicate symbol error which will call
InputSectionBase::getObjMsg to iterate over InputFile::symbols.  If a
non-local symbol appears after the non-local symbol being resolved
(violating ELF spec), its `this->symbols[i]` entry has not been filled
in, InputSectionBase::getObjMsg will crash due to
`dyn_cast<Defined>(nullptr)`.

To fix the bug, reorganize the two loops to ensure this->symbols is
complete before symbol resolution. This enforces the invariant:
InputFile::symbols has none null entry when InputFile::getSymbols() is called.

```
for each symbol
  if non-inheriting
    fill in this->symbols[i]

for each symbol starting from firstGlobal
  if non-local
    symbol resolution
```

Additionally, move the (non-local symbol in local part of .symtab)
diagnostic from Writer<ELFT>::copyLocalSymbols() to initializeSymbols().

Reviewed By: grimar, jhenderson

Differential Revision: https://reviews.llvm.org/D81988
2020-06-19 09:05:37 -07:00
Alexandre Ganea 2ae0df5be7 [CodeView] Revert 8374bf4363 and 403f953792
This reverts:
8374bf4363 [CodeView] Fix generated command-line expansion in LF_BUILDINFO. Fix the 'pdb' entry which was previously a null reference, now an empty string.
403f953792 [CodeView] Add full repro to LF_BUILDINFO record

This is causing the lld/test/COFF/pdb-relative-source-lines.test to fail: http://lab.llvm.org:8011/builders/lld-x86_64-win/builds/1096/steps/test-check-all/logs/FAIL%3A%20lld%3A%3Apdb-relative-source-lines.test
And clang/test/CodeGen/debug-info-codeview-buildinfo.c fails as well: http://lab.llvm.org:8011/builders/clang-s390x-linux/builds/33346/steps/ninja%20check%201/logs/FAIL%3A%20Clang%3A%3Adebug-info-codeview-buildinfo.c
2020-06-18 16:18:46 -04:00
Sam Clegg 21e14cce23 [lld][WebAssembly] Allow ctors functions that return values
Some projects use the constructor attribute on functions that also
return values.  In this case we just ignore them.

The error was reported in the libgpg-error project that marks
gpg_err_init with the `__constructor__` attribute.

Differential Revision: https://reviews.llvm.org/D81962
2020-06-18 13:11:40 -07:00
Alexandre Ganea 403f953792 [CodeView] Add full repro to LF_BUILDINFO record
This patch adds some missing information to the LF_BUILDINFO which allows for rebuilding an .OBJ without any external dependency but the .OBJ itself (other than the compiler executable).

Some tools need this information to reproduce a build without any knowledge of the build system. The LF_BUILDINFO therefore stores a full path to the compiler, the PWD (which is the CWD at program startup), a relative or absolute path to the TU, and the full CC1 command line. The command line needs to be freestanding (not depend on any environment variable). In the same way, MSVC doesn't store the provided command-line, but an expanded version (somehow their equivalent of CC1) which is also freestanding.

For more information see PR36198 and D43002.

Differential Revision: https://reviews.llvm.org/D80833
2020-06-18 09:17:15 -04:00
Greg McGary f7453a97ce [lld-macho] Add missing search-paths-darwin.test
Summary: Forgot to `git add` it when patching D80677
2020-06-17 20:51:26 -07:00
Greg McGary d50f44a2f7 [lld-macho] Handle framework search path, alongside library search path
Summary:
Add front-end support for `lld::macho::Configuration::frameworkSearchPath`.

Depends on D80582.

Reviewers: ruiu, pcc, MaskRay, smeenai, int3, Ktwu, alexshap, christylee

Reviewed By: int3

Subscribers: ormris, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80677
2020-06-17 20:41:28 -07:00
Jez Ng 525c7d8cda [lld-macho] Handle alignment correctly when merging InputSections
Summary:
Previously, we weren't updating isecAddr when aligning InputSections,
resulting in truncated sections under the right conditions.

Reviewers: #lld-macho, compnerd

Reviewed By: #lld-macho, compnerd

Subscribers: smeenai, compnerd, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81298
2020-06-17 20:41:28 -07:00
Jez Ng 74871cdad7 [lld-macho] Ensure __bss sections we output have file offset of zero
Summary:
llvm-mc emits `__bss` sections with an offset of zero, but we weren't expecting
that in our input, so we were copying non-zero data from the start of the file and
putting it in `__bss`, with obviously undesirable runtime results. (It appears that
the kernel will copy those nonzero bytes as long as the offset is nonzero, regardless
of whether S_ZERO_FILL is set.)

I debated on whether to make a special ZeroFillSection -- separate from a
regular InputSection -- but it seemed like too much work for now. But I'm happy
to refactor if anyone feels strongly about having it as a separate class.

Depends on D80857.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80859
2020-06-17 20:41:28 -07:00
Jez Ng a12e7d406d [lld-macho] Handle GOT relocations of non-dylib symbols
Summary:
Turns out this case is actually really common -- it happens whenever there's
a reference to an `extern` variable that ends up statically linked.

Depends on D80856.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80857
2020-06-17 20:41:28 -07:00
Jez Ng c3d98ea89f [lld-macho] Support X86_64_RELOC_GOT
Summary:
As far as I can tell, it's identical to _GOT_LOAD. llvm-mc has the following
comment explaining why _GOT exists:

```
// x86_64 distinguishes movq foo@GOTPCREL so that the linker can
// rewrite the movq to an leaq at link time if the symbol ends up in
// the same linkage unit.
```

Depends on D80855.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: MaskRay, smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80856
2020-06-17 20:41:28 -07:00
Jez Ng fcde378dcb [lld-macho] Support non-pcrel section relocs
Summary: Depends on D80854.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80855
2020-06-17 20:41:28 -07:00
Jez Ng 2f4cfa3c7a [lld-macho] Avoid explicit -arch in tests by defaulting to x86-64
Summary:
As mentioned in https://reviews.llvm.org/D81326#2093931, I'm not sure it
makes sense to use the default target triple to determine -arch.
Long-term we should probably detect it from the input object files, but
in the meantime it would be nice not to have to add it to all our tests
by using a convenient default.

Reviewers: #lld-macho

Subscribers: arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81983
2020-06-17 20:41:27 -07:00
Fangrui Song 49279ca160 [ELF] Improve --export-dynamic-symbol performance by checking whether wildcard is really used
A hasWildcard pattern iterates over symVector, which can be slow when there
are many --export-dynamic-symbol. In optimistic cases, most patterns don't use
a wildcard character. hasWildcard: false can avoid a symbol table iteration.

While here, add two tests using `[` and `?`, respectively.
2020-06-17 17:12:10 -07:00
Fangrui Song a649802c59 [lld-macho][test] Fix dylink-lazy.s after 0a5228a02c 2020-06-16 17:57:59 -07:00
Saleem Abdulrasool 0a5228a02c test: add library search path to repair the test
Update the test to no longer rely on the testing workaround in the
linker and use the external libSystem stub.
2020-06-16 17:06:21 -07:00
Saleem Abdulrasool 73312976ad lld: remove old test support path
This removes the stub library that lld injected to satisfy the
dependency on the libSystem.  Now with TBD support, we can provide the
stub library to permit the tests to function properly as they would on a
real system.

Reviewed By: smeenai
Differential Revision: https://reviews.llvm.org/D81418
2020-06-16 15:57:58 -07:00
Greg McGary 7df80e3f23 [lld-macho] Specify the complete set of command-line options for ld64
This is a complete Options.td compiled from ld(1) dated 2018-03-07 and
cross checked with ld64 source code version 512.4 dated 2018-03-18.

This is the first in a series of diffs for argument handling. Follow-ups
will include switch cases for all the new instances of `OPT_foo`, and
parsing/validation of arguments attached to options, e.g., more code
akin to `OPT_platform_version` and associated `parsePlatformVersion()`.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80582
2020-06-15 12:50:20 -07:00
Wouter van Oortmerssen d9e0bbd17b [WebAssembly] Adding 64-bit versions of all load & store ops.
Context: https://github.com/WebAssembly/memory64/blob/master/proposals/memory64/Overview.md
This is just a first step, adding the new instruction variants while keeping the existing 32-bit functionality working.
Some of the basic load/store tests have new wasm64 versions that show that the basics of the target are working.
Further features need implementation, but these will be added in followups to keep things reviewable.

Differential Revision: https://reviews.llvm.org/D80769
2020-06-15 08:31:56 -07:00
Kirill Bobyrev 9baba7cf66
Revert "[lld-macho] No need to explicitly specify -arch in tests"
This reverts commit 51c5baacf3 and also
337fb8c767 - "[lld-macho] Set REQUIRES:
x86 on more tests".

These patches cause test crashes:

http://lab.llvm.org:8011/builders/ppc64le-lld-multistage-test/builds/10054
2020-06-15 12:27:30 +02:00
Jez Ng 337fb8c767 [lld-macho] Set REQUIRES: x86 on more tests
Summary: Fixes the build break caused by D81802.
2020-06-14 19:05:12 -07:00
Jez Ng 53c796b948 [lld-macho] Properly handle & validate relocation r_length
Summary:
We should be reading / writing our addends / relocated addresses based on
r_length, and not just based on the type of the relocation. But since only
some r_length values are valid for a given reloc type, I've also added some
validation.

ld64 has code to allow for r_length = 0 in X86_64_RELOC_BRANCH relocs, but I'm
not sure how to create such a relocation...

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80854
2020-06-14 16:35:23 -07:00
Jez Ng 51c5baacf3 [lld-macho] No need to explicitly specify -arch in tests
Summary: After {D81326} landed, some tests started failing if they did
not have `-arch` specified. I think one of the reasons happened was due
to the fact that we were taking a reference to a temporary value that
was freed too early. Fixing that got the error to go away on my local
Linux machine.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D81802
2020-06-14 16:35:21 -07:00
Pavel Labath fccaa89e23 [llvm/DWARFDebugLine] Fix a typo in one warning message 2020-06-11 13:04:52 +02:00
Hongtao Yu 2638aafe12 [LLD][ThinLTO] Add --thinlto-single-module to allow compiling partial modules.
This change introduces an LLD switch --thinlto-single-module to allow compiling only a part of the input modules. This is specifically enables:

  1. Fast investigating/debugging modules of interest without spending time on compiling unrelated modules.
  2. Compiler debug dump with -mllvm -debug-only= for specific modules.

It will be useful for large applications which has 1K+ input modules for thinLTO.

The switch can be combined with `--lto-obj-path=` or `--lto-emit-asm` to obtain intermediate object files or assembly files. So far the module name matching is implemented as a fuzzy name lookup where the modules with name containing the switch value are compiled.

E.g,
Command:
     ld.lld main.o thin.a --thinlto-single-module=thin.a --lto-obj-path=single.o
log:
     [ThinLTO] Selecting thin.a(thin1.o at 168) to compile
     [ThinLTO] Selecting thin.a(thin2.o at 228) to compile
Command:
     ld.lld main.o thin.a --thinlto-single-module=thin1.o --lto-obj-path=single.o
log:
     [ThinLTO] Selecting thin.a(thin1.o at 168) to compile

Differential Revision: https://reviews.llvm.org/D80406
2020-06-10 15:32:30 -07:00
Pavel Labath de522035b9 [lld] Fix test/ELF/undef.s for 6f55b5a1
There's been a slight change in wording for the warning message about
debug_line problems.
2020-06-10 16:36:10 +02:00
gbreynoo be44b79257 [lld][test] Expand testing for dynamic-list and export-dynamic
- Expanded testing for --dynamic-list and --export-dynamic
- Fixed invalid-dynamic-list.test

Differential Revision: https://reviews.llvm.org/D80311
2020-06-10 14:06:30 +01:00
Fangrui Song b114e134bd [ELF] Fix --thinlto-index-only regression after D79300
After D79300, we don't rewrite InputFile::mb to an empty buffer.
In thinLTOCreateEmptyIndexFiles(), we should check LazyObjFile::fetched
as well as checking whether mb is a bitcode, otherwise we would overwrite (path + .thinlto.bc) with an empty index.
2020-06-09 23:10:30 -07:00
Fangrui Song ba890da287 [ELF] Demote lazy symbols relative to a discarded section to Undefined
Fixes PR45594.

In `ObjFile<ELFT>::initializeSymbols()`, for a defined symbol relative to
a discarded section (due to section group rules), it may have been
inserted as a lazy symbol. We need to demote it to an Undefined to
enable the `discarded section` error happened in a later pass.

Add `LazyObjFile::fetched` (if true) and `ArchiveFile::parsed` (if
false) to represent that there is an ongoing lazy symbol fetch and we
should replace the current lazy symbol with an Undefined, instead of
calling `Symbol::resolve` (`Symbol::resolve` should be called if the lazy
symbol was added by an unrelated archive/lazy object).

As a side result, one small issue in start-lib-comdat.s is now fixed.
The hack motivating D51892 will be unsupported: if
`.gnu.linkonce.t.__i686.get_pc_thunk.bx` in an archive is referenced
by another section, this will likely be errored unless the function is
also defined in a regular object file.
(Bringing back rL330869 would error `undefined symbol` instead of the
more relevant `discarded section`.)

Note, glibc i386's crti.o still works (PR31215), because
`.gnu.linkonce.t.__x86.get_pc_thunk.bx` is in crti.o (one of the first
regular object files in a linker command line).

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D79300
2020-06-09 11:27:34 -07:00
Saleem Abdulrasool 5fda192fed test: repair lld builder
Add the explicit `-arch x86_64` to the invocation.
2020-06-08 20:13:54 -07:00
Saleem Abdulrasool 6fe27b5fed lld: initial pass at supporting TBD
Add support to lld to use Text Based API stubs for linking.  This is
support is incomplete not filtering out platforms.  It also does not
account for architecture specific API handling and potentially does not
correctly handle trees of re-exports with inlined libraries being
treated as direct children of the top level library.
2020-06-08 18:15:40 -07:00
Saleem Abdulrasool fcdf7578aa lld: improve the `-arch` handling for MachO
Use the default target triple configured by the user to determine the
default architecture for `ld64.lld`.  Stash the architecture in the
configuration as when linking against TBDs, we will need to filter out
the symbols based upon the architecture.  Treat the Haswell slice as it
is equivalent to `x86_64` but with the extra Haswell extensions (e.g.
AVX2, FMA3, BMI1, etc).  This will make it easier to add new
architectures in the future.

This change also changes the failure mode where an invalid `-arch`
parameter will result in the linker exiting without further processing.
2020-06-08 11:04:19 -07:00
Fangrui Song ac6abc99e2 [ELF] Don't cause assertion failure if --dynamic-list or --version-script takes an empty file
Fixes PR46184
Report line 1 of the last memory buffer.
2020-06-05 15:59:54 -07:00
Saleem Abdulrasool e78431354b lld: use modern library search ordering
This merges the static and shared library and behaves as if
`-search_paths_first` was specified which is also the default behaviour
on ld64 (and now lld). Unify the paths, and use `llvm::sys::path` to
deal with the path to be truly agnostic to the host.
2020-06-05 12:12:26 -07:00
Fangrui Song 7bee6e30fe [ELF] Handle -u before input files
If both a.a and b.so define foo

```
ld.bfd -u foo a.a b.so  # foo is defined
ld.bfd a.a b.so -u foo  # foo is defined
ld.bfd -u foo b.so a.a  # foo is undefined (provided at runtime by b.so)
ld.bfd b.so a.a -u foo  # foo is undefined (provided at runtime by b.so)
```

In all cases we make foo undefined in the output.  I tend to think the
GNU ld behavior makes more sense.

* In their model, they have to treat -u as a fake object file with an
  undefined symbol before all input files, otherwise the first archive would not be fetched.
* Following their behavior allows us to drop a --warn-backrefs special case.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D81052
2020-06-05 08:44:38 -07:00
Saleem Abdulrasool 116e38fd8b lld: add basic static library search
This is a very basic static library search addition. This is the pre-Xcode4
behaviour of searching all paths for the shared version before searching for
the static version of the library. This behaviour is supposed to be inverted
with `-search_paths_first` being the default. This adds the library search
with the intention of providing the setup to merge the paths into one path
and making it controllable by `OPT_search_paths_first`.
2020-06-03 23:32:05 +00:00
Fangrui Song 3eb4bf13ba [ELF] Append " [--no-allow-shlib-undefined]" to the corresponding diagnostics
--no-allow-shlib-undefined (enabled by default when linking an
executable) rejects unresolved references in shared objects.

Users may be confused by the common diagnostics of unresolved symbols in
object files (LLD: "undefined symbol: foo"; GNU ld/gold: "undefined reference to")

Learn from GCC/clang " [-Wfoo]": append the option name to the
diagnostics. Users can find relevant information by searching
"--no-allow-shlib-undefined".  It should also be obvious to them that
the positive form --allow-shlib-undefined can suppress the error.

Also downgrade the error to a warning if --noinhibit-exec is used (compatible
with GNU ld and gold).

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D81028
2020-06-03 07:59:37 -07:00
Jez Ng f04d1c3b90 [lld-macho] Move all tests for erroneous inputs under invalid/
For consistency.

The no-id-dylib test was originally referencing the Inputs/ folder via a
relative path. Instead of updating that path, I decided to make the test
self-contained.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D80217
2020-06-02 13:19:38 -07:00
Jez Ng a04c133564 [lld-macho] Set __PAGEZERO size to 4GB
That's what ld64 uses for 64-bit targets. I figured it's best to make
this change sooner rather than later since a bunch of our tests are
relying on hardcoded addresses that depend on this value.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80177
2020-06-02 13:19:38 -07:00
Jez Ng df2a5778c3 [lld-macho] Error on encountering undefined symbols
... instead of silently emitting a reference to the zero address.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80169
2020-06-02 13:19:38 -07:00
Jez Ng 6f6d91867d [lld-macho] Add some relocation validation logic
I considered making a `Target::validate()` method, but I wasn't sure how
I felt about the overhead of doing yet another switch-dispatch on the
relocation type, so I put the validation in `relocateOne` instead...
might be a bit of a micro-optimization, but `relocateOne` does assume
certain things about the relocations it gets, and this error handling
makes that explicit, so it's not a totally unreasonable code
organization.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80049
2020-06-02 13:19:38 -07:00
Igor Kudrin 4933ab2ccb [DebugInfo] Report the format of call frame information entries [6/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Simon Atanasyan b00f0d4238 [mips] Support 64-bit relative relocations
MIPS 64-bit ABI does not provide special PC-relative relocation like
R_MIPS_PC32 in 32-bit case. But we can use a "chain of relocation"
defined by N64 ABIs. In that case one relocation record might contain up
to three relocations which applied sequentially. Width of a final relocation
mask applied to the result of relocation depends on the last relocation
in the chain. In case of 64-bit PC-relative relocation we need the following
chain: `R_MIPS_PC32 | R_MIPS_64`. The first relocation calculates an
offset, but does not truncate the result. The second relocation just
apply calculated result as a 64-bit value.

The 64-bit PC-relative relocation might be useful in generation of
`.eh_frame` sections to escape passing `-Wl,-z,notext` flags to linker.

Differential Revision: https://reviews.llvm.org/D80390
2020-06-02 11:44:11 +03:00
Fangrui Song a6ae333a0c [ELF] --wrap: don't error `undefined reference to __real_foo` (--no-allow-shlib-undefined) if foo is a wrapped definition
This is a regression after D51283.

Also, export `foo` if `__real_foo` is referenced by a shared object.
2020-06-01 23:00:51 -07:00
Fangrui Song 751f18e7d4 [ELF] Refine --export-dynamic-symbol semantics to be compatible GNU ld 2.35
GNU ld from binutils 2.35 onwards will likely support
--export-dynamic-symbol but with different semantics.
https://sourceware.org/pipermail/binutils/2020-May/111302.html

Differences:

1. -export-dynamic-symbol is not supported
2. --export-dynamic-symbol takes a glob argument
3. --export-dynamic-symbol can suppress binding the references to the definition within the shared object if (-Bsymbolic or -Bsymbolic-functions)
4. --export-dynamic-symbol does not imply -u

I don't think the first three points can affect any user.
For the fourth point, Not implying -u can lead to some archive members unfetched.
Add -u foo to restore the previous behavior.

Exact semantics:

* -no-pie or -pie: matched non-local defined symbols will be added to the dynamic symbol table.
* -shared: matched non-local STV_DEFAULT symbols will not be bound to definitions within the shared object
  even if they would otherwise be due to -Bsymbolic, -Bsymbolic-functions, or --dynamic-list.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80487
2020-06-01 11:30:03 -07:00
Fangrui Song ee9a251caf [ELF] Set DF_1_PIE for -pie
DF_1_PIE originated from Solaris (https://docs.oracle.com/cd/E36784_01/html/E36857/chapter6-42444.html ).
GNU ld since
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=5fe2850dd96483f176858fd75c098313d5b20bc2
sets the flag on non-Solaris platforms.

It can help distinguish PIE from ET_DYN.
eu-classify from elfutils uses this to recognize PIE (https://sourceware.org/git/?p=elfutils.git;a=commit;h=3f489b5c7c78df6d52f8982f79c36e9a220e8951 )

glibc uses this flag to reject dlopen'ing a PIE (https://sourceware.org/bugzilla/show_bug.cgi?id=24323 )

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80872
2020-06-01 10:19:41 -07:00
Fangrui Song 881c5eef98 [ELF] Add -z rel and -z rela
LLD supports both REL and RELA for static relocations, but emits either
of REL and RELA for dynamic relocations. The relocation entry format is
specified by each psABI.

musl ld.so supports both REL and RELA. For such ld.so implementations,
REL (.rel.dyn .rel.plt) has size benefits even if the psABI chooses RELA:
sizeof(Elf64_Rel)=16 < sizeof(Elf64_Rela)=24.

* COPY, GLOB_DAT and J[U]MP_SLOT always have 0 addend. A ld.so
  implementation does not need to read the implicit addend.
  REL is strictly better.
* A RELATIVE has a non-zero addend. Such relocations can be packed
  compactly with the RELR relocation entry format, which is out of scope
  of this patch.
* For other dynamic relocation types (e.g. symbolic relocation R_X86_64_64),
  a ld.so implementation needs to read the implicit addend. REL may have
  minor performance impact, because reading implicit addends forces
  random access reads instead of being able to blast out a bunch of
  writes while chasing the relocation array.

This patch adds -z rel and -z rela to change the relocation entry format
for dynamic relocations. I have tested that a -z rel produced x86-64
executable works with musl ld.so

-z rela may be useful for debugging purposes on processors whose psABIs
specify REL as the canonical format: addends can be easily read by a tool.

Reviewed By: grimar, mcgrathr

Differential Revision: https://reviews.llvm.org/D80496
2020-05-29 14:22:03 -07:00
Sam Clegg fd1c894a4a [lld][WebAssembly] Convert some lld tests to assembly
When we originally wrote these tests we didn't have a stable and
fleshed out assembly format.  Now we do so we should prefer that
over llvm ir for lld tests to avoid including more part of llvm
than necessary in order to run the test.

This change converts just 30 out of about 130 test files. More to
come when I have some more time.

Differential Revision: https://reviews.llvm.org/D80361
2020-05-28 16:52:01 -07:00
Hiroshi Yamauchi a7fa35a629 [ThinLTO] Compute the basic block count across modules.
Summary:
Count the per-module number of basic blocks when the module summary is computed
and sum them up during Thin LTO indexing.

This is used to estimate the working set size under the partial sample PGO.

This is split off of D79831.

Reviewers: davidxl, espindola

Subscribers: emaste, inglorion, hiraditya, MaskRay, steven_wu, dexonsmith, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80403
2020-05-28 10:33:05 -07:00
Fangrui Song 1224e619d9 [ELF][test] Fix wrap-no-real.s after D51283
Give %t3.so a DT_SONAME so that the DT_NEEDED entry in a dependent executable has a fixed length.
2020-05-27 17:43:33 -07:00
Rui Ueyama 54d2896852 [ELF] --wrap: Drop __real_ symbol from the symbol table
In D34993, we discussed and concluded that we should drop `__real_
symbol from the symbol table, but I did the opposite in D50569.
This patch is to drop `__real_` symbol.

MaskRay's note: omitting `__real_` is important if it is undefined:
otherwise a subsequent link may error due to the undefined `__real_` in .dynsym

Differential Revision: https://reviews.llvm.org/D51283
2020-05-27 16:58:00 -07:00
Fangrui Song b8a3c618d6 [ELF] Allow misaligned SHT_GNU_verneed
Bazel created interface shared objects (.ifso) may be misaligned.  We use
llvm::support::detail::packed_endian_specific_integral under the hood
which allows reading of misaligned values, so there is not a need to
diagnose (in LLD we don't intend to support sophisticated parsing for
SHT_GNU_*).
2020-05-26 11:18:19 -07:00
Fangrui Song bae7cf6746 [ELF][PPC64] Synthesize _savegpr[01]_{14..31} and _restgpr[01]_{14..31}
In the 64-bit ELF V2 API Specification: Power Architecture, 2.3.3.1. GPR
Save and Restore Functions defines some special functions which may be
referenced by GCC produced assembly (LLVM does not reference them).

With GCC -Os, when the number of call-saved registers exceeds a certain
threshold, GCC generates `_savegpr0_* _restgpr0_*` calls and expects the
linker to define them. See
https://sourceware.org/pipermail/binutils/2002-February/017444.html and
https://sourceware.org/pipermail/binutils/2004-August/036765.html . This
is weird because libgcc.a would be the natural place. However, the linker
generation approach has the advantage that the linker can generate
multiple copies to avoid long branch thunks. We don't consider the
advantage significant enough to complicate our trunk implementation, so
we take a simple approach.

* Check whether `_savegpr0_{14..31}` are used
* If yes, define needed symbols and add an InputSection with the code sequence.

`_savegpr1_*` `_restgpr0_*` and `_restgpr1_*` are similar.

Reviewed By: sfertile

Differential Revision: https://reviews.llvm.org/D79977
2020-05-26 09:35:41 -07:00
Kang Zhang 86e3abc9e6 [PowerPC] Add some InstAlias definitions
Summary:
This patch add the InstAlias definitions for below instructions.

ADDI ADDIS ADDI8 ADDIS8
RLWINM8
ISEL ISEL8
OR OR_rec ORI ORI8 XORI8
CNTLZW8 CNTLZW8_rec
TEND TSR
RFEBB
NOR NOR_rec
MTCRF
SUBF SUBF_rec SUBFC SUBFC_rec
RLDICL_32_64
TW

Reviewed By: steven.zhang

Differential Revision: https://reviews.llvm.org/D77559
2020-05-24 14:05:28 +00:00
Tobias Hieta f794808bb9 [LLD/MinGW]: Expose --thinlto-cache-dir
Differential Revision: https://reviews.llvm.org/D80438
2020-05-24 12:30:56 +03:00
Fangrui Song e32f04cdc9 [ELF] Parse SHT_GNU_verneed and respect versioned undefined symbols in shared objects
An undefined symbol in a shared object can be versioned, like `f@v1`.
We currently insert `f` as an Undefined into the symbol table, but we
should insert `f@v1` instead.

The string `v1` is inferred from SHT_GNU_versym and SHT_GNU_verneed.
This patch implements the functionality.

Failing to do this can cause two issues:

* If a versioned symbol referenced by a shared object is defined in the
  executable, we will fail to export it.
* If a versioned symbol referenced by a shared object in another object
  file, --no-allow-shlib-undefined may spuriously report an
  "undefined reference to " error. See https://bugs.llvm.org/show_bug.cgi?id=44842
  (Linking -lfftw3 -lm on Arch Linux can cause
  `undefined reference to __log_finite`)

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D80059
2020-05-23 09:55:48 -07:00
Thomas Lively d851fce4cb [lld][WebAssembly] Do not emit initialization for .bss segments
Summary:
This patch fixes a bug where initialization code for .bss segments was
emitted in the memory initialization function even though the .bss
segments were discounted in the datacount section and omitted in the
data section. This was producing invalid binaries due to out-of-bounds
segment indices on the memory.init and data.drop instructions that
were trying to operate on the nonexistent .bss segments.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80354
2020-05-21 11:33:25 -07:00
Jez Ng ce0d8beebc [lld-macho][re-land] Support X86_64_RELOC_UNSIGNED
This reverts commit db8559eee4.
2020-05-19 12:31:55 -07:00
Jez Ng 4eb6f4854e [lld-macho][re-land] Support .subsections_via_symbols
Summary:
This diff restores and builds upon @pcc and @ruiu's initial work on
subsections.

The .subsections_via_symbols directive indicates we can split each
section along symbol boundaries, unless those symbols have been marked
with `.alt_entry`.

We exercise this functionality in our tests by using order files that
rearrange those symbols.

Depends on D79668.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: thakis, llvm-commits, pcc, ruiu

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79926
2020-05-19 12:31:54 -07:00
Fangrui Song 6467649974 [ELF] Make --trace-symbol track preempted shared definitions
Note, we still name a preempted SharedSymbol "shared definition",
instead of "reference" as printed by GNU ld. This difference should not matter.

```
// GNU ld
ld.bfd: t: definition of f@v1
ld.bfd: t.so: reference to f@v1
```

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80143
2020-05-19 08:56:35 -07:00
Jez Ng 70fbbcdd34 Revert "[lld-macho] Support .subsections_via_symbols"
Due to build breakage mentioned in https://reviews.llvm.org/D79926.

This reverts commit e270b2f172.
2020-05-19 08:30:02 -07:00
Jez Ng db8559eee4 Revert "[lld-macho] Support X86_64_RELOC_UNSIGNED"
This reverts commit 1f820e3559.
2020-05-19 08:30:02 -07:00
Jez Ng 1f820e3559 [lld-macho] Support X86_64_RELOC_UNSIGNED
Note that it's only used for non-pc-relative contexts.

Reviewed By: MaskRay, smeenai

Differential Revision: https://reviews.llvm.org/D80048
2020-05-19 07:46:57 -07:00
Jez Ng e270b2f172 [lld-macho] Support .subsections_via_symbols
This diff restores and builds upon @pcc and @ruiu's initial work on
subsections.

The .subsections_via_symbols directive indicates we can split each
section along symbol boundaries, unless those symbols have been marked
with `.alt_entry`.

We exercise this functionality in our tests by using order files that
rearrange those symbols.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D79926
2020-05-19 07:46:57 -07:00
Jez Ng 55e9eb416e [lld-macho] Support -order_file
The order file indicates how input sections should be sorted within each
output section, based on the symbols contained within those sections.

This diff sets the stage for implementing and testing
`.subsections_via_symbols`, where we will break up InputSections by each
symbol and sort them more granularly.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D79668
2020-05-19 07:46:57 -07:00
Greg McGary 1aadd6ce61 [lld] Remove unused lld/test/Driver/Inputs/**/libtest.a
Under `lld/test/Driver/Inputs/`, all instances of `libtest.a` are
unreferenced. FYI, all of these are empty archives, and the files
contain only a magic number.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D80182
2020-05-19 07:19:10 -07:00
Hongtao Yu 90af55d8a9 [LLD][ELF] Use offset in thin archives to disambiguate thinLTO members
This is fixing a thinLTO module collision issue for thin archives. The problem is that we always use a zero offset to name members in a thin archive and that causes the following build error:

    ld.lld: error: Expected at most one ThinLTO module per bitcode file

which happens to a thin archive that has two members with the same object file name (whose paths will be ignored by thinLTO driver)

The fix here is to use real member offset instead as is done for non-thin archives.

Differential Revision: https://reviews.llvm.org/D79880
2020-05-15 12:02:08 -07:00
Fangrui Song e36223c85c [ELF] Enforce two dashes for Flag options not supported by GNU ld (i.e. no compatibility burden)
Announced on https://lists.llvm.org/pipermail/llvm-dev/2020-May/141416.html

Similar to D79371, but for `multiclass B` (convenience helper for defining --foo and --no-foo)

Some changed options are also used by gold, but I haven't seen their
one-dash use cases outside of lld's testsuite.
2020-05-15 11:07:25 -07:00
Georgii Rymar 969c63a2ec [LLD][ELF] - Fix section-alignment.test after yaml2obj change.
yaml2obj changed in rG7ccae2cece72. Previous version of the
yaml produced a different error and a 500mb binary.
2020-05-15 12:00:37 +03:00
Kellie Medlin 2b920ae78c [lld] Add archive file support to Mach-O backend
With this change, basic archive files can be linked together. Input
section discovery has been refactored into a function since archive
files lazily resolve their symbols / the object files containing those
symbols.

Reviewed By: int3, smeenai

Differential Revision: https://reviews.llvm.org/D78342
2020-05-14 12:58:35 -07:00
Reid Kleckner f5a79713b6 [LLD] Fix precomp-link.test
I had a stale directory at %t, so the test passed locally, but not
remotely. Skip the %t directory altogether.
2020-05-14 10:32:50 -07:00
Reid Kleckner 54a335a2f6 [COFF] Move type merging to TpiSource::mergeDebugT virtual method
This paves the way to doing more things in parallel, and allows us to
order type sources in dependency order. PDBs and PCH objects have to be
loaded before object files which use them.

This is a rebase of the unapplied remaining changes in
https://reviews.llvm.org/D59226. I found it very challenging to rebase
this across the LLD variable name style change. I recall there was a
tool for that, but I didn't take the time to use it.

Reviewers: aganea, akhuang

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79672
2020-05-14 09:47:00 -07:00
Martin Storsjö b0969be4df [LLD] [COFF] Try to fix test errors from 7f0e6c31c2 on windows
Just skip trying to match for the path separator explicitly (instead
of making it match either a forward or backwards slash), simplifying
the test a little.
2020-05-14 13:47:34 +03:00
Martin Storsjö 7f0e6c31c2 [LLD] [COFF] Add options for disabling auto import and runtime pseudo relocs
Allow disabling either the full auto import feature, or just
forbidding the cases that require runtime fixups.

As long as all auto imported variables are referenced from separate
.refptr$<name> sections, we can alias them on top of the IAT entries
and don't actually need any runtime fixups via pseudo relocations.
LLVM generates references to variables in .refptr stubs, if it
isn't known that the variable for sure is defined in the same object
module. Runtime pseudo relocs are needed if the addresses of auto
imported variables are used in constant initializers though.

Fixing up runtime pseudo relocations requires the use of
VirtualProtect (which is disallowed in WinStore/UWP apps) or
VirtualProtectFromApp. To allow any risk of ambiguity, allow
rejecting cases that would require this at the linker stage.

This adds support for the --disable-runtime-pseudo-reloc and
--disable-auto-import options in the MinGW driver (matching GNU ld.bfd)
with corresponding lld private options in the COFF driver.

Differential Revision: https://reviews.llvm.org/D78923
2020-05-14 13:05:14 +03:00
Sam Clegg 064e9907ba [lld][WebAssembly] Fix for --relocatable and signature mismatches
This is a followup to https://reviews.llvm.org/D78779.

When signatures mismatch we create set of variant symbols.  Some of
the fields in these symbols were not be initialized correct.
Specifically we were seeing isUsedInRegularObj not being set correctly,
leading to the symbol not getting included in the symbol table
and a crash writing relections in --reloctable mode.

There is larger refactor due here, but this is a minimal change the
fixes the bug at hand.

Differential Revision: https://reviews.llvm.org/D79756
2020-05-13 10:27:09 -07:00
Nico Weber 759bae956a [lld-macho] Ignore -platform_version and -syslibroot flags.
clang passes these flags; this makes it easier to try `clang -v`
output with `ld -flavor darwinnew`.

Differential Revision: https://reviews.llvm.org/D79797
2020-05-12 19:17:01 -04:00
Jez Ng 87b6fd3e02 [lld-macho] Add support for creating and reading reexported dylibs
This unblocks the linking of real programs, since many core system
functions are only available as sub-libraries of libSystem.

Differential Revision: https://reviews.llvm.org/D79228
2020-05-12 07:52:03 -07:00
Jez Ng c8c39185f3 [lld-macho] Re-add dylink-lazy test
This reverts commit eb81de2de4003e3045fdf743e093f77e37aee9bf; the
test commands just needed to be run under llvm-lit.
2020-05-12 07:52:03 -07:00
Eric Christopher eb81de2de4 Temporarily Revert "[lld-macho] Re-add dylink-lazy test" as it
appears to be still failing.

This reverts commit 723c46e645.
2020-05-11 19:47:21 -07:00
Jez Ng 723c46e645 [lld-macho] Re-add dylink-lazy test
The initial attempt didn't work on Windows; apparently Powershell has a
different syntax for running commands sequentially and concatenating
their outputs. So I've created two temporary files instead.

Differential Revision: https://reviews.llvm.org/D79697
2020-05-11 12:39:21 -07:00
Peter Smith 0ae7990b60 [ELF][ARM] Support /DISCARD/ of subset of .ARM.exidx sections
Both the .ARM.exidx and .eh_frame sections have a custom SyntheticSection
that acts as a container for the InputSections. The InputSections are added
to the SyntheticSection prior to /DISCARD/ which limits the affect a
/DISCARD/ can have to the whole SyntheticSection. In the majority of cases
this is sufficient as it is not common to discard subsets of the
InputSections. The Linux kernel has one of these scripts which has something
like:
/DISCARD/ : { *(.ARM.exidx.exit.text) *(.ARM.extab.exit.text) ... }
The .ARM.exidx.exit.text are not discarded because the InputSection has been
transferred to the Synthetic Section. The *(.ARM.extab.exit.text) sections
have not so they are discarded. When we come to write out the .ARM.exidx
sections the dangling references from .ARM.exidx.exit.text to
.ARM.extab.exit.text currently cause relocation out of range errors, but
could as easily cause a fatal error message if we check for dangling
references at relocation time.

This patch attempts to respect the /DISCARD/ command by running it on the
.ARM.exidx InputSections stored in the SyntheticSection.

The .eh_frame is in theory affected by this problem, but I don't think that
there is a dangling reference problem that can happen with these sections.

Fixes remaining part of pr44824

Differential Revision: https://reviews.llvm.org/D79687
2020-05-11 14:27:13 +01:00
Martin Storsjö ed0a57f753 [LLD] [COFF] Fix def file exporting of symbols containing periods
This fixes an accidental breakage of exporting symbols using def
files, when the symbol name contains a period, since commit
0ca06f7950, mixing up a symbol name containing a period with
the case of exporting a symbol as a forward to another dll.

Differential Revision: https://reviews.llvm.org/D79619
2020-05-10 23:30:14 +03:00
Jez Ng 0dcd7c34f1 [lld-macho] Delete dylink-lazy test for now
Summary: It's breaking Windows builds
2020-05-10 02:22:33 -07:00
Jez Ng 8a5e45aa90 [lld-macho] Try to unbreak dylink-lazy test on Windows 2020-05-10 00:23:32 -07:00
Jez Ng 198b0c57df [lld-macho] Support pc-relative section relocations
Summary: So far we've only supported symbol relocations.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79211
2020-05-09 20:56:23 -07:00
Jez Ng 7bbdbacd00 [lld-macho] Use export trie instead of symtab when linking against dylibs
Summary:
This allows us to link against stripped dylibs. Moreover, it's simply
more correct: The symbol table includes symbols that the dylib uses but
doesn't export.

This temporarily regresses our ability to do lazy symbol binding because
dyld_stub_binder isn't in libSystem's export trie. Rather, it is in one
of the sub-libraries libSystem re-exports. (This doesn't affect our
tests since we are mocking out dyld_stub_binder there.) A follow-up diff
will address this by adding support for sub-libraries.

Depends on D79114.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: mgorny, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79226
2020-05-09 20:56:22 -07:00
Jez Ng 5d3feefa0d [lld-macho] Dylib symbols should always replace undefined symbols
Summary:
Otherwise we get undefined symbol errors depending on the order of
arguments on the command line.

Depends on D78270.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79114
2020-05-09 20:56:22 -07:00
Jez Ng b3e2fc931d [lld-macho] Support calls to functions in dylibs
Summary:
This diff implements lazy symbol binding -- very similar to the PLT
mechanism in ELF.

ELF's .plt section is broken up into two sections in Mach-O:
StubsSection and StubHelperSection. Calls to functions in dylibs will
end up calling into StubsSection, which contains indirect jumps to
addresses stored in the LazyPointerSection (the counterpart to ELF's
.plt.got).

Initially, the LazyPointerSection contains addresses that point into one
of the entry points in the middle of the StubHelperSection. The code in
StubHelperSection will push on the stack an offset into the
LazyBindingSection. The push is followed by a jump to the beginning of
the StubHelperSection (similar to PLT0), which then calls into
dyld_stub_binder. dyld_stub_binder is a non-lazily bound symbol, so this
call looks it up in the GOT.

The stub binder will look up the bind opcodes in the LazyBindingSection
at the given offset. The bind opcodes will tell the binder to update the
address in the LazyPointerSection to point to the symbol, so that
subsequent calls don't have to redo the symbol resolution. The binder
will then jump to the resolved symbol.

Depends on D78269.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78270
2020-05-09 20:56:22 -07:00
Jez Ng db157d2733 [lld-macho] Follow-up to D77893
Summary:
1. Don't have isHidden() depend on isNeeded(). Whether a section is
  hidden is orthogonal from whether it is needed: hidden sections will
  never have a header regardless of whether they have a body. (I know we
  override this method with return false for synthetic sections, but
  regardless I think it's confusing to write it this way for non-synthetic
  sections.)

2. Don't call writeTo() on unneeded sections. D78270 assumes that this
  is true when implementing the stub helper section.

3. Filter out the unneeded sections early on to avoid having to deal
   with them in multiple places.

4. Remove assumption in test that the referenced file has no other symbols.
  (We should create separate input files for future tests to avoid such
  issues.)

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79460
2020-05-09 20:56:22 -07:00
Thomas Lively a1ae9566ea [WebAssembly] Disallow 'shared-mem' rather than 'atomics'
Summary:
The WebAssembly backend automatically lowers atomic operations and TLS
to nonatomic operations and non-TLS data when either are present and
the atomics or bulk-memory features are not present, respectively. The
resulting object is no longer thread-safe, so the linker has to be
told not to allow it to be linked into a module with shared
memory. This was previously done by disallowing the 'atomics' feature,
which prevented any objct with its atomic operations or TLS removed
from being linked with any object containing atomics or TLS, and
therefore preventing it from being linked into a module with shared
memory since shared memory requires atomics.

However, as of https://github.com/WebAssembly/threads/issues/144, the
validation rules are relaxed to allow atomic operations to validate
with unshared memories, which makes it perfectly safe to link an
object with stripped atomics and TLS with another object that still
contains TLS and atomics as long as the resulting module has an
unshared memory. To allow this kind of link, this patch disallows a
pseudo-feature 'shared-mem' rather than 'atomics' to communicate to
the linker that the object is not thread-safe. This means that the
'atomics' feature is available to accurately reflect whether or not an
object has atomics enabled.

As a drive-by tweak, this change also requires that bulk-memory be
enabled in addition to atomics in order to use shared memory. This is
because initializing shared memories requires bulk-memory operations.

Reviewers: aheejin, sbc100

Subscribers: dschuff, jgravelle-google, hiraditya, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79542
2020-05-08 13:52:39 -07:00
Wei Mi 538208f6c0 [lld] Add a new output section ".text.unknown" for funtions with unknown hotness
For sampleFDO, because the optimized build uses profile generated from previous
release, often we couldn't tell a function without profile was truely cold or
just newly created so we had to treat them conservatively and put them in .text
section instead of .text.unlikely. The result was when we persue the best
performance by locking .text.hot and .text in memory, we wasted a lot of memory
to keep cold functions inside. This problem has been largely solved for regular
sampleFDO using profile-symbol-list (https://reviews.llvm.org/D66374), but for
the case when we use partial profile, we still waste a lot of memory because
of it.

In https://reviews.llvm.org/D62540, we propose to save functions with unknown
hotness information in a special section called ".text.unknown", so that
compiler will treat those functions as luck-warm, but runtime can choose not
to mlock the special section in memory or use other strategy to save memory.
That will solve most of the memory problem even if we use a partial profile.

The patch adds the support in lld for the special section.For sampleFDO,
because the optimized build uses profile generated from previous release,
often we couldn't tell a function without profile was truely cold or just
newly created so we had to treat them conservatively and put them in .text
section instead of .text.unlikely. The result was when we persue the best
performance by locking .text.hot and .text in memory, we wasted a lot of
memory to keep cold functions inside. This problem has been largely solved
for regular sampleFDO using profile-symbol-list
(https://reviews.llvm.org/D66374), but for the case when we use partial
profile, we still waste a lot of memory because of it.

In https://reviews.llvm.org/D62540, we propose to save functions with unknown
hotness information in a special section called ".text.unknown", so that
compiler will treat those functions as luck-warm, but runtime can choose not
to mlock the special section in memory or use other strategy to save memory.
That will solve most of the memory problem even if we use a partial profile.

The patch adds the support in lld for the special section.

Differential Revision: https://reviews.llvm.org/D79590
2020-05-08 11:14:48 -07:00
Fangrui Song e20a215992 [ELF] Add convenience TableGen classes to enforce two dashes for options not supported by GNU ld
Announced on https://lists.llvm.org/pipermail/llvm-dev/2020-May/141416.html

For many options, we have to support either one or two dash to be
compatible with GNU ld. For newer and lld specific options, we can enforce strict double dashes.

Affected options:

* --thinlto-*
* --lto-*
* --shuffle-sections=

This patch does not change `-plugin-opt=*` because clang driver passes
`-plugin-opt=*` and I don't intend to cause churn.

In 2000, GNU ld tried something similar with --omagic
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=e4897a3288f37d5f69e8acd256a6e83e607fe8d8

Reviewed By: tejohnson, psmith

Differential Revision: https://reviews.llvm.org/D79371
2020-05-08 07:37:06 -07:00
Reid Kleckner d71c3c425c [COFF] Dump string table size for COFF file headers
I couldn't find this info in any other dumper, so it might as well be
here.
2020-05-06 15:48:36 -07:00
Sam Clegg f03b6e785b [lld][WebAssembly] Honor --allow-undefined for data symbols too
This was originally the way this worked before before
https://reviews.llvm.org/D60882.

In retrospect it seems inconsistent that `--allow-undefined` doesn't
work for all symbols.  See:
https://groups.google.com/g/emscripten-discuss/c/HSRgQiIq1gI/m/Kt9oFWHiAwAJ

I'm also planning a followup change which implement the full
`--unresolved-symbols=..` flags supported by ELF linkers (both ld and
ld.lld) since it seems more standard.

Differential Revision: https://reviews.llvm.org/D79247
2020-05-06 12:39:29 -07:00
Alexandre Ganea 6adc45d3fd [LLD][COFF] Move debug info for thread-local variables into PDB global stream
Before this patch, the debug record S_GTHREAD32 which represents global thread_local symbols, was emitted by LLD into the respective module stream. This makes Visual Studio unable to display thread_local symbols in the debugger.

After this patch, S_GTHREAD32 is moved into the globals stream. This matches MSVC behavior.

Differential Revision: https://reviews.llvm.org/D79005
2020-05-06 15:23:58 -04:00
Sid Manning 0e6536fd97 [Hexagon] Add R_HEX_GD_PLT_B22/32_PCREL relocations
Extended versions of GD_PLT_B22_PCREL. These surface when -mlong-calls
is used.

Differential Revision: https://reviews.llvm.org/D79191
2020-05-05 11:47:51 -05:00
Peter Smith 48aebfc908 [ELF][ARM] Do not create .ARM.exidx sections for out of range inputs
A linker will create .ARM.exidx sections for InputSections that don't
have them. This can cause a relocation out of range error If the
InputSection happens to be extremely far away from the other sections.
This is often the case for the vector table on older ARM CPUs as the only
two places that the table can be placed is 0 or 0xffff0000. We fix this
by removing InputSections that need a linker generated .ARM.exidx
section if that would cause an error.

Differential Revision: https://reviews.llvm.org/D79289
2020-05-05 09:59:45 +01:00
Zakk Chen ad5fad0ac5 [LTO] Suppress emission of empty combined module by default
Summary:
That unless the user requested an output object (--lto-obj-path), the an
unused empty combined module is not emitted.

This changed is helpful for some target (ex. RISCV-V) which encoded the
ABI info in IR module flags (target-abi). Empty unused module has no ABI
info so the linker would get the linking error during merging
incompatible ABIs.

Reviewers: tejohnson, espindola, MaskRay

Subscribers: emaste, inglorion, arichardson, hiraditya, simoncook, MaskRay, steven_wu, dexonsmith, PkmX, dang, lenary, s.egerton, luismarques, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78988
2020-05-04 18:31:09 -07:00
Fangrui Song 6939fe6e08 [lld-macho] Support X86_64_RELOC_SIGNED_{1,2,4}
We currently only support extern relocations.
`X86_64_RELOC_SIGNED_{1,2,4}` are like X86_64_RELOC_SIGNED, but with the
implicit addend fixed to 1, 2, and 4, respectively.
See the comment in `lib/Target/X86/MCTargetDesc/X86MachObjectWriter.cpp RecordX86_64Relocation`.

Reviewed By: int3

Differential Revision: https://reviews.llvm.org/D79311
2020-05-04 15:15:35 -07:00
Fangrui Song c49f83b6e9 [ELF] Don't advance sh_offset for an empty section whose PT_LOAD is removed (due to p_memsz=0)
removeEmptyPTLoad() removes empty (p_memsz=0) PT_LOAD segments.  In
assignFileOffsets(), setFileOffset() unnecessarily advances file offsets
for containing empty sections.

This is exposed by arm Linux kernel's multi_v5_defconfig
(see https://bugs.llvm.org/show_bug.cgi?id=45632)

```
ld.lld (max-page-size=65536):
  [34] .init.data        PROGBITS        c0c24000 c34000 0128ac 00  WA  0   0 4096
  [35] .text_itcm        PROGBITS        fffe0000 c50000 000000 00  WA  0   0  1
  [36] .data_dtcm        PROGBITS        fffe8000 c58000 000000 00  WA  0   0  1
  [37] .data             PROGBITS        c0c38000 c58000 0647a0 00  WA  0   0 32

arm-linux-gnueabi-ld (max-page-size=65536):
  [23] .init.data        PROGBITS        c0c12000 c22000 0128ac 00  WA  0   0 4096
  [24] .text_itcm        PROGBITS        fffe0000 ca2558 000000 00   W  0   0  1
  [25] .data_dtcm        PROGBITS        fffe8000 ca2558 000000 00   W  0   0  1
  [26] .data             PROGBITS        c0c26000 c36000 0647a0 00  WA  0   0 32
```

This patch clears OutputSection::ptLoad if ptLoad is removed by
removeEmptyPTLoad(). Conceptually this removes "dangling" references.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D79254
2020-05-04 08:07:34 -07:00
Peter Smith 3834385f27 [ELF] Move SHF_LINK_ORDER till OutputSection addresses are known
Sections with the SHF_LINK_ORDER flag must be ordered in the same relative
order as the Sections they have a link to. When using a linker script an
arbitrary expression may be used for the virtual address of the
OutputSection. In some cases the virtual address does not monotonically
increase as the OutputSection index increases, so if we base the ordering
of the SHF_LINK_ORDER sections on the index then we can get the order
wrong. We fix this by moving SHF_LINK_ORDER resolution till after we have
created OutputSection virtual addresses.

Differential Revision: https://reviews.llvm.org/D79286
2020-05-04 14:25:25 +01:00
Thomas Preud'homme d735c7048c [test] Fix lld's ELF/linkerscript/thunk-gen-mips.s
Summary:
Lld test ELF/linkerscript/thunk-gen-mips.s was accidentally disabled due
to the use of wrong FileCheck directives. As a result the test seems to
have bitrotted as it fails to pass if fixing the directive. To ease
updates to the test in case of change of the __start address the checks
have been changed to use numeric variables to express all the addresses
based on the __start address.

Reviewed By: atanasyan

Differential Revision: https://reviews.llvm.org/D79270
2020-05-02 22:49:23 +01:00
Kellie Medlin 6cb073133c [lld] Merge Mach-O input sections
Summary: Similar to other formats, input sections in the MachO
implementation are now grouped under output sections. This is primarily
a refactor, although there's some new logic (like resolving the output
section's flags based on its inputs).

Differential Revision: https://reviews.llvm.org/D77893
2020-05-01 16:57:18 -07:00
Fangrui Song 5c86b08a6f [ELF][test] Improve tests
Prepare for the upcomong change that removes unneeded sh_offset
advancement for empty sections whose PT_LOAD are removed.
2020-05-01 11:27:51 -07:00
Sam Clegg 0a6c4d8d2e [WebAssmebly] Add support for defined wasm globals in MC and lld
This change add support for defined wasm globals in the .s format,
the MC layer, and wasm-ld

Currently there is no support custom initialization and all wasm
globals are initialized to zero.

Fixes: PR45742

Differential Revision: https://reviews.llvm.org/D79137
2020-04-30 12:43:15 -07:00
Thomas Preud'homme 9ecddde321 [test] Fix ELF/linkerscript/input-archive.s w/ @ in path
Lld test ELF/linkerscript/input-archive.s fails when path contain a @
because is not accepted in unquoted token in linker scripts which leads
to the path being broken in 2 around the @. This commit quotes the path
used in the linker script created by this and similar testcases allowing
the test to pass even in the presence of an @ sign in the path.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D79103
2020-04-30 20:14:22 +01:00
Fangrui Song b257d3c8a8 [ELF][PPC64] Suppress toc-indirect to toc-relative relaxation if R_PPC64_TOC16_LO is seen
The current implementation assumes that R_PPC64_TOC16_HA is always followed
by R_PPC64_TOC16_LO_DS. This can break with R_PPC64_TOC16_LO:

  // Load the address of the TOC entry, instead of the value stored at that address
  addis 3, 2, .LC0@tloc@ha  # R_PPC64_TOC16_HA
  addi  3, 3, .LC0@tloc@l   # R_PPC64_TOC16_LO
  blr

which is used by boringssl's util/fipstools/delocate/delocate.go
https://github.com/google/boringssl/blob/master/crypto/fipsmodule/FIPS.md has some documentation.
In short, this tool converts an assembly file to avoid any potential relocations.
The distance to an input .toc is not a constant after linking, so it cannot use an `addis;ld` pair.
Instead, it jumps to a stub which loads the TOC entry address with `addis;addi`.

This patch checks the presence of R_PPC64_TOC16_LO and suppresses
toc-indirect to toc-relative relaxation if R_PPC64_TOC16_LO is seen.
This approach is conservative and loses some relaxation opportunities but is easy to implement.

  addis 3, 2, .LC0@toc@ha  # no relaxation
  addi  3, 3, .LC0@toc@l   # no relaxation
  li    9, 0
  addis 4, 2, .LC0@toc@ha  # can relax but suppressed
  ld    4, .LC0@toc@l(4)   # can relax but suppressed

Also note that interleaved R_PPC64_TOC16_HA and R_PPC64_TOC16_LO_DS is
possible and this patch accounts for that.

  addis 3, 2, .LC1@toc@ha  # can relax
  addis 4, 2, .LC2@toc@ha  # can relax
  ld    3, .LC1@toc@l(3)   # can relax
  ld    4, .LC2@toc@l(4)   # can relax

Reviewed By: #powerpc, sfertile

Differential Revision: https://reviews.llvm.org/D78431
2020-04-30 09:16:51 -07:00
Fangrui Song b912b887d8 [ELF] Add --print-archive-stats=
gold has an option --print-symbol-counts= which prints:

  // For each archive
  archive $archive $members $fetched_members
  // For each object file
  symbols $object $defined_symbols $used_defined_symbols

In most cases, `$defined_symbols = $used_defined_symbols` unless weak
symbols are present. Strangely `$used_defined_symbols` includes symbols defined relative to --gc-sections discarded sections.
The `symbols` lines do not appear to be useful.

`archive` lines are useful: `$fetched_members=0` lines correspond to
unused archives. The information can be used to trim dependencies.

This patch implements --print-archive-stats= which prints the number of
members and the number of fetched members for each archive.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D78983
2020-04-29 18:04:37 -07:00
Fangrui Song e96d7b5e9e [ELF] Add --rosegment to complement --no-rosegment
This option can cancel --no-rosegment and it just seems right to have
a corresponding positive option for a --no-* negative option.

Anecdotally, gold had --rosegment but did not have --no-rosegment.
I added --no-rosegment (https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=9a6c68caa9543e09b064b7ac7c2b658f277bc19c) for binutils>=2.35
2020-04-29 18:00:00 -07:00
Jez Ng e82c5e17b5 [lld-macho] Support X86_64_RELOC_BRANCH
Relatively straightforward diff, to set the stage for calling functions
in dylibs.

Differential Revision: https://reviews.llvm.org/D78269
2020-04-29 15:45:01 -07:00
Jez Ng 918948db4d [lld-macho] Support reading of universal binaries
Differential Revision: https://reviews.llvm.org/D77006
2020-04-29 15:44:44 -07:00
Jez Ng 89285a1a97 [lld-macho] Disable colors in errors when not printing to a pty
This makes for better tests (and is just the right thing to do)

Differential Revision: https://reviews.llvm.org/D79069
2020-04-29 15:44:35 -07:00
Jez Ng 9854edd817 [lld-macho] Implement basic export trie
Build the trie by performing a three-way radix quicksort: We start by
sorting the strings by their first characters, then sort the strings
with the same first characters by their second characters, and so on
recursively. Each time the prefixes diverge, we add a node to the trie.
Thanks to @ruiu for the idea.

I used llvm-mc's radix quicksort implementation as a starting point. The
trie offset fixpoint code was taken from
MachONormalizedFileBinaryWriter.cpp.

Differential Revision: https://reviews.llvm.org/D76977
2020-04-29 15:44:27 -07:00
Fangrui Song 1ccde53342 [ELF] --gdb-index: support .debug_loclists
--gdb-index currently crashes when reading a translation unit with
DWARF v5 .debug_loclists . Call stack:

```
SyntheticSections.cpp GdbIndexSection::create
SyntheticSections.cpp readAddressAreas
DWARFUnit.cpp DWARFUnit::tryExtractDIEsIfNeeded
DWARFListTable.cpp DWARFListTableHeader::extract
...
DWARFDataExtractor.cpp DWARFDataExtractor::getRelocatedValue
lld/ELF/DWARF.cpp LLDDwarfObj<ELFT>::find (sec.sec is nullptr)
...

```

This patch adds support for .debug_loclists to make `DWARFUnit::tryExtractDIEsIfNeeded` happy.
Building --gdb-index does not need .debug_loclists

Reviewed By: dblaikie, grimar

Differential Revision: https://reviews.llvm.org/D79061
2020-04-29 15:04:13 -07:00
Fangrui Song 53ff95254d Reland D78837 [lld] Remove special cases from default ld driver mode.
Drops the behavior from rL217112.

Use the Gnu driver mode by default for all platforms when ld is
invoked. Other names for the program (such as link or ld64) continue
working as before.

Reviewed By: MaskRay, srhines, smeenai, ruiu

Differential Revision: https://reviews.llvm.org/D78837
2020-04-29 14:45:44 -07:00
Sean Fertile f9106e85c4 Revert "[ELF][PPC64] Don't perform toc-indirect to toc-relative relax... "
This reverts commit 03ffe58605.

Full tile of reverted commit is:
[ELF][PPC64] Don't perform toc-indirect to toc-relative relaxation for
R_PPC64_TOC16_HA not followed by R_PPC64_TOC16_LO_DS

Breaks the multistage lld PowerPC buildbot.
2020-04-29 10:30:35 -04:00
Saleem Abdulrasool 216833b32b Revert "Temporarily revert "build: use `find_package(Python3)` if available""
This reverts commit 35edd704e0.

Revert the revert and extend the patch further to account for the use of
the `PYTHONINTERP_FOUND`.
2020-04-29 01:38:08 +00:00
Jez Ng 62b8f32f76 [lld-macho][reland] Add support for emitting dylibs with a single symbol
This got reverted due to UBSAN errors in a diff lower in the stack,
which is being fixed in https://reviews.llvm.org/D79050. This diff is
otherwise identical to the original https://reviews.llvm.org/D76908
(which was committed in 9598778bd1 and reverted in b52bc2653b).

Differential Revision: https://reviews.llvm.org/D79051
2020-04-28 17:08:32 -07:00
Jez Ng 4f0cccdd7a [lld-macho][reland] Add basic symbol table output
This diff implements basic support for writing a symbol table.

Attributes are loosely supported for extern symbols and not at all for
other types.

Initial version by Kellie Medlin <kelliem@fb.com>

Originally committed in a3d95a50ee and reverted in fbae153ca5 due to
UBSAN erroring over unaligned writes. That has been fixed in the
current diff with the following changes:

```
diff --git a/lld/MachO/SyntheticSections.cpp b/lld/MachO/SyntheticSections.cpp
--- a/lld/MachO/SyntheticSections.cpp
+++ b/lld/MachO/SyntheticSections.cpp
@@ -133,6 +133,9 @@ SymtabSection::SymtabSection(StringTableSection &stringTableSection)
     : stringTableSection(stringTableSection) {
   segname = segment_names::linkEdit;
   name = section_names::symbolTable;
+  // TODO: When we introduce the SyntheticSections superclass, we should make
+  // all synthetic sections aligned to WordSize by default.
+  align = WordSize;
 }

 size_t SymtabSection::getSize() const {
diff --git a/lld/MachO/Writer.cpp b/lld/MachO/Writer.cpp
--- a/lld/MachO/Writer.cpp
+++ b/lld/MachO/Writer.cpp
@@ -371,6 +371,7 @@ void Writer::assignAddresses(OutputSegment *seg) {
     ArrayRef<InputSection *> sections = p.second;
     for (InputSection *isec : sections) {
       addr = alignTo(addr, isec->align);
+      // We must align the file offsets too to avoid misaligned writes of
+      // structs.
+      fileOff = alignTo(fileOff, isec->align);
       isec->addr = addr;
       addr += isec->getSize();
       fileOff += isec->getFileSize();
@@ -396,6 +397,7 @@ void Writer::writeSections() {
     uint64_t fileOff = seg->fileOff;
     for (auto &sect : seg->getSections()) {
       for (InputSection *isec : sect.second) {
+        fileOff = alignTo(fileOff, isec->align);
         isec->writeTo(buf + fileOff);
         fileOff += isec->getFileSize();
       }
```

I don't think it's easy to write a test for alignment (that doesn't
involve brittly hard-coding file offsets), so there isn't one... but
UBSAN builds pass now.

Differential Revision: https://reviews.llvm.org/D79050
2020-04-28 17:07:06 -07:00
Eric Christopher 35edd704e0 Temporarily revert "build: use `find_package(Python3)` if available"
as it seems to be causing multiple people problems with running tests
and building.

This reverts commit c4c3883b00.
2020-04-28 16:41:22 -07:00
Fangrui Song 03ffe58605 [ELF][PPC64] Don't perform toc-indirect to toc-relative relaxation for R_PPC64_TOC16_HA not followed by R_PPC64_TOC16_LO_DS
The current implementation assumes that R_PPC64_TOC16_HA is always followed
by R_PPC64_TOC16_LO_DS. This can break with:

// Load the address of the TOC entry, instead of the value stored at that address
  addis 3, 2, .LC0@tloc@ha  # R_PPC64_TOC16_HA
  addi  3, 3, .LC0@tloc@l   # R_PPC64_TOC16_LO
  blr

which is used by boringssl's util/fipstools/delocate/delocate.go
https://github.com/google/boringssl/blob/master/crypto/fipsmodule/FIPS.md has some documentation.
In short, this tool converts an assembly file to avoid any potential relocations.
The distance to an input .toc is not a constant after linking, so the assembly cannot use an `addis;ld` pair.
Instead, delocate changes the code to jump to a stub (`addis;addi`) which loads the TOC entry address.

Reviewed By: sfertile

Differential Revision: https://reviews.llvm.org/D78431
2020-04-28 12:13:27 -07:00
Shoaib Meenai fbae153ca5 Revert "[lld-macho] Add basic symbol table output"
This reverts commit a3d95a50ee.

Reverting due to UBSan failures:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/40817/steps/check-lld%20ubsan/logs/stdio
2020-04-28 11:34:03 -07:00
Shoaib Meenai b52bc2653b Revert "[lld-macho] Add support for emitting dylibs with a single symbol"
This reverts commit 9598778bd1.

Reverting due to UBSan failures:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/40817/steps/check-lld%20ubsan/logs/stdio
2020-04-28 11:34:03 -07:00
Saleem Abdulrasool c4c3883b00 build: use `find_package(Python3)` if available
This is primarily motivated by the desire to move from Python2 to
Python3.  `PYTHON_EXECUTABLE` is ambiguous.  This explicitly identifies
the python interpreter in use.  Since the LLVM build seems to be able to
completed successfully with python3, use that across the build.  The old
path aliases `PYTHON_EXECUTABLE` to be treated as Python3.
2020-04-28 09:24:27 -07:00
Igor Kudrin 9f65f5acca [LLD][ELF] Eliminate symbols of merged .ARM.exidx sections.
GNU tools generate mapping symbols "$d" for .ARM.exidx sections. The
symbols are added to the symbol table much earlier than the merging
takes place, and after that, they become dangling. Before the patch,
LLD output those symbols as SHN_ABS with the value of 0. The patch
removes such symbols from the symbol table.

Differential Revision: https://reviews.llvm.org/D78820
2020-04-28 18:58:40 +07:00
Jez Ng 9598778bd1 [lld-macho] Add support for emitting dylibs with a single symbol
Summary:
Add logic for emitting the correct set of load commands and segments
when `-dylib` is passed.

I haven't gotten to implementing a real export trie yet, so we can only
emit a single symbol, but it's enough to replace the YAML test files
introduced in D76252.

Differential Revision: https://reviews.llvm.org/D76908
2020-04-27 13:33:46 -07:00
Jez Ng a3d95a50ee [lld-macho] Add basic symbol table output
This diff implements basic support for writing a symbol table.

- Attributes are loosely supported for extern symbols and not at all for
  other types

Immediate future work will involve implementing section merging.

Initial version by Kellie Medlin <kelliem@fb.com>

Differential Revision: https://reviews.llvm.org/D76742
2020-04-27 13:33:15 -07:00
Jez Ng 6f63216c3d [lld-macho] Extend SyntheticSections to cover all segment load commands
Previously, the special segments `__PAGEZERO` and `__LINKEDIT` were
implemented as special LoadCommands. This diff implements them using
special sections instead which have an `isHidden()` attribute. We do not
emit section headers for hidden sections, but we use their addresses and
file offsets to determine that of their containing segments. In addition
to allowing us to share more segment-related code, this refactor is also
important for the next step of emitting dylibs:

1) dylibs don't have segments like __PAGEZERO, so we need an easy way of
   omitting them w/o messing up segment indices
2) Unlike the kernel, which is happy to run an executable with
   out-of-order segments, dyld requires dylibs to have their segment
   load commands arranged in increasing address order. The refactor
   makes it easier to implement sorting of sections and segments.

Differential Revision: https://reviews.llvm.org/D76839
2020-04-27 12:58:12 -07:00
Nico Weber 76f74d15fe remove temporary logging again now that things are figured out 2020-04-27 14:01:37 -04:00
Hongtao Yu 964ef8eecc [lld] Support --lto-emit-asm and --plugin-opt=emit-asm
Summary: The switch --plugin-opt=emit-asm can be used with the gold linker to dump the final assembly code generated by LTO in a user-friendly way. Unfortunately it doesn't work with lld. I'm hooking it up with lld. With that switch, lld emits assembly code into the output file (specified by -o) and if there are multiple input files, each of their assembly code will be emitted into a separate file named by suffixing the output file name with a unique number, respectively. The linking then stops after generating those assembly files.

Reviewers: espindola, wenlei, tejohnson, MaskRay, grimar

Reviewed By: tejohnson, MaskRay, grimar

Subscribers: pcc, emaste, inglorion, arichardson, hiraditya, MaskRay, steven_wu, dexonsmith, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77231
2020-04-27 11:00:46 -07:00
Sam Clegg 4b8e2d8e81 [lld][WebAssembly] Fix crash on function signature mismatch with --relocatable
These stub new function were not being added to the symbol table
which in turn meant that we were crashing when trying to output
relocations against them.

Differential Revision: https://reviews.llvm.org/D78779
2020-04-25 10:26:11 -07:00
Igor Kudrin 66e4eb9c1b [LLD][ELF] Implement --discard-* for cases when -r or --emit-relocs are used.
When discarding local symbols with --discard-all or --discard-locals,
the ones which are used in relocations should be preserved. LLD used
the simplest approach and just ignored those switches when -r or
--emit-relocs was specified.

The patch implements handling the --discard-* switches for the cases
when relocations are kept by identifying used local symbols and allowing
removing only unused ones. This makes the behavior of LLD compatible
with GNU linkers.

Differential Revision: https://reviews.llvm.org/D77807
2020-04-25 18:59:41 +07:00
Peter Smith 3b1622d63a [LLD][ELF][ARM] recommit Fix ARM Exidx order for non monotonic section order
Fixed error detected by msan. The size field of the .ARM.exidx synthetic
section needs to be initialized to at least estimation level before
calling assignAddresses as that will use the size field.

This was previously reverted in 1ca16fc4f5.

Differential Revision: https://reviews.llvm.org/D78422
2020-04-24 13:47:28 +01:00
Peter Smith 1ca16fc4f5 Revert "[LLD][ELF][ARM] Fix ARM Exidx order for non monotonic section order"
This reverts commit f969c2aa65.

There are some msan buildbot failures sanitzer-x86_64-linux-fast that
I need to investigate.

Differential Revision: https://reviews.llvm.org/D78422
2020-04-23 16:58:50 +01:00
Peter Smith f969c2aa65 [LLD][ELF][ARM] Fix ARM Exidx order for non monotonic section order
The contents of the .ARM.exidx section must be ordered by SHF_LINK_ORDER
rules. We don't need to know the precise address for this order, but we
do need to know the relative order of sections. We have been using the
sectionIndex for this purpose, this works when the OutputSection order
has a monotonically increasing virtual address, but it is possible to
write a linker script with non-monotonically increasing virtual address.
For these cases we need to evaluate the base address of the OutputSection
so that we can order the .ARM.exidx sections properly.

This change moves the finalisation of .ARM.exidx till after the first
call to AssignAddresses. This permits us to sort on virtual address which
is linker script safe. It also permits a fix for part of pr44824 where
we generate .ARM.exidx section for the vector table when that table is so
far away it is out of range of the .ARM.exidx section. This fix will come
in a follow up patch.

Differential Revision: https://reviews.llvm.org/D78422
2020-04-23 15:46:44 +01:00
Sergej Jaskiewicz 2899103108 [TimeProfiler] Emit clock synchronization point
Time profiler emits relative timestamps for events (the number of
microseconds passed since the start of the current process).

This patch allows combining events from different processes while
preserving their relative timing by emitting a new attribute
"beginningOfTime". This attribute contains the system time that
corresponds to the zero timestamp of the time profiler.

This has at least two use cases:

- Build systems can use this to merge time traces from multiple compiler
  invocations and generate statistics for the whole build. Tools like
  ClangBuildAnalyzer could also leverage this feature.

- Compilers that use LLVM as their backend by invoking llc/opt in
  a child process. If such a compiler supports generating time traces
  of its own events, it could merge those events with LLVM-specific
  events received from llc/opt, and produce a more complete time trace.

A proof-of-concept script that merges multiple logs that
contain a synchronization point into one log:
https://github.com/broadwaylamb/merge_trace_events

Differential Revision: https://reviews.llvm.org/D78030
2020-04-23 01:09:31 +03:00
Sergej Jaskiewicz a5bf02815d [TimeProfiler] Emit real process ID and thread names
Differential Revision: https://reviews.llvm.org/D78027
2020-04-23 00:12:51 +03:00
Fangrui Song c384ca3c6a [ELF] For relative paths in INPUT() and GROUP(), search the directory of the current linker script before searching other paths
For a relative path in INPUT() or GROUP(), this patch changes the search order by adding the directory of the current linker script.
The new search order (consistent with GNU ld >= 2.35 regarding the new test `test/ELF/input-relative.s`):

1. the directory of the current linker script (GNU ld from Binutils 2.35 onwards; https://sourceware.org/bugzilla/show_bug.cgi?id=25806)
2. the current working directory
3. library paths (-L)

This behavior makes it convenient to replace a .so or .a with a linker script with additional input. For example, glibc

```
% cat /usr/lib/x86_64-linux-gnu/libm.a
/* GNU ld script
*/
OUTPUT_FORMAT(elf64-x86-64)
GROUP ( /usr/lib/x86_64-linux-gnu/libm-2.29.a /usr/lib/x86_64-linux-gnu/libmvec.a )
```

could be simplified as `GROUP(libm-2.29.a libmvec.a)`.

Another example is to make libc++.a a linker script:
```
INPUT(libc++.a.1 libc++abi.a)
```

Note, -l is not affected.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D77779
2020-04-22 12:34:20 -07:00
Peter Smith 673612a035 [LLD][ELF][ARM] Replace adr, ldr with .inst .reloc in test [NFC]
After D78301 MC no longer emits a relocation for this case. Change to use
.inst and .reloc to synthesize the same instruction and relocation. One
more test case I missed.
2020-04-22 12:55:26 +01:00
Fangrui Song 01d2a01e79 [ELF] Fix a null pointer dereference when relocating a Local-Exec TLS relocation for a lazy symbol
If there is no SHF_TLS section, there will be no PT_TLS and Out::tlsPhdr may be a nullptr.
If the symbol referenced by an R_TLS is lazy, we should treat the symbol as undefined.

Also reorganize tls-in-archive.s and tls-weak-undef.s . They do not test what they intended to test.
2020-04-21 15:39:31 -07:00
Jez Ng 060efd24c7 [lld-macho] Add basic support for linking against dylibs
This diff implements:

* dylib loading (much of which is being restored from @pcc and @ruiu's
  original work)
* The GOT_LOAD relocation, which allows us to load non-lazy dylib
  symbols
* Basic bind opcode emission, which tells `dyld` how to populate the GOT

Differential Revision: https://reviews.llvm.org/D76252
2020-04-21 13:43:19 -07:00
Fangrui Song 497c76e96d [ELF] Keep local symbols when both --emit-relocs and --discard-all are specified
This fixes a bug as exposed by D77807.

Add tests for {--emit-relocs,-r} x {--discard-locals,--discard-all}. They add coverage for previously undertested cases:

* STT_SECTION associated to GCed sections (`gc`)
* STT_SECTION associated to retained sections (`text`)
* STT_SECTION associated to non-SHF_ALLOC sections (`.comment`)
* STB_LOCAL in GCed sections (`unused_gc`)

Reviewed By: grimar, ikudrin

Differential Revision: https://reviews.llvm.org/D78389
2020-04-21 08:28:12 -07:00
Fangrui Song 58207d6fe1 [ELF] Fix "TLS attribute mismatch" false positives for STT_NOTYPE undefined symbols
D13550 added the diagnostic to address/work around a crash.
The rule was refined by D19836 (test/ELF/tls-archive.s) to exclude Lazy symbols.

https://bugs.llvm.org/show_bug.cgi?id=45598 reported another case where the current logic has a false positive:

Bitcode does not record undefined module-level inline assembly symbols
(`IRSymtab.cpp:Builder::addSymbol`). Such an undefined symbol does not
have the FB_tls bit and lld will not consider it STT_TLS. When the symbol is
later replaced by a STT_TLS Defined, lld will error "TLS attribute mismatch".

This patch fixes this false positive by allowing a STT_NOTYPE undefined
symbol to be replaced by a STT_TLS.

Considered alternative:

Moving the diagnostics to scanRelocs() can improve the diagnostics (PR36049)
but that requires a fair amount of refactoring. We will need more
RelExpr members. It requires more thoughts whether it is worthwhile.

See `test/ELF/tls-mismatch.s` for behavior differences. We will fail to
diagnose a likely runtime bug (STT_NOTYPE non-TLS relocation referencing
a TLS definition). This is probably acceptable because compiler
generated code sets symbol types properly.

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D78438
2020-04-21 07:56:35 -07:00
Fangrui Song 6e1fe78569 [ELF][test] Reorganize "TLS attribute mismatch" tests 2020-04-21 07:56:35 -07:00
Eli Friedman 875e26d343 Fix lld testcases I missed in D78413 (9b9454a) 2020-04-20 12:48:57 -07:00
Fangrui Song 232578804a [ELF] Add --warn-backrefs-exclude=<glob>
D77522 changed --warn-backrefs to not warn for linking sandwich
problems (-ldef1 -lref -ldef2). This removed lots of false positives.

However, glibc still has some problems. libc.a defines some symbols
which are normally in libm.a and libpthread.a, e.g. __isnanl/raise.

For a linking order `-lm -lpthread -lc`, I have seen:

```
// different resolutions: GNU ld/gold select libc.a(s_isnan.o) as the definition
backward reference detected: __isnanl in libc.a(printf_fp.o) refers to libm.a(m_isnanl.o)

// different resolutions: GNU ld/gold select libc.a(raise.o) as the definition
backward reference detected: raise in libc.a(abort.o) refers to libpthread.a(pt-raise.o)
```

To facilitate deployment of --warn-backrefs, add --warn-backrefs-exclude= so that
certain known issues (which may be impractical to fix) can be whitelisted.

Deliberate choices:

* Not a comma-separated list (`--warn-backrefs-exclude=liba.a,libb.a`).
  -Wl, splits the argument at commas, so we cannot use commas.
  --export-dynamic-symbol is similar.
* Not in the style of `--warn-backrefs='*' --warn-backrefs=-liba.a`.
  We just need exclusion, not inclusion. For easier build system
  integration, we should avoid order dependency. With the current
  scheme, we enable --warn-backrefs, and indivial libraries can add
  --warn-backrefs-exclude=<glob> to their LDFLAGS.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D77512
2020-04-20 07:52:15 -07:00
Peter Smith 61bccda9d9 [LLD][ELF][ARM] Convert ADR/LDR to .inst .reloc
After D78301 MC no longer emits a relocation for this case. Change to use
.inst and .reloc to synthesize the same instruction and relocation.
2020-04-20 13:59:05 +01:00
Nico Weber 2f2667b4ef add more temporary logging for a bot-only failure 2020-04-18 20:22:52 -04:00
Tobias Hieta 87383e408d [ELF][ARM] Increase default max-page-size from 4096 to 6536
See http://lists.llvm.org/pipermail/llvm-dev/2020-April/140549.html

For the record, GNU ld changed to 64k max page size in 2014
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=7572ca8989ead4c3425a1500bc241eaaeffa2c89
"[RFC] ld/ARM: Increase maximum page size to 64kB"

Android driver forced 4k page size in AArch64 (D55029) and ARM (D77746).

A binary linked with max-page-size=4096 does not run on a system with a
higher page size configured. There are some systems out there that do
this and it leads to the binary getting `Killed!` by the kernel.

In the non-linker-script cases, when linked with -z noseparate-code
(default), the max-page-size increase should not cause any size
difference. There may be some VMA usage differences, though.

Reviewed By: psmith, MaskRay

Differential Revision: https://reviews.llvm.org/D77330
2020-04-18 08:19:45 -07:00
Nico Weber c7c3b877c8 add temporary logging to help diagnose a bot-only failure 2020-04-17 22:06:37 -04:00
Reid Kleckner 91a6bfed61 [COFF] Assign unique identifiers to ObjFiles from LTO
Use the unique filenames that are used when /lldsavetemps is passed.
After this change, module names for LTO blobs in PDBs will be unique.
Visual Studio and probably other debuggers expect module names to be
unique.

Revert some changes from 1e0b158db (2017) that are no longer necessary
after removing MSVC LTO support.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D78221
2020-04-17 17:15:12 -07:00
LemonBoy aff950e95d [ELF] Support a few more SPARCv9 relocations
Implemented a bunch of relocations found in binaries with medium/large code model and the Local-Exec TLS model. The binaries link and run fine in Qemu.
In addition, the emulation `elf64_sparc` is now recognized.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D77672
2020-04-17 08:12:15 -07:00
Wouter van Oortmerssen 48139ebc3a [WebAssembly] Add int32 DW_OP_WASM_location variant
This to allow us to add reloctable global indices as a symbol.
Also adds R_WASM_GLOBAL_INDEX_I32 relocation type to support it.

See discussion in https://github.com/WebAssembly/debugging/issues/12
2020-04-16 16:32:17 -07:00
Fangrui Song cece7af586 [ELF][test] Add --match-full-lines to map-file.s to check leading and trailing spaces
Since we are going to touch so many lines, do some other cleanups.
Delete Inputs/map-file3.s and Inputs/map-file4.s which are tiny.
2020-04-15 11:49:52 -07:00
Fangrui Song cd5d5ce235 [ELF] Refactor the way we handle -plugin-opt= (GCC collect2 or clang LTO related options)
GCC collect2 passes several options to the linker even if LTO is not used
(note, lld does not support GCC LTO). The lto-wrapper may be a relative
path (especially during development, when gcc is in a build directory), e.g.

  -plugin-opt=relative/path/to/lto-wrapper

We need to ignore such options, which are currently interpreted by
cl::ParseCommandLineOptions() and will fail with `error: --plugin-opt: ld.lld: Unknown command line argument 'relative/path/to/lto-wrapper'`
because the path is apparently not an option registered by an `llvm:🆑:opt`.

See lto-plugin-ignore.s for how we interpret various -plugin-opt= options now.

Reviewed By: grimar, tejohnson

Differential Revision: https://reviews.llvm.org/D78158
2020-04-15 08:00:50 -07:00
Thomas Lively 6474d1b20e [lld][WebAssembly] Do not require --shared-memory with --relocatable
Summary:
wasm-ld requires --shared-memory to be passed when the atomics feature
is enabled because historically atomic operations were only valid with
shared memories. This change relaxes that requirement for when
building relocatable objects because their memories are not
meaningful. This technically maintains the validity of object files
because the threads spec now allows atomic operations with unshared
memories, although we don't support that elsewhere in the tools yet.

This fixes and Emscripten build issue reported at
https://bugs.chromium.org/p/webp/issues/detail?id=463.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D78072
2020-04-14 13:49:28 -07:00
Georgii Rymar 09331fd742 [LLD][ELF][Test] - Fixed the broken test case.
There was no ":" after "SEC", hence the check was
disabled. Found with the use of D78024.
2020-04-14 13:21:49 +03:00
Brian Cain f3da6b7ab5 Add duplex to R_HEX_GOT_16_X
Building 'espresso' from llvm-test-suite revealed missing support
for duplex instructions with R_HEX_GOT_16_X.
2020-04-13 19:32:44 -05:00
Martin Storsjö 12c9e2f111 [LLD] [COFF] Fix alignment of thunks for ARM/ARM64
The alignment of ARM64 range extension thunks was fixed in
7c81649219, but ARM range extension thunks, and import
and delay import thunks also need aligning (like all code on ARM
platforms).

I'm adding a test for alignment of ARM64 import thunks - not
specifically adding tests for misalignment of all of them though.

Differential Revision: https://reviews.llvm.org/D77796
2020-04-13 23:27:15 +03:00
Sam Clegg 9225ff6278 [lld][WebAssembly] Add test for --export of empty string
The actual bug was fixed in https://reviews.llvm.org/D74589

Differential Revision: https://reviews.llvm.org/D77084
2020-04-11 18:21:40 -07:00
Fangrui Song c81daab7d3 [ELF][test] Reorganize format-binary.test
Also drop dependency on the output directory name (lit implementation detail).
2020-04-10 17:00:12 -07:00
Fangrui Song 7f36cb1f1a [AArch64InstPrinter] Change printAlignedLabel to print the target address in hexadecimal form
Similar to D76580 (x86) and D76591 (PPC).

```
// llvm-objdump -d output (before)
10000: 08 00 00 94                   bl      #32
10004: 08 00 00 94                   bl      #32

// llvm-objdump -d output (after)
10000: 08 00 00 94                   bl      0x10020
10004: 08 00 00 94                   bl      0x10024

// GNU objdump -d. The lack of 0x is not ideal due to ambiguity.
10000:       94000008        bl      10020 <bar+0x18>
10004:       94000008        bl      10024 <bar+0x1c>
```

The new output makes it easier to find the jump target.

Differential Revision: https://reviews.llvm.org/D77853
2020-04-10 09:21:09 -07:00
Thomas Lively c496d84b4f [lld][WebAssembly] Handle 4gb max memories
Summary:
A previous change (53211a) had updated the argument parsing to handle
large max memories, but 4294967296 would still wrap to zero after the
options were parsed. This change updates the configuration to use a
64-bit integer to store the max memory to avoid that overflow.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77437
2020-04-09 13:06:41 -07:00
Andrew Ng d08105482e [ELF][test] Improve reproduce tests and enable for Windows
This patch changes the reproduce tests so that they no longer extract
the "long" paths of the generated reproduce tar archives. This
extraction prevented them from being run on Windows due to potential
issues relating to the Windows path length limit.

This patch also reduces the use of diff in these tests, as this was
raised as a performance concern in review D77659 and deemed unnecessary.

Differential Revision: https://reviews.llvm.org/D77750
2020-04-09 16:06:55 +01:00
Peter Smith e81e4fcb92 [LLD][ELF][ARM] Fix 32-bit Arm builders, by making test case smaller
Use AT(address) to force the creation of 3 small segments rather than
a single giant one, causing the Arm v8 builder to run out of memory.
2020-04-09 10:33:02 +01:00
Fangrui Song a27a7b98cd [ELF] --warn-backrefs: don't warn if -u/--export-dynamic-symbol
Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D77630
2020-04-08 09:33:22 -07:00
Andrew Ng 3db215089f [ELF][test] Add reproduce test for dependent libraries
Differential Revision: https://reviews.llvm.org/D77659
2020-04-08 13:00:08 +01:00
Peter Smith 28b172e341 [LLD][ELF][ARM] Implement ARM pc-relative relocations for ADR and LDR
The R_ARM_ALU_PC_G0 and R_ARM_LDR_PC_G0 relocations are used by the
ADR and LDR pseudo instructions, and are the basis of the group
relocations that can load an arbitrary constant via a series of add, sub
and ldr instructions.

The relocations need to be obtained via the .reloc directive.

R_ARM_ALU_PC_G0 is much more complicated as the add/sub instruction uses
a modified immediate encoding of an 8-bit immediate rotated right by an
even 4-bit field. This means that the range of representable immediates
is sparse. We extract the encoding and decoding functions for the modified
immediate from llvm/lib/Target/ARM/MCTargetDesc/ARMAddressingModes.h as
this header file is not accessible from LLD. Duplication of code isn't
ideal, but as these are well-defined mathematical functions they are
unlikely to change.

Differential Revision: https://reviews.llvm.org/D75349
2020-04-08 12:43:44 +01:00
Eric Astor a39b14f0b4 [ms] Add new /PDBSTREAM option to lld-link allowing injection of streams into PDB files.
Summary:
/PDBSTREAM:<name>=<file> adds the contents of <file> to stream <name> in the resulting PDB.

This allows native uses with workflows that (for example) add srcsrv streams to PDB files to provide a location for the build's source files.

Results should be equivalent to linking with lld-link, then running Microsoft's pdbstr tool with the command line:
pdbstr.exe -w -p:<PDB LOCATION> -s:<name> -i:<file>
except in cases where the named stream overlaps with a default named stream, such as "/names". In those cases, the added stream will be overridden, making the /pdbstream option a no-op.

Reviewers: thakis, rnk

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D77310
2020-04-07 16:19:38 -04:00
Fangrui Song 03c825c224 [ELF] --warn-backrefs: don't warn for linking sandwich problems
This is an alternative design to D77512.

D45195 added --warn-backrefs to detect

* A. certain input orders which GNU ld either errors ("undefined reference")
  or has different resolution semantics
* B. (byproduct) some latent multiple definition problems (-ldef1 -lref -ldef2) which I
  call "linking sandwich problems". def2 may or may not be the same as def1.

When an archive appears more than once (-ldef -lref -ldef), lld and GNU
ld may have the same resolution but --warn-backrefs may warn. This is
not uncommon. For example, currently lld itself has such a problem:

```
liblldCommon.a liblldCOFF.a ... liblldCommon.a
  _ZN3lld10DWARFCache13getDILineInfoEmm in liblldCOFF.a refers to liblldCommon.a(DWARF.cpp.o)
libLLVMSupport.a also appears twice and has a similar warning
```

glibc has such problems. It is somewhat destined because of its separate
libc/libpthread/... and arbitrary grouping. The situation is getting
improved over time but I have seen:
```
-lc __isnanl references -lm
-lc _IO_funlockfile references -lpthread
```

There are also various issues in interaction with other runtime
libraries such as libgcc_eh and libunwind:
```
-lc __gcc_personality_v0 references -lgcc_eh
-lpthread __gcc_personality_v0 references -lgcc_eh
-lpthread _Unwind_GetCFA references -lunwind
```

These problems are actually benign. We want --warn-backrefs to focus on
its main task A and defer task B (which is also useful) to a more
specific future feature (see gold --detect-odr-violations and
https://bugs.llvm.org/show_bug.cgi?id=43110).

Instead of warning immediately, we store the message and only report it
if no subsequent lazy definition exists.

The use of the static variable `backrefDiags` is similar to `undefs` in
Relocations.cpp

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D77522
2020-04-07 10:25:23 -07:00
Fangrui Song 4e907e93fb [ELF] -M/-Map: fix VMA/LMA/Size columns of symbol assignments when address/size>=2**32
SymbolAssignment::addr stores the location counter. The type should be
uint64_t instead of unsigned. The upper half of the address space is
commonly used by operating system kernels.

Similarly, SymbolAssignment::size should be an uint64_t. A kernel linker
script can move the location counter from 0 to the upper half of the
address space.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D77445
2020-04-07 10:15:15 -07:00
Sriraman Tallam 94317878d8 LLD Support for Basic Block Sections
This is part of the Propeller framework to do post link code layout
optimizations. Please see the RFC here:
https://groups.google.com/forum/#!msg/llvm-dev/ef3mKzAdJ7U/1shV64BYBAAJ and the
detailed RFC doc here:
https://github.com/google/llvm-propeller/blob/plo-dev/Propeller_RFC.pdf

This patch adds lld support for basic block sections and performs relaxations
after the basic blocks have been reordered.

After the linker has reordered the basic block sections according to the
desired sequence, it runs a relaxation pass to optimize jump instructions.
Currently, the compiler emits the long form of all jump instructions. AMD64 ISA
supports variants of jump instructions with one byte offset or a four byte
offset. The compiler generates jump instructions with R_X86_64 32-bit PC
relative relocations. We would like to use a new relocation type for these jump
instructions as it makes it easy and accurate while relaxing these instructions.

The relaxation pass does two things:

First, it deletes all explicit fall-through direct jump instructions between
adjacent basic blocks. This is done by discarding the tail of the basic block
section.

Second, If there are consecutive jump instructions, it checks if the first
conditional jump can be inverted to convert the second into a fall through and
delete the second.

The jump instructions are relaxed by using jump instruction mods, something
like relocations. These are used to modify the opcode of the jump instruction.
Jump instruction mods contain three values, instruction offset, jump type and
size. While writing this jump instruction out to the final binary, the linker
uses the jump instruction mod to determine the opcode and the size of the
modified jump instruction. These mods are required because the input object
files are memory-mapped without write permissions and directly modifying the
object files requires copying these sections. Copying a large number of basic
block sections significantly bloats memory.

Differential Revision: https://reviews.llvm.org/D68065
2020-04-07 06:55:57 -07:00
Nico Weber 448b777b86 Stop passing site cfg files via --param to llvm-lit.
This has been unnecessary since https://reviews.llvm.org/D37756.

https://reviews.llvm.org/D37838 removed it for llvm.

This removes it from clang, lld, clang-tools-extra (and the GN build).

No intended behavior change.

Differential Revision: https://reviews.llvm.org/D77585
2020-04-07 08:20:40 -04:00
Hans Wennborg 717a460867 Drop the number of thinlto jobs in some tests to avoid crashing on 32-bit windows 2020-04-07 09:52:12 +02:00
Heejin Ahn c09acd5dd1 [WebAssembly] Handle event exports
Summary: This handles exports of events, which was missing.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D77566
2020-04-06 11:28:38 -07:00
Fangrui Song c1c679e2d2 [ELF] Make --version-script/--dynamic-list work for lazy symbols fetched by LTO libcalls
Fixes https://bugs.llvm.org/show_bug.cgi?id=45391

The LTO code generator happens after version script scanning and may
create references which will fetch some lazy symbols.

Currently a version script does not assign VER_NDX_LOCAL to lazy symbols
and such symbols will be made global after they are fetched.

Change findByVersion and findAllByVersion to work on lazy symbols.
For unfetched lazy symbols, we should keep them non-local (D35263).
Check isDefined() in computeBinding() as a compensation.

This patch fixes a companion bug that --dynamic-list does not export
libcall fetched symbols.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D77280
2020-04-06 09:47:06 -07:00
Fangrui Song 26e6880434 [ELF][test] Reorganize warn-backrefs.s 2020-04-05 10:28:32 -07:00
Fangrui Song 9195b01911 [ELF][PPC64] Enable R_PPC64_REL14 trunks
The thunk implementation is available but an assertion disallows it.
Linux kernel has such a use case: in arch/powerpc/kernel/exceptions-64s.S:handle_page_fault,
beq+ ret_from_except_lite may get out of range.

Link: https://github.com/ClangBuiltLinux/linux/issues/951

Differential Revision: https://reviews.llvm.org/D76904
2020-04-04 10:59:17 -07:00
Fangrui Song a13212582a [ELF][test] Test that thunks are processed before finalizeSynthetic(in.symTab)
finalizeSynthetic(in.symTab) calls sortSymTabSymbols() to order local
symbols before non-local symbols.

The newly added tests ensure that thunk symbols are added before
finalizeSynthetic(in.symTab), otherwise .symtab would be out of order.
2020-04-04 10:57:37 -07:00
Fangrui Song 56decd982d [ELF] Allow invalid sh_size%sh_entsize!=0 for non-SHF_MERGE sections
Fixes https://bugs.llvm.org/show_bug.cgi?id=45370
Fixes https://github.com/Clozure/ccl/issues/273

.stab holds a table of 12-byte entries. GNU as before 2.35 incorrectly
sets sh_entsize(.stab) to 20 on 64-bit architectures:
https://sourceware.org/bugzilla/show_bug.cgi?id=25768

We should not emit the confusing error:
"SHF_MERGE section size (...) must be a multiple of sh_entsize (20)

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D77368
2020-04-03 08:48:30 -07:00
Sid Manning c484b3e334 [Hexagon] Fix issue with non-preemptible STT_TLS symbols
A PC-relative relocation referencing a non-preemptible absolute symbol
(due to STT_TLS) is not representable in -pie/-shared mode.

Differential Revision: https://reviews.llvm.org/D77021
2020-04-03 08:55:23 -05:00
Peter Smith da74537e87 [LLD][ELF][ARM] use .reloc directive for Thumb assembler tests
In the near future llvm-mc will resolve the fixups that generate
R_ARM_THUMB_PC8 and R_ARM_THUMB_PC12 at assembly time (see comments in
D72892), and forbid inter-section references. Change the LLD tests for
these relocations to use .inst and .reloc to avoid LLD tests failing when
this happens. The tests generate the same instructions, relocations
and symbols.

I will need to make equivalent changes for D75349 Arm equivalent
relocations, but this is still in review so these don't need changing
before llvm-mc.

Differential Revision: https://reviews.llvm.org/D77200
2020-04-03 11:44:31 +01:00
Fangrui Song 6acd300375 Reland D75382 "[lld] Initial commit for new Mach-O backend"
With a fix for http://lab.llvm.org:8011/builders/clang-cmake-armv8-lld/builds/3636

Also trims some unneeded dependencies.
2020-04-02 12:03:43 -07:00
Oliver Stannard af39151f3c Revert "[lld] Initial commit for new Mach-O backend"
This is causing buildbot failures on 32-bit hosts, for example:
http://lab.llvm.org:8011/builders/clang-cmake-armv8-lld/builds/3636

This reverts commit 03f43b3aca.
2020-04-02 13:23:30 +01:00
Fangrui Song 85adce3d73 [PPCInstPrinter] Change B to print the target address in hexadecimal form
Follow-up of D76591 and D76907
2020-04-01 22:38:24 -07:00
Igor Kudrin b0b1f451ae [LLD][ELF] Follow the common pattern in a message about an undefined vtable symbol.
In most cases, LLD prints its multiline diagnostic messages starting
additional lines with ">>> ". That greatly helps external tools to parse
the output, simplifying combining several lines of the log back into one
message. The patch fixes the only message I found that does not follow
the common pattern.

Differential Revision: https://reviews.llvm.org/D77132
2020-04-02 11:39:03 +07:00
Kazuaki Ishizaki 7c5fcb3591 [lld] NFC: fix trivial typos in comments
Differential Revision: https://reviews.llvm.org/D72339
2020-04-02 01:21:36 +09:00
Fangrui Song bb4a36ea28 [ELF] Propagate LMA offset to sections with neither AT() nor AT>
Fixes https://bugs.llvm.org/show_bug.cgi?id=45313
Also fixes linkerscript/{at4.s,overlay.test} LMA address issues exposed by
011b785505.
Related: D74297

This patch improves emulation of GNU ld's heuristics on the difference
between the LMA and the VMA:
https://sourceware.org/binutils/docs/ld/Output-Section-LMA.html#Output-Section-LMA

New test linkerscript/lma-offset.s (based on at4.s) demonstrates some behaviors.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D76995
2020-04-01 08:19:06 -07:00
Fangrui Song f2036a15d3 [ELF] Print symbols with non-default versions for better "undefined symbol" diagnostics
When reporting an "undefined symbol" diagnostic:

* We don't print @ for the reference.
* We don't print @ or @@ for the definition. https://bugs.llvm.org/show_bug.cgi?id=45318

This can lead to confusing diagnostics:

```
// foo may be foo@v2
ld.lld: error: undefined symbol: foo
>>> referenced by t1.o:(.text+0x1)
// foo may be foo@v1 or foo@@v1
>>> did you mean: foo
>>> defined in: t.so
```

There are 2 ways a symbol in symtab may get truncated:

* A @@ definition may be truncated *early* by SymbolTable::insert().
  The name ends with a '\0'.
* A @ definition/reference may be truncated *later* by Symbol::parseSymbolVersion().
  The name ends with a '@'.

This patch detects the second case and improves the diagnostics. The first case is
not improved but the second case is sufficient to make diagnostics not confusing.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D76999
2020-04-01 08:04:36 -07:00
scentini 51d594d33c Replace output file name in test with %t 2020-04-01 16:40:02 +02:00
Sam Clegg b5767010a8 [lld][WebAssembly] Early error if output file cannot be created.
This matches the behaviour of the ELF driver.

Also move the `createFiles` to be `checkConfig` and report `no input
files` there.   Again this is mostly to match the structure of the ELF
linker better.

Differential Revision: https://reviews.llvm.org/D76960
2020-03-31 21:42:38 -07:00
Fangrui Song 0f44a09757 [ELF][test] Add 'REQUIRES: x86' to threads.s 2020-03-31 15:25:01 -07:00
Fangrui Song 4af7560b37 [PPCInstPrinter] Print conditional branches as `bt 2, $target` instead of `bt 2, .+$imm`
Follow-up of D76591.

Reviewed By: #powerpc, sfertile

Differential Revision: https://reviews.llvm.org/D76907
2020-03-31 15:05:38 -07:00
Jez Ng 03f43b3aca [lld] Initial commit for new Mach-O backend
Summary:
This is the first commit for the new Mach-O backend, designed to roughly
follow the architecture of the existing ELF and COFF backends, and
building off work that @ruiu and @pcc did in a branch a while back. Note
that this is a very stripped-down commit with the bare minimum of
functionality for ease of review. We'll be following up with more diffs
soon.

Currently, we're able to generate a simple "Hello World!" executable
that runs on OS X Catalina (and possibly on earlier OS X versions; I
haven't tested them). (This executable can be obtained by compiling
`test/MachO/relocations.s`.) We're mocking out a few load commands to
achieve this -- for example, we can't load dynamic libraries, but
Catalina requires binaries to be linked against `dyld`, so we hardcode
the emission of a `LC_LOAD_DYLIB` command. Other mocked out load
commands include LC_SYMTAB and LC_DYSYMTAB.

Differential Revision: https://reviews.llvm.org/D75382
2020-03-31 11:58:47 -07:00
Fangrui Song eb4663d8c6 [lld][COFF][ELF][WebAssembly] Replace --[no-]threads /threads[:no] with --threads={1,2,...} /threads:{1,2,...}
--no-threads is a name copied from gold.
gold has --no-thread, --thread-count and several other --thread-count-*.

There are needs to customize the number of threads (running several lld
processes concurrently or customizing the number of LTO threads).
Having a single --threads=N is a straightforward replacement of gold's
--no-threads + --thread-count.

--no-threads is used rarely. So just delete --no-threads instead of
keeping it for compatibility for a while.

If --threads= is specified (ELF,wasm; COFF /threads: is similar),
--thinlto-jobs= defaults to --threads=,
otherwise all available hardware threads are used.

There is currently no way to override a --threads={1,2,...}. It is still
a debate whether we should use --threads=all.

Reviewed By: rnk, aganea

Differential Revision: https://reviews.llvm.org/D76885
2020-03-31 08:46:12 -07:00
Peter Smith 2539b4ae47 [LLD][ELF] Allow empty (.init|.preinit|.fini)_array to be RELRO
The default GNU linker script uses the following idiom for the array
sections. I'll use .init_array here, but this also applies to
.preinit_array and .fini_array sections.

  .init_array    :
  {
    PROVIDE_HIDDEN (__init_array_start = .);
    KEEP (*(.init_array))
    PROVIDE_HIDDEN (__init_array_end = .);
  }

The C-library will take references to the _start and _end symbols to
process the array. This will make LLD keep the OutputSection even if there
are no .init_array sections. As the current check for RELRO uses the
section type for .init_array the above example with no .init_array
InputSections fails the checks as there are no .init_array sections to give
the OutputSection a type of SHT_INIT_ARRAY. This often leads to a
non-contiguous RELRO error message.

The simple fix is to a textual section match as well as a section type
match.

Differential Revision: https://reviews.llvm.org/D76915
2020-03-31 12:53:12 +01:00
James Henderson b4d0384a6c [ELF][test] Add test for --gc-sections + many sections
This test covers the case where --gc-sections is used when there are
many sections. In particular, it ensures that there is no adverse
interaction with common and absolute symbols.

Reviewed by: MaskRay, grimar

Differential Revision: https://reviews.llvm.org/D76003
2020-03-31 12:41:19 +01:00
James Henderson 0aa9c5429b [lld][ELF][test] Improve deplib.s
The test had a few style issues, and I noticed a hole in the coverage
(namely that the search order wasn't tested). Adding cases for the hole
in turn meant other cases weren't important.

The .so test case isn't important, since the code is shared code, so
I've removed it. Additionally, I've modified the usage of the "bar"
directive to show that an unneeded library must still be present, or the
link will fail, even though it isn't linked in.

Reviewed by: MaskRay, grimar

Differential Revision: https://reviews.llvm.org/D76851
2020-03-31 12:41:18 +01:00
Fangrui Song 51475e4023 [ELF][test] Add linkerscript/linkorder-linked-to.s
Delete relocatable-linkorder.s which is covered.
2020-03-30 15:17:29 -07:00
Nico Weber 20eb719f99 lld: Reduce number of references to undefined printed from 10 to 3.
As of a while ago, lld groups all undefined references to a single
symbol in a single diagnostic. Back then, I made it so that we
print up to 10 references to each undefined symbol.

Having used this for a while, I never wished there were more
references, but I sometimes found that this can print a lot of
output. lld prints up to 10 diagnostics by default, and if
each has 10 references (which I've seen in practice), and each
undefined symbol produces 2 (possibly very long) lines of output,
that's over 200 lines of error output.

Let's try it with just 3 references for a while and see how
that feels in practice.

Differential Revision: https://reviews.llvm.org/D77017
2020-03-30 14:31:32 -04:00
Fangrui Song 673e81eee4 [ELF] Allow SHF_LINK_ORDER and non-SHF_LINK_ORDER to be mixed
Currently, `error: incompatible section flags for .rodata` is reported
when we mix SHF_LINK_ORDER and non-SHF_LINK_ORDER sections in an output section.

This is overconstrained. This patch allows mixed flags with the
requirement that SHF_LINK_ORDER sections must be contiguous. Mixing
flags is used by Linux aarch64 (https://github.com/ClangBuiltLinux/linux/issues/953)

  .init.data : { ... KEEP(*(__patchable_function_entries)) ... }

When the integrated assembler is enabled, clang's -fpatchable-function-entry=N[,M]
implementation sets the SHF_LINK_ORDER flag (D72215) to fix a number of
garbage collection issues.

Strictly speaking, the ELF specification does not require contiguous
SHF_LINK_ORDER sections but for many current uses of SHF_LINK_ORDER like
.ARM.exidx/__patchable_function_entries there has been a requirement for
the sections to be contiguous on top of the requirements of the ELF
specification.

This patch also imposes one restriction: SHF_LINK_ORDER sections cannot
be separated by a symbol assignment or a BYTE command. Not allowing BYTE
is a natural extension that a non-SHF_LINK_ORDER cannot be a separator.
Symbol assignments can delimiter the contents of SHF_LINK_ORDER
sections.  Allowing SHF_LINK_ORDER sections across symbol assignments
(especially __start_/__stop_) can make things hard to explain. The
restriction should not be a problem for practical use cases.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D77007
2020-03-30 10:03:55 -07:00
Fangrui Song 2d19270efc [ELF][test] Improve linkerscript/linkorder.s 2020-03-30 09:34:29 -07:00
Fangrui Song 00c76f3496 [ELF][test] Improve arm-exidx-output.s to test different output text sections
Delete arm-exidx-link.s which is now covered by arm-exidx-output.s

Differential Revision: https://reviews.llvm.org/D76409
2020-03-29 13:12:20 -07:00
Matt Schulte fdc41aa22c [lld][ELF] Mark empty NOLOAD output sections SHT_NOBITS instead of SHT_PROGBITS
This fixes PR# 45336.
Output sections described in a linker script as NOLOAD with no input sections would be marked as SHT_PROGBITS.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D76981
2020-03-28 10:07:58 -07:00
Thomas Preud'homme 6eb345af5a [lld test] Tighten ELF/pre_init_fini_array.s test
Summary:
Make use of FileCheck numeric variable to tighten symbol address checks
in ELF/pre_init_fini_array.s test.

Reviewers: khemant, compnerd, echristo, jhenderson, espindola, MaskRay

Reviewed By: MaskRay

Subscribers: emaste, MaskRay, llvm-commits, rupprecht

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76852
2020-03-27 21:44:12 +00:00
Reid Kleckner c579a5b1d9 [COFF] Don't treat DWARF sections as GC roots
DWARF sections are typically live and not COMDAT, so they would be
treated as GC roots. Enabling DWARF would essentially keep all code with
debug info alive, preventing any section GC.

Fixes PR45273

Reviewed By: mstorsjo, MaskRay

Differential Revision: https://reviews.llvm.org/D76935
2020-03-27 12:37:43 -07:00
Fangrui Song 34bdddf9a1 [ELF][test] Split basic.s 2020-03-27 11:56:02 -07:00
Alexandre Ganea 09158252f7 [ThinLTO] Allow usage of all hardware threads in the system
Before this patch, it wasn't possible to extend the ThinLTO threads to all SMT/CMT threads in the system. Only one thread per core was allowed, instructed by usage of llvm::heavyweight_hardware_concurrency() in the ThinLTO code. Any number passed to the LLD flag /opt:lldltojobs=..., or any other ThinLTO-specific flag, was previously interpreted in the context of llvm::heavyweight_hardware_concurrency(), which means SMT disabled.

One can now say in LLD:
/opt:lldltojobs=0 -- Use one std::thread / hardware core in the system (no SMT). Default value if flag not specified.
/opt:lldltojobs=N -- Limit usage to N threads, regardless of usage of heavyweight_hardware_concurrency().
/opt:lldltojobs=all -- Use all hardware threads in the system. Equivalent to /opt:lldltojobs=$(nproc) on Linux and /opt:lldltojobs=%NUMBER_OF_PROCESSORS% on Windows. When an affinity mask is set for the process, threads will be created only for the cores selected by the mask.

When N > number-of-hardware-threads-in-the-system, the threads in the thread pool will be dispatched equally on all CPU sockets (tested only on Windows).
When N <= number-of-hardware-threads-on-a-CPU-socket, the threads will remain on the CPU socket where the process started (only on Windows).

Differential Revision: https://reviews.llvm.org/D75153
2020-03-27 10:20:58 -04:00
Dan Gohman 66bfbedbdf [WebAssembly] Support wasm exports with zero-length names.
Zero-length strings are valid export names in WebAssembly, so allow
users to specify them.

Differential Revision: https://reviews.llvm.org/D71793
2020-03-26 16:20:43 -07:00
Paolo Severini aff75e1a1f [lld][Wasm] Wasm-ld emits invalid .debug_ranges entries for non-live symbols
When the debug info contains a relocation against a dead symbol, wasm-ld
may emit spurious range-list terminator entries (entries with Start==0
and End==0). This change fixes this by emitting the WasmRelocation
Addend as End value for a non-live symbol.

Reviewed by: sbc100, dblaikie

Differential Revision: https://reviews.llvm.org/D74781
2020-03-26 14:26:31 -07:00
Heejin Ahn f033f201a7 [WebAssembly] Add test for event section order change
Summary:
This adds a test for D76752. Now the global section comes after the
event section, and this change makes sure it is satisfied.

Reviewers: sbc100, tlively

Reviewed By: tlively

Subscribers: dschuff, jgravelle-google, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D76823
2020-03-26 11:05:24 -07:00
Fangrui Song 3eef47407b [PPCInstPrinter] Change printBranchOperand(calltarget) to print the target address in hexadecimal form
```
// llvm-objdump -d output (before)
0: bl .-4
4: bl .+0
8: bl .+4

// llvm-objdump -d output (after) ; GNU objdump -d
0: bl 0xfffffffc / bl 0xfffffffffffffffc
4: bl 0x4
8: bl 0xc
```

Many Operand's are not annotated as OPERAND_PCREL.
They are not affected (e.g. `b .+67108860`). I plan to fix them in future patches.

Modified test/tools/llvm-objdump/ELF/PowerPC/branch-offset.s to test
address space wraparound for powerpc32 and powerpc64.

Reviewed By: sfertile, jhenderson

Differential Revision: https://reviews.llvm.org/D76591
2020-03-26 08:32:29 -07:00
James Henderson 3ff3c6986b [lld][ELF] Fix error message
The error previously talked about a "section header" but was actually
referring to a program header.

Reviewed by: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D76846
2020-03-26 15:30:24 +00:00
Fangrui Song 87de9a0786 [X86InstPrinter] Change printPCRelImm to print the target address in hexadecimal form
```
// llvm-objdump -d output (before)
400000: e8 0b 00 00 00   callq 11
400005: e8 0b 00 00 00   callq 11

// llvm-objdump -d output (after)
400000: e8 0b 00 00 00  callq 0x400010
400005: e8 0b 00 00 00  callq 0x400015

// GNU objdump -d. The lack of 0x is not ideal because the result cannot be re-assembled
400000: e8 0b 00 00 00  callq 400010
400005: e8 0b 00 00 00  callq 400015
```

In llvm-objdump, we pass the address of the next MCInst. Ideally we
should just thread the address of the current address, unfortunately we
cannot call X86MCCodeEmitter::encodeInstruction (X86MCCodeEmitter
requires MCInstrInfo and MCContext) to get the length of the MCInst.

MCInstPrinter::printInst has other callers (e.g llvm-mc -filetype=asm, llvm-mca) which set Address to 0.
They leave MCInstPrinter::PrintBranchImmAsAddress as false and this change is a no-op for them.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D76580
2020-03-26 08:28:59 -07:00
Fangrui Song 72ffc14e13 [ELF][test] Delete unneeded binding directive (.weak or .globl)
Future MC may error for a changed symbol binding.
This may be a preferred resolution to https://bugs.llvm.org/show_bug.cgi?id=38921

See https://sourceware.org/pipermail/binutils/2020-March/110399.html
2020-03-25 23:34:46 -07:00
Sylvain Audi b91905a263 [lld-link] Support /map option, matching link.exe 's /map output format
Added support for /map and /map:[filepath].
The output was derived from Microsoft's Link.exe output when using that same option.
Note that /MAPINFO support was not added.

The previous implementation of MapFile.cpp/.h was meant for /lldmap, and was renamed to LLDMapFile.cpp/.h
MapFile.cpp/.h is now for /MAP
However, a small fix was added to lldmap, replacing a std::sort with std::stable_sort to enforce reproducibility.

Differential Revision: https://reviews.llvm.org/D70557
2020-03-24 09:48:00 -04:00
Fangrui Song 9e33c09647 [ELF] Keep orphan section names (.rodata.foo .text.foo) unchanged if !hasSectionsCommand
This behavior matches GNU ld and seems reasonable.

```
// If a SECTIONS command is not specified
.text.* -> .text
.rodata.* -> .rodata
.init_array.* -> .init_array
```

A proposed Linux feature CONFIG_FG_KASLR may depend on the GNU ld behavior.

Reword a comment about -z keep-text-section-prefix and a comment about
CommonSection (deleted by rL286234).

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D75225
2020-03-23 10:30:06 -07:00
Fangrui Song 0a076f2660 [ELF][test] Clean text-section-prefix.s 2020-03-23 09:38:59 -07:00
Fangrui Song 011b785505 [ELF] Create readonly PT_LOAD in the presence of a SECTIONS command
This essentially drops the change by r288021 (discussed with Georgii Rymar
and Peter Smith and noted down in the release note of lld 10).

GNU ld>=2.31 enables -z separate-code by default for Linux x86. By
default (in the absence of a PHDRS command) a readonly PT_LOAD is
created, which is different from its traditional behavior.

Not emulating GNU ld's traditional behavior is good for us because it
improves code consistency (we create a readonly PT_LOAD in the absence
of a SECTIONS command).

Users can add --no-rosegment to restore the previous behavior (combined
readonly and read-executable sections in a single RX PT_LOAD).
2020-03-19 19:11:11 -07:00
Fangrui Song 09ac859c13 [ELF][test] Make tests less address sensitive and delete redundant tests 2020-03-19 18:04:47 -07:00
Georgii Rymar bb7d2b1780 [LLD][ELF] - Disambiguate "=fillexp" with a primary expression to allow =0x90 /DISCARD/
Fixes https://bugs.llvm.org/show_bug.cgi?id=44903

It is about the following case:

```
SECTIONS {
  .foo : { *(.foo) } =0x90909090
  /DISCARD/ : { *(.bar) }
}
```

Here while parsing the fill expression we treated the
"/" of "/DISCARD/" as operator.

With this change, suggested by Fangrui Song, we do
not allow expressions with operators (e.g. "0x1100 + 0x22")
that are not wrapped into round brackets. It should not
be an issue for users, but helps to resolve parsing ambiguity.

Differential revision: https://reviews.llvm.org/D74687
2020-03-19 12:49:25 +03:00
Fangrui Song 2bd1ca28c1 [ELF][test] Delete unneeded --no-threads 2020-03-17 12:58:24 -07:00
Fangrui Song 536ba6373f [Object] Change ELFObjectFile<ELFT>::getFileFormatName() to use BFD names
Follow-up for D74433

What the function returns are almost standard BFD names, except that "ELF" is
in uppercase instead of lowercase.

This patch changes "ELF" to "elf" and changes ARM/AArch64 to use their BFD names.
MIPS and PPC64 have endianness differences as well, but this patch does not intend to address them.

Advantages:

* llvm-objdump: the "file format " line matches GNU objdump on ARM/AArch64 objects
* "file format " line can be extracted and fed into llvm-objcopy -O literally.
  (https://github.com/ClangBuiltLinux/linux/issues/779 has such a use case)

Affected tools: llvm-readobj, llvm-objdump, llvm-dwarfdump, MCJIT (internal implementation detail, not exposed)

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D76046
2020-03-16 07:42:04 -07:00
Rui Ueyama a2923b2a1e Implement CET Shadow Stack (Intel Controlflow Enforcement Technology) support on Windows
Patch by Petr Penzin.

Windows support for CET is limited to shadow stack, which is enabled
by setting a PE bit in the linker.

Docs:

MSVC linker flag:
https://docs.microsoft.com/en-us/cpp/build/reference/cetcompat?view=vs-2019

IMAGE_DLLCHARACTERISTICS_EX_CET_COMPAT PE bit:
https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#extended-dll-characteristics

Differential Revision: https://reviews.llvm.org/D70606
2020-03-16 17:51:32 +09:00
Fangrui Song f0374e7db2 [test] lld/test/: change llvm-objdump single-dash long options to double-dash options 2020-03-15 17:48:36 -07:00
Shoaib Meenai e890453d6d [ELF] Add test for freebsd bfdname
Make sure the output error contains the full bfdname. Suggested by
George Rimar in https://reviews.llvm.org/D76109#inline-693972.
2020-03-13 14:16:53 -07:00
Alexandre Ganea a7325298e1 [CodeView] Align type records on 4-bytes when emitting PDBs
When emitting PDBs, the TypeStreamMerger class is used to merge .debug$T records from the input .OBJ files into the output .PDB stream.
Records in .OBJs are not required to be aligned on 4-bytes, and "The Netwide Assembler 2.14" generates non-aligned records.

When compiling with -DLLVM_ENABLE_ASSERTIONS=ON, an assert was triggered in MergingTypeTableBuilder when non-ghash merging was used.
With ghash merging there was no assert.
As a result, LLD could potentially generate a non-aligned TPI stream.

We now align records on 4-bytes when record indices are remapped, in TypeStreamMerger::remapIndices().

Differential Revision: https://reviews.llvm.org/D75081
2020-03-13 12:22:19 -04:00
Sid Manning 5a5a075c5b [LLD][ELF][Hexagon] Support GDPLT transforms
Hexagon ABI specifies that call x@gdplt is transformed to call __tls_get_addr.

Example:
     call x@gdplt
is changed to
     call __tls_get_addr

When x is an external tls variable.

Differential Revision: https://reviews.llvm.org/D74443
2020-03-13 11:02:11 -05:00
Shoaib Meenai 2822852ffc [ELF] Correct error message when OUTPUT_FORMAT is used
Any OUTPUT_FORMAT in a linker script overrides the emulation passed on
the command line, so record the passed bfdname and use that in the error
message about incompatible input files.

This prevents confusing error messages. For example, if you explicitly
pass `-m elf_x86_64` to LLD but accidentally include a linker script
which sets `OUTPUT_FORMAT(elf32-i386)`, LLD would previously complain
about your input files being compatible with elf_x86_64, which isn't the
actual issue, and is confusing because the input files are in fact
x86-64 ELF files.

Interestingly enough, this also prevents a segfault! When we don't pass
`-m` and we have an object file which is incompatible with the
`OUTPUT_FORMAT` set by a linker script, the object file is checked for
compatibility before it's added to the objectFiles vector.
config->emulation, objectFiles, and sharedFiles will all be empty, so
we'll attempt to access bitcodeFiles[0], but bitcodeFiles is also empty,
so we'll segfault. This commit prevents the segfault by adding
OUTPUT_FORMAT as a possible source of machine configuration, and it also
adds an llvm_unreachable to diagnose similar issues in the future.

Differential Revision: https://reviews.llvm.org/D76109
2020-03-12 22:54:53 -07:00
Fangrui Song eb4b5a36a6 [ELF] Move --print-map(-M)/--cref before checkSections() and openFile()
-M output can be useful when diagnosing an "error: output file too large" problem (emitted in openFile()).

I just ran into such a situation where I had to debug an erronerous
Linux kernel linker script. It tried to create a file larger than
INT64_MAX bytes.

This patch could have helped https://bugs.llvm.org/show_bug.cgi?id=44715 as well.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D75966
2020-03-12 08:00:18 -07:00
Fangrui Song fbf41b5267 [ELF] Simplify sh_addr computation and warn if sh_addr is not a multiple of sh_addralign
See `docs/ELF/linker_script.rst` for the new computation for sh_addr and sh_addralign.
`ALIGN(section_align)` now means: "increase alignment to section_align"
(like yet another input section requirement).

The "start of section .foo changes from 0x11 to 0x20" warning no longer
makes sense. Change it to warn if sh_addr%sh_addralign!=0.

To decrease the alignment from the default max_input_align,
use `.output ALIGN(8) : {}` instead of `.output : ALIGN(8) {}`
See linkerscript/section-address-align.test as an example.

When both an output section address and ALIGN are set (can be seen as an
"undefined behavior" https://sourceware.org/ml/binutils/2020-03/msg00115.html),
lld may align more than GNU ld, but it makes a linker script working
with GNU ld hard to break with lld.

This patch can be considered as restoring part of the behavior before D74736.

Differential Revision: https://reviews.llvm.org/D75724
2020-03-11 09:35:42 -07:00
David Bozier 6e2804ce6b [LLD] Add support for --unique option
Summary:
Places orphan sections into a unique output section. This prevents the merging of orphan sections of the same name.
Matches behaviour of GNU ld --unique. --unique=pattern is not implemented.

Motivated user case shown in the test has 2 local symbols as they would appear if C++ source has been compiled with -ffunction-sections. The merging of these sections in the case of a partial link (-r) may limit the effectiveness of -gc-sections of a subsequent link.

Reviewers: espindola, jhenderson, bd1976llvm, edd, andrewng, JonChesterfield, MaskRay, grimar, ruiu, psmith

Reviewed By: MaskRay, grimar

Subscribers: emaste, arichardson, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D75536
2020-03-10 12:20:21 +00:00
Fangrui Song 92b5b980d2 [ELF] Postpone evaluation of ORIGIN/LENGTH in a MEMORY command
```
createFiles(args)
 readDefsym
 readerLinkerScript(*mb)
  ...
   readMemory
    readMemoryAssignment("ORIGIN", "org", "o") // eagerly evaluated
target = getTarget();
link(args)
 writeResult<ELFT>()
  ...
   finalizeSections()
    script->processSymbolAssignments()
     addSymbol(cmd) // with this patch, evaluated here
```

readMemoryAssignment eagerly evaluates ORIGIN/LENGTH and returns an uint64_t.
This patch postpones the evaluation to make

* --defsym and symbol assignments
* `CONSTANT(COMMONPAGESIZE)` (requires a non-null `lld:🧝:target`)

work. If the expression somehow requires interaction with memory
regions, the circular dependency may cause the expression to evaluate to
a strange value. See the new test added to memory-err.s

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D75763
2020-03-09 08:31:41 -07:00
Fangrui Song 7b8341b257 [ELF][test] Improve MEMORY tests 2020-03-06 14:01:20 -08:00
Benjamin Barenblat f0f4d41631 [lld][test] Make tests pass when the test directory matches `bar`
Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D72360
2020-03-06 13:27:02 -08:00
Alexey Lapshin dcf6494abe LLD already has a mechanism for caching creation of DWARCContext:
llvm::call_once(initDwarfLine, [this]() { initializeDwarf(); });

Though it is not used in all places.

I need that patch for implementing "Remove obsolete debug info" feature
(D74169). But this caching mechanism is useful by itself, and I think it
would be good to use it without connection to "Remove obsolete debug info"
feature. So this patch changes inplace creation of DWARFContext with
its cached version.

Depends on D74308

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D74773
2020-03-06 21:17:07 +03:00
Georgii Rymar e4ceb8f421 [lib/ObjectYAML] - Make `ELFYAML::Relocation::Offset` optional.
Currently `yaml2obj` require `Offset` field in a relocation description.
There are many cases when `Offset` is insignificant in a context of a test case.

Making `Offset` optional allows to simplify our test cases.
This is what this patch does.

Also, with this patch `obj2yaml` does not dump a zero offset of a relocation.

Differential revision: https://reviews.llvm.org/D75608
2020-03-06 13:59:58 +03:00
Fangrui Song 71e2ca6e32 [llvm-objdump] -d: print `00000000 <foo>:` instead of `00000000 foo:`
The new behavior matches GNU objdump. A pair of angle brackets makes tests slightly easier.

`.foo:` is not unique and thus cannot be used in a `CHECK-LABEL:` directive.
Without `-LABEL`, the CHECK line can match the `Disassembly of section`
line and causes the next `CHECK-NEXT:` to fail.

```
Disassembly of section .foo:

0000000000001634 .foo:
```

Bdragon: <> has metalinguistic connotation. it just "feels right"

Reviewed By: rupprecht

Differential Revision: https://reviews.llvm.org/D75713
2020-03-05 18:05:28 -08:00
Fangrui Song 791efb148f [ARM] Rewrite ARMAttributeParser
* Delete boilerplate
* Change functions to return `Error`
* Test parsing errors
* Update callers of ARMAttributeParser::parse() to check the `Error` return value.

Since this patch touches nearly everything in the file, I apply
http://llvm.org/docs/Proposals/VariableNames.html and change variable
names to lower case.

Reviewed By: compnerd

Differential Revision: https://reviews.llvm.org/D75015
2020-03-05 10:57:27 -08:00
Fangrui Song f9a0056016 [llvm-objdump] --syms: make flags closer to GNU objdump
This fixes several issues. The behavior changes are:

A SHN_COMMON symbol does not have the 'g' flag.
An undefined symbol does not have 'g' or 'l' flag.
A STB_GLOBAL SymbolRef::ST_Unknown symbol has the 'g' flag.
A STB_LOCAL SymbolRef::ST_Unknown symbol has the 'l' flag.

Reviewed By: rupprecht

Differential Revision: https://reviews.llvm.org/D75659
2020-03-05 09:59:53 -08:00
Jordan Rupprecht c140810ea1 [llvm-readobj] Include section name of notes.
This changes the output of `llvm-readelf -n` from:

```
Displaying notes found at file offset 0x<...> with length 0x<...>:
```

to:

```
Displaying notes found in: .note.foo
```

And similarly, adds a `Name:` field to the `llvm-readobj -n` output for notes.

This change not only increases GNU compatibility, it also makes it much easier to read notes. Note that we still fall back to printing the file offset/length in cases where we don't have a section name, such as when printing notes in program headers or printing notes in a partially stripped file (GNU readelf does the same).

Fixes llvm.org/PR41339.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D75647
2020-03-05 09:53:14 -08:00
Fangrui Song 9e1319df7e [llvm-readelf] Make --all output order closer to GNU readelf
https://bugs.llvm.org/show_bug.cgi?id=43403

The new order makes it easy to compare the two tools' --all.

Reviewed By: grimar, rupprecht

Differential Revision: https://reviews.llvm.org/D75592
2020-03-04 12:22:12 -08:00
Fangrui Song c72d60d42f [llvm-objdump] --syms: print st_size as "%016" PRIx64 instead of "%08" PRIx64 for 64-bit objects
This is GNU objdump's behavior and it is reasonable to match.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D75588
2020-03-04 12:09:27 -08:00
Sam Clegg 928e9e1723 [lld][WebAssembly] Add support for --rsp-quoting
This also changes to default style to match the host.

Reviewed By: ruiu

Differential Revision: https://reviews.llvm.org/D75577
2020-03-04 11:41:33 -08:00
Fangrui Song 1044ee827a [ELF][test] Improve llvm-objdump -t tests
In many cases, llvm-nm or llvm-readelf is more suitable.
2020-03-04 10:48:36 -08:00
Stephan Herhut 573c9d666c [lld] Avoid creating files outside of work directory.
Summary:
A test is passing `-o -` to lld in the hope of writing the output to
standard out but that is not the case. Instead it creates a file named
`-.lto.o`. This fixes it by creating a temporary file in the work
directory.

Differential Revision: https://reviews.llvm.org/D75605
2020-03-04 14:46:12 +01:00
evgeny 670a40360e Attempt to fix buildbot after 497c110e 2020-03-04 13:05:02 +03:00
evgeny 497c110e87 [lld][ELF][COFF] Fix archived bitcode files naming
Differential revision: https://reviews.llvm.org/D75422
2020-03-04 12:46:31 +03:00
Hans Wennborg 916be8fd6a Revert abb00753 "build: reduce CMake handling for zlib" (PR44780)
and follow-ups:
a2ca1c2d "build: disable zlib by default on Windows"
2181bf40 "[CMake] Link against ZLIB::ZLIB"
1079c68a "Attempt to fix ZLIB CMake logic on Windows"

This changed the output of llvm-config --system-libs, and more
importantly it broke stand-alone builds. Instead of piling on more fix
attempts, let's revert this to reduce the risk of more breakages.
2020-03-03 11:03:09 +01:00
Fangrui Song 08ff4dc9ad [LTO] onfig::addSaveTemps: clear ResolutionFile upon an error
Otherwise ld.lld -save-temps will crash when writing to ResolutionFile.

llvm-lto2 -save-temps does not crash because it exits immediately.

Reviewed By: evgeny777

Differential Revision: https://reviews.llvm.org/D75426
2020-03-02 17:49:04 -08:00
Pavel Labath 12048a9182 [lld] Fix test failure from d978656fd0
Tweak the test to account for the slightly different wording of the
error message.
2020-03-02 11:28:48 +01:00
Fangrui Song 315f8a55f5 [ELF][PPC32] Don't report "relocation refers to a discarded section" for .got2
Similar to D63182 [ELF][PPC64] Don't report "relocation refers to a discarded section" for .toc

Reviewed By: Bdragon28

Differential Revision: https://reviews.llvm.org/D75419
2020-03-01 19:54:40 -08:00
Fangrui Song 00925aadb3 [ELF][PPC32] Fix canonical PLTs when the order does not match the PLT order
Reviewed By: Bdragon28

Differential Revision: https://reviews.llvm.org/D75394
2020-02-28 22:23:14 -08:00
Alexey Lapshin 0a2d415bd0 [LLD] Report errors occurred while parsing debug info as warnings.
Summary:
Extracted from D74773. Currently, errors happened while parsing
debug info are reported as errors. DebugInfoDWARF library treats such
errors as "Recoverable errors". This patch makes debug info errors
to be reported as warnings, to support DebugInfoDWARF approach.

Reviewers: ruiu, grimar, MaskRay, jhenderson, espindola

Reviewed By: MaskRay, jhenderson

Subscribers: emaste, aprantl, arichardson, arphaman, llvm-commits

Tags: #llvm, #debug-info, #lld

Differential Revision: https://reviews.llvm.org/D75234
2020-02-29 00:03:18 +03:00
Sam Clegg a57f1a5435 [lld][WebAssembly] Handle mixed strong and weak undefined symbols
When there are both strong and weak references to an undefined
symbol ensure that the strong reference prevails in the output symbol
generating the correct error.

Test case copied from lld/test/ELF/weak-and-strong-undef.s

Differential Revision: https://reviews.llvm.org/D75322
2020-02-28 10:16:10 -08:00
Peter Smith 1b025665c9 [ELF][LLD][ARM] Add missing REQUIRES: arm to tests
Fix buildbots that don't build ARM backend.
2020-02-28 11:46:01 +00:00
Peter Smith 6b035b607f [LLD][ELF][ARM] Implement Thumb pc-relative relocations for adr and ldr
MC will now output the R_ARM_THM_PC8, R_ARM_THM_PC12 and
R_ARM_THM_PREL_11_0 relocations. These are short-ranged relocations that
are used to implement the adr rd, literal and ldr rd, literal pseudo
instructions.

The instructions use a new RelExpr called R_ARM_PCA in order to calculate
the required S + A - Pa expression, where Pa is AlignDown(P, 4) as the
instructions add their immediate to AlignDown(PC, 4). We also do not want
these relocations to generate or resolve against a PLT entry as the range
of these relocations is so short they would never reach.

The R_ARM_THM_PC8 has a special encoding convention for the relocation
addend, the immediate field is unsigned, yet the addend must be -4 to
account for the Thumb PC bias. The ABI (not the architecture) uses the
convention that the 8-byte immediate of 0xff represents -4.

Differential Revision: https://reviews.llvm.org/D75042
2020-02-28 11:29:29 +00:00
Fangrui Song b305b8a256 [ELF][test] Rename SHF_LINK_ORDER related "metadata" to "linkorder"
Test cleanups.
2020-02-26 17:36:59 -08:00
Fangrui Song 37c7f0d945 [ELF] --orphan-handling=: don't warn/error for input SHT_REL[A] retained by --emit-relocs
They are purposefully skipped by input section descriptions (rL295324).
Similarly, --orphan-handling= should not warn/error for them.
This behavior matches GNU ld.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D75151
2020-02-26 10:32:54 -08:00
Fangrui Song 423194098b [ELF] --orphan-handling=: don't warn/error for unused synthesized sections
This makes --orphan-handling= less noisy.
This change also improves our compatibility with GNU ld.

GNU ld special cases .symtab, .strtab and .shstrtab . We need output section
descriptions for .symtab, .strtab and .shstrtab to suppress:

  <internal>:(.symtab) is being placed in '.symtab'
  <internal>:(.shstrtab) is being placed in '.shstrtab'
  <internal>:(.strtab) is being placed in '.strtab'

With --strip-all, .symtab and .strtab can be omitted (note, --strip-all is not compatible with --emit-relocs).

Reviewed By: nickdesaulniers

Differential Revision: https://reviews.llvm.org/D75149
2020-02-26 08:56:12 -08:00
Fangrui Song 93331a17e8 [ELF] Support archive:file syntax in input section descriptions
Fixes https://bugs.llvm.org/show_bug.cgi?id=44450

https://sourceware.org/binutils/docs/ld/Input-Section-Basics.html#Input-Section-Basics
The following two rules are not implemented.

* `archive:` matches every file in the archive.
* `:file` matches a file not in an archive.

Reviewed By: grimar, ruiu

Differential Revision: https://reviews.llvm.org/D75100
2020-02-25 07:57:43 -08:00
Igor Kudrin bd2df13ee0 [DebugInfo] Fix printing CIE offsets in EH FDEs.
While the value of the CIE pointer field in a DWARF FDE record is
an offset to the corresponding CIE record from the beginning of
the section, for EH FDE records it is relative to the current offset.
Previously, we did not make that distinction when dumped both kinds
of FDE records and just printed the same value for the CIE pointer
field and the CIE offset; that was acceptable for DWARF FDEs but was
wrong for EH FDEs.

This patch fixes the issue by explicitly printing the offset of the
linked CIE object.

Differential Revision: https://reviews.llvm.org/D74613
2020-02-25 17:10:29 +07:00
Rafael Ávila de Espíndola 7b44f0428a Add a llvm::shuffle and use it in lld
With this --shuffle-sections=seed produces the same result in every
host.

Reviewed By: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D74971
2020-02-22 10:05:29 -08:00
Fangrui Song dbd7281aa7 [ELF] Shuffle .init_array/.fini_array with --shuffle-sections=
Useful for detecting static initialization order fiasco.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D74887
2020-02-21 08:16:07 -08:00
Fangrui Song de0dda54d3 [ELF] Warn changed output section address
When the output section address (addrExpr) is specified, GNU ld warns if
sh_addr is different. This patch implements the warning.

Note, LinkerScript::assignAddresses can be called more than once. We
need to record the changed section addresses, and only report the
warnings after the addresses are finalized.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D74741
2020-02-21 08:13:29 -08:00
Fangrui Song 6ed8e20143 [ELF] Ignore the maximum of input section alignments for two cases
Follow-up for D74286.

Notations:

* alignExpr: the computed ALIGN value
* max_input_align: the maximum of input section alignments

This patch changes the following two cases to match GNU ld:

* When ALIGN is present, GNU ld sets output sh_addr to alignExpr, while lld use max(alignExpr, max_input_align)
* When addrExpr is specified but alignExpr is not, GNU ld sets output sh_addr to addrExpr, while lld uses `advance(0, max_input_align)`

Note, sh_addralign is still set to max(alignExpr, max_input_align).

lma-align.test is enhanced a bit to check we don't overalign sh_addr.

fixSectionAlignments() sets addrExpr but not alignExpr for the `!hasSectionsCommand` case.
This patch sets alignExpr as well so that max_input_align will be respected.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D74736
2020-02-21 08:12:00 -08:00
Tamas Petz 14d7b802db [LLD][ELF][ARM] Add test cases for R_ARM_THM_MOV*-type relocs
Summary: This patch adds missing tests cases for R_ARM_THM_MOVW/MOVT* relocations for the sake of completeness.

Reviewers: peter.smith, MaskRay, espindola, psmith

Reviewed By: MaskRay, psmith

Subscribers: psmith, danielkiss, emaste, kristof.beyls, llvm-commits

Tags: #llvm, #lld

Differential Revision: https://reviews.llvm.org/D74827
2020-02-20 10:00:42 +01:00
Fangrui Song d6d640ebae [ELF][test] Fix section sh_type and sh_flags
A future MC change may add a warning/error when a .section directive
specifies incorrect sh_flags/sh_type.
2020-02-19 22:01:42 -08:00
Sam Clegg 06f1a5c9c2 [lld][WebAssembly] Allow symbols with explict import names to be undefined at link time.
Differential Revision: https://reviews.llvm.org/D74110
2020-02-19 18:02:49 -08:00
Fangrui Song 878159038b [ELF][test] Fix --symbol-ordering-file tests of shuffle_sections.s 2020-02-19 17:13:06 -08:00
Fangrui Song 40463301e4 [ELF][test] Fix shuffle_sections.s
C++ standard libraries have different random function implementations.
We cannot expect a particular order.
2020-02-19 17:10:32 -08:00
Rafael Ávila de Espíndola d48d339156 [lld][ELF] Add --shuffle-sections=seed to shuffle input sections
Summary:
This option causes lld to shuffle sections by assigning different
priorities in each run.

The use case for this is to introduce randomization in benchmarks. The
idea is inspired by the paper "Producing Wrong Data Without Doing
Anything Obviously Wrong!"
(https://www.inf.usi.ch/faculty/hauswirth/publications/asplos09.pdf). Unlike
the paper, we shuffle individual sections, not just input files.

Doing this in lld is particularly convenient as the --reproduce option
makes it easy to collect all the necessary bits for relinking the
program being benchmarked. Once that it is done, all that is needed is
to add --shuffle-sections=0 to the response file and relink before each
run of the benchmark.

Differential Revision: https://reviews.llvm.org/D74791
2020-02-19 13:44:12 -08:00
Tamas Petz 6e326882da [LLD][ELF][ARM] Fix support for SBREL type relocations
With this patch lld recognizes ARM SBREL relocations.
R_ARM*_MOVW_BREL relocations are not tested because they are not used.

Patch by Tamas Petz

Differential Revision: https://reviews.llvm.org/D74604
2020-02-19 10:07:46 +00:00
Daniel Kiss b6162622c0 [LLD][ELF][AArch64] Change the semantics of -z pac-plt.
Summary:
Generate PAC protected plt only when "-z pac-plt" is passed to the
linker. GNU toolchain generates when it is explicitly requested[1].
When pac-plt is requested then set the GNU_PROPERTY_AARCH64_FEATURE_1_PAC
note even when not all function compiled with PAC but issue a warning.
Harmonizing the warning style for BTI/PAC/IBT.
Generate BTI protected PLT if case of "-z force-bti".

[1] https://www.sourceware.org/ml/binutils/2019-03/msg00021.html

Reviewers: peter.smith, espindola, MaskRay, grimar

Reviewed By: peter.smith, MaskRay

Subscribers: tatyana-krasnukha, emaste, arichardson, kristof.beyls, MaskRay, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D74537
2020-02-18 09:56:57 +01:00
Yuanfang Chen 4ad7685258 Revert "Revert "Reland "[Support] make report_fatal_error `abort` instead of `exit`"""
This reverts commit 80a34ae311 with fixes.

Previously, since bots turning on EXPENSIVE_CHECKS are essentially turning on
MachineVerifierPass by default on X86 and the fact that
inline-asm-avx-v-constraint-32bit.ll and inline-asm-avx512vl-v-constraint-32bit.ll
are not expected to generate functioning machine code, this would go
down to `report_fatal_error` in MachineVerifierPass. Here passing
`-verify-machineinstrs=0` to make the intent explicit.
2020-02-13 10:16:06 -08:00
Yuanfang Chen 17122ec10a Revert "Revert "Revert "Reland "[Support] make report_fatal_error `abort` instead of `exit`""""
This reverts commit bb51d24330.
2020-02-13 10:08:05 -08:00
Yuanfang Chen bb51d24330 Revert "Revert "Reland "[Support] make report_fatal_error `abort` instead of `exit`"""
This reverts commit 80a34ae311 with fixes.

On bots llvm-clang-x86_64-expensive-checks-ubuntu and
llvm-clang-x86_64-expensive-checks-debian only,
llc returns 0 for these two tests unexpectedly. I tweaked the RUN line a little
bit in the hope that LIT is the culprit since this change is not in the
codepath these tests are testing.
llvm\test\CodeGen\X86\inline-asm-avx-v-constraint-32bit.ll
llvm\test\CodeGen\X86\inline-asm-avx512vl-v-constraint-32bit.ll
2020-02-13 10:02:53 -08:00
Fangrui Song 6c73246179 [ELF] Fix a null pointer dereference when --emit-relocs and --strip-debug are used together
Fixes https://bugs.llvm.org//show_bug.cgi?id=44878

When --strip-debug is specified, .debug* are removed from inputSections
while .rel[a].debug* (incorrectly) remain.

LinkerScript::addOrphanSections() requires the output section of a relocated
InputSectionBase to be created first.

.debug* are not in inputSections ->
output sections .debug* are not created ->
getOutputSectionName(.rel[a].debug*) dereferences a null pointer.

Fix the null pointer dereference by deleting .rel[a].debug* from inputSections as well.

Reviewed By: grimar, nickdesaulniers

Differential Revision: https://reviews.llvm.org/D74510
2020-02-13 08:56:38 -08:00