Commit Graph

13458 Commits

Author SHA1 Message Date
Jez Ng 11e2c253ac [lld-macho] Try to fix invalid-stub test on Windows
Summary: Path separator issues...
2020-08-12 20:58:47 -07:00
Jez Ng e48d1262b8 [lld-macho] Support -rpath
Pretty straightforward; just emits LC_RPATH for dyld to consume.

Note that lld itself does not yet support dylib lookup via @rpath.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D85701
2020-08-12 19:50:28 -07:00
Jez Ng 437e6bd286 [lld-macho] Implement -force_load
It's similar to lld-ELF's `-whole-archive`, but applied to individual
archives instead of to a series of them.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D85550
2020-08-12 19:50:27 -07:00
Jez Ng 180ad756ec [lld-macho] Support larger dylib symbol ordinals in bindings
Do folks care if we don't have a test for this? Creating 16
dylibs to trigger this straightforward code path seems a little tedious

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D85467
2020-08-12 19:50:25 -07:00
Jez Ng c3eb1e2754 [lld-macho] Add error handling for malformed TBD files
Previously, lld would crash while complaining that `Expected<T>
must be checked before access or destruction`.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D85403
2020-08-12 19:50:13 -07:00
Jez Ng 7e6d675499 [lld-macho] Avoid unnecessary shared_ptr in DylibFile ctor
DylibFile doesn't store a pointer to its InterfaceFile
parameter, so there's no need to use a shared_ptr.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D85402
2020-08-12 19:50:12 -07:00
Jez Ng a499898e86 [lld-macho] Generate ObjC symbols from .tbd files
I followed similar logic in TapiFile.cpp.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D85255
2020-08-12 19:50:10 -07:00
Jez Ng 3c9100fb78 [lld-macho] Support dynamic linking of thread-locals
References to symbols in dylibs work very similarly regardless of
whether the symbol is a TLV. The main difference is that we have a
separate `__thread_ptrs` section that acts as the GOT for these
thread-locals.

We can identify thread-locals in dylibs by a flag in their export trie
entries, and we cross-check it with the relocations that refer to them
to ensure that we are not using a GOT relocation to reference a
thread-local (or vice versa).

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D85081
2020-08-12 19:50:09 -07:00
Samuel Kostial 304264e73d [lld][WebAssembly] Emit all return types of multivalue functions
We previously were incorrectly emitting only the first result type.

Differential Revision: https://reviews.llvm.org/D85783
2020-08-12 13:14:15 -07:00
Wouter van Oortmerssen b52fc59e17 [WebAssembly] Fixed memory.init always using 64-bit ptr
(because the is64 flag was tested incorrectly in LLD).
2020-08-12 12:23:36 -07:00
Petr Hosek 31e5f7120b [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-08-11 20:22:11 -07:00
Fangrui Song e973c1375e [ELF] Move the outSecOff addend from relocAlloc/relocNonAlloc/... to InputSectionBase::relocate
For an InputSection, the `buf` argument of `InputSectionBase::relocate` points
to the content of the containing OutputSection, instead of the content of the
InputSection itself, so `outSecOff` needs to be added in its callees.  This is
counter-intuitive and leads to many `- outSecOff` and `+ outSecOff`.

This patch makes `InputSection::writeTo` call `InputSectionBase::relocate` with
`outSecOff` added. relocAlloc/relocNonAlloc/relocateNonAllocForRelocatable can
thus be simplified now.

Updated test:

* non-abs-reloc.s: A minor offset bug is fixed for a diagnostic in `relocateNonAlloc`

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D85618
2020-08-11 08:06:38 -07:00
Fangrui Song 73ca96ebc1 [ELF][test] Enhance x86-64-split-stack-prologue-adjust-success.s & non-abs-reloc.s
Catch problems when outSecOff is not zero and outSecOff is added twice or not added.
2020-08-11 08:06:36 -07:00
Pavel Labath c3817728e7 [lld] s/dyn_cast/isa
Fixes some unused variable warnings with gcc.
2020-08-11 15:22:44 +02:00
Georgii Rymar 026e0bf984 [LLD][ELF] - Update and fix gnu-ifunc* tests.
It turns that gnu-ifunc-plt-i386.s and gnu-ifunc-plt.s tests are broken.

Initially they were implemented in D27581 and tested that `IRELATIVE` relocations
are placed after other relocations in `.rel.plt`.

Later, we started to place `IRELATIVE` relocations to `.rela.dyn` (D65651).

Also, at some point `.plt` was renamed to `.iplt` (D71520).

Now, `gnu-ifunc*` tests mentioned do not test what they intended to test initially:
they should test that `IRELATIVE` relocations are placed after other ones in
`.rela.dyn`. Also, comments needs to be updated accordingly after changes performed.

This patch updates them.

Differential revision: https://reviews.llvm.org/D85642
2020-08-11 15:14:33 +03:00
Greg McGary 49fb1c2e90 [lld-macho] improve handling of -platform_version
This improves the handling of `-platform_version` by addressing the FIXME in the code to process the arguments.

Reviewed By: int3, #lld-macho

Differential Revision: https://reviews.llvm.org/D81413
2020-08-10 18:47:16 -07:00
Greg McGary a379f2c251 [lld-macho] Handle command-line option -sectcreate SEG SECT FILE
Handle command-line option `-sectcreate SEG SECT FILE`, which inputs a binary blob from `FILE` into `SEG,SECT`

Reviewed By: int3

Differential Revision: https://reviews.llvm.org/D85501
2020-08-10 18:47:13 -07:00
Alexandre Ganea a3036b3863 Re-Re-land: [CodeView] Add full repro to LF_BUILDINFO record
This patch adds the missing information to the LF_BUILDINFO record, which allows for rebuilding a .CPP without any external dependency but the .OBJ itself (other than the compiler).

Some external tools that we are using (Recode, Live++) are extracting the information to reproduce a build without any knowledge of the build system. The LF_BUILDINFO stores a full path to the compiler, the PWD (CWD at program startup), a relative or absolute path to the TU, and the full CC1 command line. The command line needs to be freestanding (not depend on any environment variables). In the same way, MSVC doesn't store the provided command-line, but an expanded version (somehow their equivalent of CC1) which is also freestanding.

For more information see PR36198 and D43002.

Differential Revision: https://reviews.llvm.org/D80833
2020-08-10 13:36:30 -04:00
Wouter van Oortmerssen 582fd474dd [WebAssembly] wasm64: fix memory.init operand types
I had assumed they would all become in i64, but this is not necessary as long as data segments stay 32-bit, see:
https://github.com/WebAssembly/memory64/blob/master/proposals/memory64/Overview.md

Differential Revision: https://reviews.llvm.org/D85552
2020-08-10 10:15:20 -07:00
Alex Richardson e912fffd3a [ELF] Avoid creating a 2.1GB output file in arm-exidx-range.s
Currently both sections will be placed in the same PT_LOAD and therefore
lld generates a contiguous output file containing both sections.
By using AT(0xffff0000) the .vectors is placed a separate PT_LOAD and the
resulting file is now only a few kilobytes.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D85661
2020-08-10 18:08:07 +01:00
Sam Clegg b34ec5969f [lld][WebAssembly] Handle weakly referenced symbols when lazy (archive) version is see first
When a weak reference of a lazy symbol occurs we were not correctly
updating the lazy symbol.  We need to tag the existing lazy symbol
as weak and, in the case of a function symbol, give it a signature.

Without the signature we can't then create the dummy function which
is needed when an weakly undefined function is called.

We had tests for weakly referenced lazy symbols but we were only
tests in the case where the reference was seen before the lazy
symbol.

See: https://github.com/WebAssembly/wasi-libc/pull/214

Differential Revision: https://reviews.llvm.org/D85567
2020-08-10 08:16:55 -07:00
Fangrui Song 0334578edc [ELF] --wrap: don't leave the original symbol as SHN_UNDEF in .symtab or .dynsym 2020-08-08 18:18:20 -07:00
Petr Hosek a4d78d23c5 Revert "[CMake] Simplify CMake handling for zlib"
This reverts commit ccbc1485b5 which
is still failing on the Windows MLIR bots.
2020-08-08 17:08:23 -07:00
Petr Hosek ccbc1485b5 [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-08-08 16:44:08 -07:00
Fangrui Song 99cd56906a [ELF] --wrap: set isUsedInRegularObj of __wrap_ if it is defined or shared
Fixes PR47017 (a regression when fixing PR46169): if __wrap_ is shared,
it is not exported.
2020-08-08 09:24:31 -07:00
Fangrui Song d30d461938 [ELF] Support .cfi_signal_frame
glibc/sysdeps/unix/sysv/linux/x86_64/sigaction.c libc.a(sigaction.o) has a CIE
with the augmentation string "zRS". Support 'S' to allow --icf={safe,all}.
2020-08-07 22:08:44 -07:00
Fangrui Song 164a02d0fa [ELF]: --icf: don't fold sections referencing sections with LCDA after D84610 2020-08-07 13:42:25 -07:00
Jez Ng 25367dfefb [lld-macho] Add .tbd support for frameworks
Required for e.g. linking iOS apps since they don't have a platform-native
SDK

Reviewed By: #lld-macho, compnerd, smeenai

Differential Revision: https://reviews.llvm.org/D85153
2020-08-07 11:04:54 -07:00
Jez Ng ca85e37338 [lld-macho] Support static linking of thread-locals
Note: What ELF refers to as "TLS", Mach-O seems to refer to as "TLV", i.e.
thread-local variables.

This diff implements support for TLV relocations that reference defined
symbols. On x86_64, TLV relocations are always used with movq opcodes, so for
defined TLVs, we don't need to create a synthetic section to store the
addresses of the symbols -- we can just convert the `movq` to a `leaq`.

One notable quirk of Mach-O's TLVs is that absolute-address relocations
inside TLV-defining sections behave differently -- their addresses are
no longer absolute, but relative to the start of the target section.
(AFAICT, RIP-relative relocations are not allowed in these sections.)

Reviewed By: #lld-macho, compnerd, smeenai

Differential Revision: https://reviews.llvm.org/D85080
2020-08-07 11:04:52 -07:00
Jez Ng 4e43f18048 [lld-macho] Ensure .tbss sections are also considered as ZeroFilled
This diff makes the behavior in {D80859} and {D81888} apply to
thread-local ZeroFill sections too. I realized this was necessary whie
trying to implement thread-local variables.

Reviewed By: #lld-macho, compnerd, MaskRay

Differential Revision: https://reviews.llvm.org/D85079
2020-08-07 11:04:41 -07:00
Victor Huang 6c64f05b90 [PowerPC] Add compatibility check for PPC PLT stubs
Compatibility checks for PPC64PltCallStub and PPC64PCRelPLTStub are
added in this patch to prevent the usage of incompatible thunk/stub.

Reviewed By: sfertile, nemanjai, stefanp

Differential Revision: https://reviews.llvm.org/D85459
2020-08-07 13:45:18 +00:00
Christian Kühnel f3cc4df51d Revert "[CMake] Simplify CMake handling for zlib"
This reverts commit 1adc494bce.
This patch broke the Windows compilation on buildbot and pre-merge testing:
http://lab.llvm.org:8011/builders/mlir-windows/builds/5945
https://buildkite.com/llvm-project/llvm-master-build/builds/780
2020-08-07 09:36:49 +02:00
Fangrui Song 004be4037e [ELF] Change tombstone values to (.debug_ranges/.debug_loc) 1 and (other .debug_*) 0
tl;dr See D81784 for the 'tombstone value' concept. This patch changes our behavior to be almost the same as GNU ld (except that we also use 1 for .debug_loc):

* .debug_ranges & .debug_loc: 1 (LLD<11: 0+addend; GNU ld uses 1 for .debug_ranges)
* .debug_*: 0 (LLD<11: 0+addend; GNU ld uses 0; future LLD: -1)

We make the tweaks because:

1) The new tombstone is novel and needs more time to be adopted by consumers before it's the default.
2) The old (gold) strategy had problems with zero-length functions - so rather than going back that, we're going to the GNU ld strategy which doesn't have that problem.
3) One slight tweak to (2) is to apply the .debug_ranges workaround to .debug_loc for the same reasons it applies to debug_ranges - to avoid terminating lists early.

-----

http://lists.llvm.org/pipermail/llvm-dev/2020-July/143482.html

The tombstone value -1 in .debug_line caused problems to lldb (fixed by D83957;
will be included in 11.0.0) and breakpad (fixed by
https://crrev.com/c/2321300). It may potentially affects other DWARF consumers.

For .debug_ranges & .debug_loc: 1, an argument preferring 1 (GNU ld for .debug_ranges) over -2 is that:
```
{-1, -2}    <<< base address selection entry
{0, length} <<< address range
```
may create a situation where low_pc is greater than high_pc. So we use
1, the GNU ld behavior for .debug_ranges

For other .debug_* sections, there haven't been many reports. One issue is that
bloaty (src/dwarf.cc) can incorrectly count address ranges in .debug_ranges . To
reduce similar disruption, this patch changes the tombstone values to be similar to GNU ld.

This does mean another behavior change to the default trunk behavior. Sorry
about it. The default trunk behavior will be similar to release/11.x while we work on a transition plan for LLD users.

Reviewed By: dblaikie, echristo

Differential Revision: https://reviews.llvm.org/D84825
2020-08-06 15:30:08 -07:00
Fangrui Song a6db64ef4a [ELF] Allow sections after a non-SHF_ALLOC section to be covered by PT_LOAD
GNU ld allows sections after a non-SHF_ALLOC section to be covered by PT_LOAD
(PR37607) and assigns addresses to non-SHF_ALLOC output sections (similar to
SHF_ALLOC NOBITS sections. The location counter is not advanced).

This patch tries to fix PR37607 (remove a special case in
`Writer<ELFT>::createPhdrs`). To make the created PT_LOAD meaningful, we cannot
reset dot to 0 for a middle non-SHF_ALLOC output section. This results in
removal of two special cases in LinkerScript::assignOffsets. Non-SHF_ALLOC
non-orphan sections can have non-zero addresses like in GNU ld.

The zero address rule for non-SHF_ALLOC sections is weakened to apply to orphan
only. This results in a special case in createSection and findOrphanPos, respectively.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85100
2020-08-06 08:27:15 -07:00
Muhammad Omair Javaid d9e191cb17 Revert "[ELF] Allow sections after a non-SHF_ALLOC section to be covered by PT_LOAD"
This reverts commit 030ddc0a0b.

This breaks http://lab.llvm.org:8011/builders/lldb-arm-ubuntu
and http://lab.llvm.org:8011/builders/lldb-aarch64-ubuntu

Differential Revision: https://reviews.llvm.org/D85100
2020-08-06 16:30:05 +05:00
Fangrui Song 3af4039876 [ELF][test] Add another -triple=x86_64 to linkorder-mixed.s
Fixes an error on macOS:
error: mach-o section specifier uses an unknown section type
2020-08-05 17:00:37 -07:00
Shoaib Meenai 8b9e86fea6 [lld-macho] Fix errant search and replace. NFC 2020-08-05 16:59:53 -07:00
Fangrui Song 279e4cf782 [ELF] Fix type of ciesWithLSDA after D84610 2020-08-05 16:33:54 -07:00
Fangrui Song eb45b978b7 [ELF][test] Add REQUIES: x86 and -triple=x86_64 to linkorder-mixed.s 2020-08-05 16:28:44 -07:00
Fangrui Song b216c80cc2 [ELF] Allow SHF_LINK_ORDER sections to have sh_link=0
Part of https://bugs.llvm.org/show_bug.cgi?id=41734

The semantics of SHF_LINK_ORDER have been extended to represent metadata
sections associated with some other sections (usually text).

The associated text section may be discarded (e.g. LTO) and we want the
metadata section to have sh_link=0 (D72899, D76802).

Normally the metadata section is only referenced by the associated text
section. sh_link=0 means the associated text section is discarded, and
the metadata section will be garbage collected. If there is another
section (.gc_root) referencing the metadata section, the metadata
section will be retained. It's the .gc_root consumer's job to validate
the metadata sections.

  # This creates a SHF_LINK_ORDER .meta with sh_link=0
  .section .meta,"awo",@progbits,0
  1:
  .section .meta,"awo",@progbits,foo
  2:

  .section .gc_root,"a",@progbits
  .quad 1b
  .quad 2b

Reviewed By: pcc, jhenderson

Differential Revision: https://reviews.llvm.org/D72904
2020-08-05 16:17:42 -07:00
Petr Hosek 1adc494bce [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-08-05 16:07:11 -07:00
Fangrui Song 030ddc0a0b [ELF] Allow sections after a non-SHF_ALLOC section to be covered by PT_LOAD
GNU ld allows sections after a non-SHF_ALLOC section to be covered by PT_LOAD
(PR37607) and assigns addresses to non-SHF_ALLOC output sections (similar to
SHF_ALLOC NOBITS sections. The location counter is not advanced).

This patch tries to fix PR37607 (remove a special case in
`Writer<ELFT>::createPhdrs`). To make the created PT_LOAD meaningful, we cannot
reset dot to 0 for a middle non-SHF_ALLOC output section. This results in
removal of two special cases in LinkerScript::assignOffsets. Non-SHF_ALLOC
non-orphan sections can have non-zero addresses like in GNU ld.

The zero address rule for non-SHF_ALLOC sections is weakened to apply to orphan
only. This results in a special case in createSection and findOrphanPos, respectively.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D85100
2020-08-05 09:30:23 -07:00
Fangrui Song 21b4f8060a [ELF] --icf: don't fold text sections with LSDA
Fix PR36272 and PR46835

A .eh_frame FDE references a text section and (optionally) a LSDA (in
.gcc_except_table).  Even if two text sections have identical content and
relocations (e.g. a() and b()), we cannot fold them if their LSDA are different.

```
void foo();
void a() {
  try { foo(); } catch (int) { }
}
void b() {
  try { foo(); } catch (float) { }
}
```

Scan .eh_frame pieces with LSDA and disallow referenced text sections to be
folded. If two .gcc_except_table have identical semantics (usually identical
content with PC-relative encoding), we will lose folding opportunity.
For ClickHouse (an exception-heavy application), this can reduce --icf=all efficiency
from 9% to 5%. There may be some percentage we can reclaim without affecting
correctness, if we analyze .eh_frame and .gcc_except_table sections.

gold 2.24 implemented a more complex fix (resolution to
https://sourceware.org/bugzilla/show_bug.cgi?id=21066) which combines the
checksum of .eh_frame CIE/FDE pieces.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D84610
2020-08-05 09:16:28 -07:00
Fangrui Song acb66b9111 [ELF] --oformat=binary: use LMA to compute file offsets
--oformat=binary is rare (used in a few places in FreeBSD, see `stand/i386/mbr/Makefile` `LDFLAGS_BIN`)
The result should be identical to a normal output transformed by `objcopy -O binary`.

The current implementation ignores addresses and lays out sections by
respecting output section alignments. It can fail when an output section
address is specified, e.g. `.rodata ALIGN(16) :` (PR33651).

Fix PR33651 by respecting LMA. The code is similar to
`tools/llvm-objcop/ELF/Object.cpp` BinaryWriter::finalize after D71035 and D79229.
Unforunately for an output section without PT_LOAD, we assume its LMA is equal
to its VMA. So the result is still incorrect when an output section LMA
(`AT(...)`) is specified

Also drop `alignTo(off, config->wordsize)`. GNU ld does not round up the file size.

Differential Revision: https://reviews.llvm.org/D85086
2020-08-05 09:10:01 -07:00
Saleem Abdulrasool 0ccda7c232 MachO: support `-syslibroot`
This adds support for the `-syslibroot` option.  This is required to
make the library search order actually function.  With this, it is now
possible to link a test Darwin x86_64 program with lld on Darwin.

Differential Revision: https://reviews.llvm.org/D82252
Reviewed By: Jez Ng
2020-08-05 08:41:24 -07:00
Hans Wennborg 3ab01550b6 Revert "[CMake] Simplify CMake handling for zlib"
This quietly disabled use of zlib on Windows even when building with
-DLLVM_ENABLE_ZLIB=FORCE_ON.

> Rather than handling zlib handling manually, use find_package from CMake
> to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
> HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
> set to YES, which requires the distributor to explicitly select whether
> zlib is enabled or not. This simplifies the CMake handling and usage in
> the rest of the tooling.
>
> This is a reland of abb0075 with all followup changes and fixes that
> should address issues that were reported in PR44780.
>
> Differential Revision: https://reviews.llvm.org/D79219

This reverts commit 10b1b4a231 and follow-ups
64d99cc6ab and
f9fec0447e.
2020-08-05 12:31:44 +02:00
Fangrui Song bcea3a7a28 Add test utility 'split-file'
See https://lists.llvm.org/pipermail/llvm-dev/2020-July/143373.html
"[llvm-dev] Multiple documents in one test file" for some discussions.

This patch has explored several alternatives. The current semantics are similar to
what @dblaikie proposed.
`split-file filename output` splits the input file into multiple parts separated by
regex `^(.|//)--- filename` and write each part to the file `output/filename`
(`filename` can include path separators).

Use case A (organizing input of different formats (e.g. linker
script+assembly) in one file).

```
# RUN: split-file %s %t
# RUN: llvm-mc %t/asm -o %t.o
# RUN: ld.lld -T %t/lds %t.o -o %t
This is sometimes better than the %S/Inputs/ approach because the user
can see the auxiliary files immediately and don't have to open another file.

# asm
...
# lds
...
```

Use case B (for utilities which don't have built-in input splitting
feature):

```
// RUN: split-file %s %t
// RUN: llc < %t/1.ll | FileCheck %s --check-prefix=CASE1
// RUN: llc < %t/2.ll | FileCheck %s --check-prefix=CASE2
Combing tests prudently can improve readability.
For example, when testing parsing errors if the recovery mechanism isn't possible,
grouping the tests in one file can more readily see test coverage/strategy.

//--- 1.ll
...
//--- 2.ll
...
```

Since this is a new utility, there is no git history concerns for
UpperCase variable names. I use lowerCase variable names like mlir/lld.

Reviewed By: jhenderson, lattner

Differential Revision: https://reviews.llvm.org/D83834
2020-08-03 20:42:09 -07:00
Petr Hosek 81eeabbd97 [ELF] Add --dependency-file option
Clang and GCC have a feature (-MD flag) to create a dependency file
in a format that build systems such as Make or Ninja can read, which
specifies all the additional inputs such .h files.

This change introduces the same functionality to lld bringing it to
feature parity with ld and gold which gained this feature recently.
See https://sourceware.org/bugzilla/show_bug.cgi?id=22843 for more
details and discussion.

The implementation corresponds to -MD -MP compiler flag where the
generated dependency file also includes phony targets which works
around the errors where the dependency is removed. This matches the
format used by ld and gold.

Fixes PR42806

Differential Revision: https://reviews.llvm.org/D82437
2020-08-03 16:59:13 -07:00
Fangrui Song e281376e99 [ELF] --wrap: set isUsedInRegularObj of __wrap_ only if it is defined
Fixes PR46169
2020-08-01 18:19:14 -07:00
Sriraman Tallam 938adf42e6 Fix a test typo which caused a breakage. 2020-07-31 12:26:53 -07:00
Sriraman Tallam a77afc62d9 New test for basic block sections options.
This tests lld basic block sections options:
+ --lto-basic-block-sections=
+ --lto-unique-basic-block-section-names

Differential Revision: https://reviews.llvm.org/D84462
2020-07-31 12:02:52 -07:00
Sriraman Tallam ca6b6d40ff Rename basic block sections options to be consistent.
D68049 created options for basic block sections: -fbasic-block-sections=,
-funique-basic-block-section-names. Rename options in llc and lld (--lto-)
to be consistent. Specifically,

+ Rename basicblock-sections to basic-block-sections
+ Rename unique-bb-section-names to unique-basic-block-section-names

Differential Revision: https://reviews.llvm.org/D84462
2020-07-31 11:50:55 -07:00
Petr Hosek 0bd918c828 Revert "[ELF] Add --dependency-file option"
This reverts commit b4c7657ba6 which
seems to be breaking certain bots with assertion error.
2020-07-31 01:12:59 -07:00
Wolfgang Pieb 9f02258942 Make a test case more flexible on Windows wrt the number of backslashes. 2020-07-30 20:38:12 -07:00
Zequan Wu 763671f387 [COFF] Port CallGraphSort to COFF from ELF 2020-07-30 15:21:44 -07:00
Jez Ng c89e46e767 [lld-macho] Add comment for literal argument 2020-07-30 14:38:58 -07:00
Jez Ng 98210796e1 [lld-macho] Make __LINKEDIT sections contiguous
codesign (or more specifically libstuff) checks that each section in
__LINKEDIT ends where the next one starts -- no gaps are permitted. This
diff achieves it by aligning every section's start and end points to
WordSize.

Remarks: ld64 appears to satisfy the constraint by adding padding bytes
when generating the __LINKEDIT data, e.g. by emitting BIND_OPCODE_DONE
(which is a 0x0 byte) repeatedly. I think the approach this diff takes
is a bit more elegant, but I'm not sure if it's too restrictive. In
particular, it assumes padding always uses the zero byte. But we can
revisit this later.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D84718
2020-07-30 14:30:07 -07:00
Jez Ng 22e6648a18 [lld-macho] Implement -headerpad
Tools like `install_name_tool` and `codesign` may modify the Mach-O
header and increase its size. The linker has to provide padding to make this
possible. This diff does that, plus sets its default value to 32 bytes (which
is what ld64 does).

Unlike ld64, however, we lay out our sections *exactly* `-headerpad` bytes from
the header, whereas ld64 just treats the padding requirement as a lower bound.
ld64 actually starts laying out the non-header sections in the __TEXT segment
from the end of the (page-aligned) segment rather than the front, so its
binaries typically have more than `-headerpad` bytes of actual padding.
We should consider implementing the same alignment behavior.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D84714
2020-07-30 14:29:31 -07:00
Jez Ng 3587de2281 [lld-macho] Support __dso_handle for C++
The C++ ABI requires dylibs to pass a pointer to __cxa_atexit which does
e.g. cleanup of static global variables. The C++ spec says that the pointer
can point to any address in one of the dylib's segments, but in practice
ld64 seems to set it to point to the header, so that's what's implemented
here.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D83603
2020-07-30 14:28:41 -07:00
Peiyuan Song da324f9904 [LLD] [Mingw] Don't export symbols from profile generate
Differential Revision: https://reviews.llvm.org/D84756
2020-07-30 23:33:20 +03:00
Petr Hosek b4c7657ba6 [ELF] Add --dependency-file option
Clang and GCC have a feature (-MD flag) to create a dependency file
in a format that build systems such as Make or Ninja can read, which
specifies all the additional inputs such .h files.

This change introduces the same functionality to lld bringing it to
feature parity with ld and gold which gained this feature recently.
See https://sourceware.org/bugzilla/show_bug.cgi?id=22843 for more
details and discussion.

The implementation corresponds to -MD -MP compiler flag where the
generated dependency file also includes phony targets which works
around the errors where the dependency is removed. This matches the
format used by ld and gold.

Fixes PR42806

Differential Revision: https://reviews.llvm.org/D82437
2020-07-30 12:31:20 -07:00
Fangrui Song ed7bde0e4b [ELF][test] Fix ppc64-reloc-pcrel34-overflow.s 2020-07-29 13:01:31 -07:00
Victor Huang 8dbea4785c [PowerPC] Support for R_PPC64_REL24_NOTOC calls where the caller has no TOC and the callee is not DSO local
This patch supports the situation where caller does not have a valid TOC and
calls using the R_PPC64_REL24_NOTOC relocation and the callee is not DSO local.
In this case the call cannot be made directly since the callee may or may not
require a valid TOC pointer. As a result this situation require a PC-relative
plt stub to set up r12.

Reviewed By: sfertile, MaskRay, stefanp

Differential Revision: https://reviews.llvm.org/D83669
2020-07-29 19:49:28 +00:00
Andrew Ng 8725a49409 [ELF][test] Add test coverage of `__real_` to wrap-plt.s
Differential Revision: https://reviews.llvm.org/D84749
2020-07-29 14:10:38 +01:00
Fangrui Song dd405f1a53 Revert D83834 "Add test utility 'extract'"
This reverts commit d054c7ee2e.

There are discussions about the utility name, its functionality and user interface.
Revert before we reach consensus.
2020-07-28 13:26:33 -07:00
Hafiz Abid Qadeer 1f166edeb4 [lld][linkerscript] Fix handling of DEFINED.
Current implementation did not check that symbols is actually defined. Only checked for presence.  GNU ld documentation says,

"Return 1 if symbol is in the linker global symbol table and is defined before the statement using DEFINED in the script, otherwise return 0."

https://sourceware.org/binutils/docs/ld/Builtin-Functions.html#Builtin-Functions

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D83758
2020-07-28 21:18:01 +01:00
Martin Storsjö 745eb02496 [LLD] [MinGW] Implement the --no-seh flag
Previously this flag was just ignored. If set, set the
IMAGE_DLL_CHARACTERISTICS_NO_SEH bit, regardless of the normal safeSEH
machinery.

In mingw configurations, the safeSEH bit might not be set in e.g. object
files built from handwritten assembly, making it impossible to use the
normal safeseh flag. As mingw setups don't generally use SEH on 32 bit
x86 at all, it should be fine to set that flag bit though - hook up
the existing GNU ld flag for controlling that.

Differential Revision: https://reviews.llvm.org/D84701
2020-07-28 21:08:37 +03:00
Jez Ng a5b89c2853 [lld-macho] Fix no-filelist test on Windows 2020-07-28 11:04:43 -07:00
Jez Ng d32e32500f [lld-macho] Fix segment filesize calculation
The previous approach of adding up the file sizes of the
component sections ignored the fact that the sections did not have to be
contiguous in the file. As such, it was underestimating the true size.

I discovered this issue because `codesign` checks whether `__LINKEDIT`
extends to the end of the file. Since we were underestimating segment
sizes, this check failed.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D84574
2020-07-28 10:02:19 -07:00
Jez Ng 4853a86022 [lld-macho] Support -filelist
XCode passes files in using this flag

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D84486
2020-07-28 10:02:19 -07:00
Christy Lee bd4757cc4e [ELF] --reproduce should include lto sample profile
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D84569
2020-07-28 09:41:41 -07:00
Petr Hosek 64d99cc6ab [CMake] Move find_package(ZLIB) to LLVMConfig
This way, downstream projects don't have to invoke find_package(ZLIB)
reducing the amount of boilerplate.

Differential Revision: https://reviews.llvm.org/D84691
2020-07-27 17:13:55 -07:00
Logan Smith a52aea0ba6 Use INTERFACE_COMPILE_OPTIONS to disable -Wsuggest-override for any target that links to gtest
This cleans up several CMakeLists.txt's where -Wno-suggest-override was manually specified. These test targets now inherit this flag from the gtest target.

Some unittests CMakeLists.txt's, in particular Flang and LLDB, are not touched by this patch. Flang manually adds the gtest sources itself in some configurations, rather than linking to LLVM's gtest target, so this fix would be insufficient to cover those cases. Similarly, LLDB has subdirectories that manually add the gtest headers to their include path without linking to the gtest target, so those subdirectories still need -Wno-suggest-override to be manually specified to compile without warnings.

Differential Revision: https://reviews.llvm.org/D84554
2020-07-27 08:37:01 -07:00
Martin Storsjö 343ffa70fc [LLD] [COFF] Fix mingw comdat associativity for leader symbols with a different name
For a weak symbol func in a comdat, the actual leader symbol ends up
named like .weak.func.default*. Likewise, for stdcall on i386, the symbol
may be named _func@4, while the section suffix only is "func", which the
previous implementation didn't handle.

This fixes unwinding through weak functions when using
-ffunction-sections in mingw environments.

Differential Revision: https://reviews.llvm.org/D84607
2020-07-27 17:32:08 +03:00
Isaac Richter fa1145a8d2 [lld][ELF] Add LOG2CEIL builtin ldscript function
This patch adds support for the LOG2CEIL builtin function in linker scripts: https://sourceware.org/binutils/docs/ld/Builtin-Functions.html#index-LOG2CEIL_0028exp_0029

As documented for LD, and to keep compatibility, LOG2CEIL(0) returns 0 (not -inf).

The test vectors are somewhat arbitrary. We check minimum values (0-4); middle values (2^32, and 2^32+1); and the maximum value (2^64-1).

The checks for LOG2CEIL explicitly use full 64-bit values (16 hex digits). This is needed to properly verify that -inf and other interesting results aren't returned. (For some reason, all other tests in operators.test use only 14 digits.)

Differential revision: https://reviews.llvm.org/D84054
2020-07-27 12:16:43 +03:00
Martin Storsjö 8dc8203932 [LLD] [COFF] Fix test to properly test all aspects of c3b1d730d6. NFC.
Previously, the test could pass with one part of c3b1d730d6 removed.
2020-07-26 23:10:20 +03:00
Jez Ng 9282d04e04 [lld-macho] Support lookup of dylibs in frameworks
Needed for testing Objective-C programs (since e.g. Core
Foundation is a framework)

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D83925
2020-07-26 12:46:46 -07:00
Jez Ng 06a0dd2467 [lld-macho] Ignore -dependency_info and its argument
XCode passes in this flag, which we do not yet implement. Skip
over the argument for now so we can at least successfully parse the
linker invocation.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D84485
2020-07-24 15:55:27 -07:00
Jez Ng 31d5885842 [lld-macho] Partial support for weak definitions
This diff adds support for weak definitions, though it doesn't handle weak
symbols in dylibs quite correctly -- we need to emit binding opcodes for them
in the weak binding section rather than the lazy binding section.

What *is* covered in this diff:

1. Reading the weak flag from symbol table / export trie, and writing it to the
   export trie
2. Refining the symbol table's rules for choosing one symbol definition over
   another. Wrote a few dozen test cases to make sure we were matching ld64's
   behavior.

We can now link basic C++ programs.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D83532
2020-07-24 15:55:25 -07:00
Georgii Rymar ae4279bd3e [LLD][ELF] - Linkerscript: report location for the "unclosed comment in a linker script" error.
Currently we print "error: unclosed comment in a linker script", which doesn't
provide information about the real error location.

Fixes https://bugs.llvm.org/show_bug.cgi?id=46793.

Differential revision: https://reviews.llvm.org/D84300
2020-07-24 11:38:26 +03:00
Petr Hosek 10b1b4a231 [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-07-23 23:05:36 -07:00
Fangrui Song d054c7ee2e Add test utility 'extract'
See https://lists.llvm.org/pipermail/llvm-dev/2020-July/143373.html
"[llvm-dev] Multiple documents in one test file" for some discussions.

`extract part filename` splits the input file into multiple parts separated by
regex `^(.|//)--- ` and extract the specified part to stdout or the
output file (if specified).

Use case A (organizing input of different formats (e.g. linker
script+assembly) in one file).

```
// RUN: extract lds %s -o %t.lds
// RUN: extract asm %s -o %t.s
// RUN: llvm-mc %t.s -o %t.o
// RUN: ld.lld -T %t.lds %t.o -o %t
This is sometimes better than the %S/Inputs/ approach because the user
can see the auxiliary files immediately and don't have to open another file.
```

Use case B (for utilities which don't have built-in input splitting
feature):

```
// RUN: extract case1 %s | llc | FileCheck %s --check-prefix=CASE1
// RUN: extract case2 %s | llc | FileCheck %s --check-prefix=CASE2
Combing tests prudently can improve readability.
This is sometimes better than having multiple test files.
```

Since this is a new utility, there is no git history concerns for
UpperCase variable names. I use lowerCase variable names like mlir/lld.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83834
2020-07-23 19:15:35 -07:00
Petr Hosek 38c71b7c85 Revert "[CMake] Simplify CMake handling for zlib"
This reverts commit 1d09ecf361 since
it breaks sanitizer bots.
2020-07-23 15:12:42 -07:00
Petr Hosek 1d09ecf361 [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-07-23 14:47:25 -07:00
Fangrui Song 4e80c768c2 [ELF] Support -r --gc-sections
-r --gc-sections is usually not useful because it just makes intermediate output
smaller. https://bugs.llvm.org/show_bug.cgi?id=46700#c7 mentions a use case:
validating the absence of undefined symbols ealier than in the final link.

After D84129 (SHT_GROUP support in -r links), we can support -r
--gc-sections without extra code. So let's allow it.

Reviewed By: grimar, jhenderson

Differential Revision: https://reviews.llvm.org/D84131
2020-07-23 08:16:01 -07:00
Logan Smith 77e0e9e17d Reapply "Try enabling -Wsuggest-override again, using add_compile_options instead of add_compile_definitions for disabling it in unittests/ directories."
add_compile_options is more sensitive to its location in the file than add_definitions--it only takes effect for sources that are added after it. This updated patch ensures that the add_compile_options is done before adding any source files that depend on it.

Using add_definitions caused the flag to be passed to rc.exe on Windows and thus broke Windows builds.
2020-07-22 17:50:19 -07:00
Logan Smith 97a0f80c46 Revert "Try enabling -Wsuggest-override again, using add_compile_options instead of add_compile_definitions for disabling it in unittests/ directories."
This reverts commit 388c9fb1af.
2020-07-22 15:07:01 -07:00
Logan Smith 388c9fb1af Try enabling -Wsuggest-override again, using add_compile_options instead of add_compile_definitions for disabling it in unittests/ directories.
Using add_compile_definitions caused the flag to be passed to rc.exe on Windows and thus broke Windows builds.
2020-07-22 14:19:34 -07:00
Louis Dionne afa1afd410 [CMake] Bump CMake minimum version to 3.13.4
This upgrade should be friction-less because we've already been ensuring
that CMake >= 3.13.4 is used.

This is part of the effort discussed on llvm-dev here:

  http://lists.llvm.org/pipermail/llvm-dev/2020-April/140578.html

Differential Revision: https://reviews.llvm.org/D78648
2020-07-22 14:25:07 -04:00
Hans Wennborg 3eec657825 Revert "Enable -Wsuggest-override in the LLVM build" and the follow-ups.
After lots of follow-up fixes, there are still problems, such as
-Wno-suggest-override getting passed to the Windows Resource Compiler
because it was added with add_definitions in the CMake file.

Rather than piling on another fix, let's revert so this can be re-landed
when there's a proper fix.

This reverts commit 21c0b4c1e8.
This reverts commit 81d68ad27b.
This reverts commit a361aa5249.
This reverts commit fa42b7cf29.
This reverts commit 955f87f947.
This reverts commit 8b16e45f66.
This reverts commit 308a127a38.
This reverts commit 274b6b0c7a.
This reverts commit 1c7037a2a5.
2020-07-22 20:23:58 +02:00
Logan Smith 81d68ad27b [lld] Disable -Wsuggest-override for unittests 2020-07-21 17:40:59 -07:00
Fangrui Song 86ab98b001 [ELF] -r: rewrite SHT_GROUP content if some members are combined or discarded
* If two group members are combined, we should leave just one index in the SHT_GROUP content.
* If a group member is discarded (/DISCARD/ or upcoming -r --gc-sections combination),
  we should drop its index in the SHT_GROUP content. LLD currently crashes (`getOutputSection()` is null).

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D84129
2020-07-21 08:49:45 -07:00
Victor Huang 91cce1a2bc [PowerPC] Implement R_PPC64_REL24_NOTOC local calls, callee requires a TOC
The PC Relative code now allows for calls that are marked with the relocation
R_PPC64_REL24_NOTOC. This indicates that the caller does not have a valid TOC
pointer in R2 and does not require R2 to be restored after the call.

This patch is added to support local calls to callees that require a TOC

Reviewed By: sfertile, MaskRay, nemanjai, stefanp

Differential Revision: https://reviews.llvm.org/D83504
2020-07-20 17:46:49 +00:00
Sylvain Audi 3a108ab256 [LLD][COFF] Skip computation of the undefined symbols references that are not shown
The "undefined symbol" error message from lld-link displays up to 3 references to that symbol, and the number of extra references not shown.

This patch removes the computation of the strings for those extra references.

It fixes a freeze of lld-link we accidentally encountered when activating asan on a large project, without linking with the asan library.
In that case, __asan_report_load8 was referenced more than 2 million times, causing the computation of that many display strings, of which only 3 were used.

Differential Revision: https://reviews.llvm.org/D83510
2020-07-20 13:45:16 -04:00
Hans Wennborg 8a197e0b16 Require shell for lld/test/ELF/arm-exidx-range.s
The test fails in 32-bit Windows builds for unclear reasons:

ld.lld: error: failed to open
C:\src\llvm_package_1100-rc1\build32_stage0\tools\lld\test\ELF\Output\arm-exidx-range.s.tmp:
The parameter is incorrect.
2020-07-20 17:49:10 +02:00
Georgii Rymar ef377d3f2d [LLD][ELF] - Update the lld/test/ELF/mips-merge-abiflags.s after llvm-readelf change.
The warning message was updated
(https://reviews.llvm.org/rG256aea816da45bd3bc52317cbc89e704095a8991)
2020-07-20 12:17:49 +03:00
Fangrui Song 7099a4b56b [ELF][test] Update test after DW_EH_PE_sdata4/DW_EH_PE_sdata8 change 2020-07-18 09:07:50 -07:00
Michele Scandale 53880b8cb9 [CMake] Make `intrinsics_gen` dependency unconditional.
The `intrinsics_gen` target exists in the CMake exports since r309389
(see LLVMConfig.cmake.in), hence projects can depend on `intrinsics_gen`
even it they are built separately from LLVM.

Reviewed By: MaskRay, JDevlieghere

Differential Revision: https://reviews.llvm.org/D83454
2020-07-17 16:43:17 -07:00
Benjamin Kramer 9a0689e072 Make helpers static. NFC. 2020-07-17 13:49:11 +02:00
Wouter van Oortmerssen cc1b9b680f [WebAssembly] 64-bit (function) pointer fixes.
Accounting for the fact that Wasm function indices are 32-bit, but in wasm64 we want uniform 64-bit pointers.
Includes reloc types for 64-bit table indices.

Differential Revision: https://reviews.llvm.org/D83729
2020-07-16 14:10:22 -07:00
Wouter van Oortmerssen 29f8c9f6c2 [WebAssembly] Triple::wasm64 related cleanup
Differential Revision: https://reviews.llvm.org/D83713
2020-07-16 12:01:10 -07:00
Hans Wennborg 7ab7b979d2 Bump the trunk major version to 12
and clear the release notes.
2020-07-15 12:05:05 +02:00
Georgii Rymar 2e10b7a39b [LLD][ELF] - Update test after yaml2obj change. NFC.
Names of YAML keys changed in rG7a587ca93200c49e47fe205ce037895c81c5a542
2020-07-15 13:00:48 +03:00
Martin Storsjö 92f7bd3a39 [LLD] [MinGW] Implement the --file-alignment and --section-alignment options
Differential Revision: https://reviews.llvm.org/D83720
2020-07-15 08:34:08 +03:00
Petr Hosek bcd27d9d73 Revert "[CMake] Simplify CMake handling for zlib"
This reverts commit 8c1a79dc12 because
it fails when zlib isn't installed.
2020-07-14 19:56:10 -07:00
Petr Hosek 8c1a79dc12 [CMake] Simplify CMake handling for zlib
Rather than handling zlib handling manually, use find_package from CMake
to find zlib properly. Use this to normalize the LLVM_ENABLE_ZLIB,
HAVE_ZLIB, HAVE_ZLIB_H. Furthermore, require zlib if LLVM_ENABLE_ZLIB is
set to YES, which requires the distributor to explicitly select whether
zlib is enabled or not. This simplifies the CMake handling and usage in
the rest of the tooling.

This is a reland of abb0075 with all followup changes and fixes that
should address issues that were reported in PR44780.

Differential Revision: https://reviews.llvm.org/D79219
2020-07-14 19:30:08 -07:00
Igor Kudrin c4fc26b4c0 [ELF] Do not leave undefined symbols (specified by -init and -fini) if they are defined in non-fetched archive members
After D69985, symbols for "-init" and "-fini" were unconditionally
marked as used even if they were just lazy symbols seen when scanning
archives. That resulted in exposing them in the symbol table of an
output file, as Undefined, which added unwanted dependencies. The patch
fixes the issue by checking the kind of the symbols before the marking.

Differential Revision: https://reviews.llvm.org/D83549
2020-07-14 16:35:17 +07:00
Georgii Rymar fbb6c9df28 [LLD][ELF] - Fix the test after llvm-readelf output change.
An issue for llvm-readelf was fixed in
https://reviews.llvm.org/rG84a170178c4431b7536c83ff0e5ce80774d08df6

Now it produce no output for this test.
2020-07-13 15:18:30 +03:00
Hans Wennborg e73d0b5719 [COFF] Error on unexpected .pdata size
Previously, lld would crash if the .pdata size was not an even multiple
of the expected .pdata entry size. This makes it error gracefully instead.

(We hit this in Chromium due to an assembler problem: https://crbug.com/1101577)

Differential revision: https://reviews.llvm.org/D83479
2020-07-13 13:38:39 +02:00
Ben Dunbobbin 6bda276f93 [LLD][ELF][Windows] small improvement to D82567
Bail early if there is no existing output file to be overwritten.

Differential Revision: https://reviews.llvm.org/D83272
2020-07-13 12:02:20 +01:00
Georgii Rymar af16a45683 [LLD][ELF] - Allow relocation sections to appear before their target sections.
It allows handling cases when we have SHT_REL[A] sections before target
sections in objects.

This fixes https://bugs.llvm.org/show_bug.cgi?id=46632

which says: "Normally it is not what compilers would emit. We have to support it,
because some custom tools might want to use this feature, which is not restricted by ELF gABI"

Differential revision: https://reviews.llvm.org/D83469
2020-07-13 13:59:54 +03:00
Ayke van Laethem 69e60c9dc7
[LLD][ELF][AVR] Implement the missing relocation types
Implements the missing relocation types for AVR target.
The results have been cross-checked with binutils.

Original patch by LemonBoy. Some changes by me.

Differential Revision: https://reviews.llvm.org/D78741
2020-07-12 18:18:54 +02:00
Eric Christopher 256e4d46a6 Fix signed vs unsigned comparison warnings a different way. 2020-07-10 22:52:50 -07:00
Alexandre Ganea b71499ac9e Revert "Re-land [CodeView] Add full repro to LF_BUILDINFO record"
This reverts commit add59ecb34 and 41d2813a5f.
2020-07-10 19:46:16 -04:00
Alexandre Ganea add59ecb34 Re-land [CodeView] Add full repro to LF_BUILDINFO record
This patch adds some missing information to the LF_BUILDINFO which allows for rebuilding an .OBJ without any external dependency but the .OBJ itself (other than the compiler executable).

Some tools need this information to reproduce a build without any knowledge of the build system. The LF_BUILDINFO therefore stores a full path to the compiler, the PWD (which is the CWD at program startup), a relative or absolute path to the TU, and the full CC1 command line. The command line needs to be freestanding (not depend on any environment variable). In the same way, MSVC doesn't store the provided command-line, but an expanded version (somehow their equivalent of CC1) which is also freestanding.

For more information see PR36198 and D43002.

Differential Revision: https://reviews.llvm.org/D80833
2020-07-10 13:59:28 -04:00
Victor Huang 118366dcb6 [PowerPC] Implement R_PPC64_REL24_NOTOC calls, callee also has no TOC
The PC Relative code allows for calls that are marked with the relocation
R_PPC64_REL24_NOTOC. This indicates that the caller does not have a valid TOC
pointer in R2 and does not require R2 to be restored after the call.

This patch is added to support local calls to callees tha also do not have a TOC.

Reviewed By: sfertile, MaskRay, stefanp

Differential Revision: https://reviews.llvm.org/D82816
2020-07-10 07:23:32 -05:00
Eric Christopher 98eec7700c Temporarily Revert "Fix [-Werror,-Wsign-compare] warnings arising from subsection symbols patch."
as it's causing build errors with another clang so I'll need to approach
this differently.

This reverts commit c282708316.
2020-07-09 13:46:59 -07:00
Eric Christopher c282708316 Fix [-Werror,-Wsign-compare] warnings arising from subsection symbols patch. 2020-07-09 11:14:00 -07:00
Stefan Pintilie 6f7727db47 [PowerPC] Fix test case from beb52b12cb
Forgot to add the REQUIRES ppc line to the test.
2020-07-09 12:39:24 -05:00
Stefan Pintilie beb52b12cb [PowerPC] Support PCRelative Callees for R_PPC64_REL24 Relocation
The R_PPC64_REL24 is used in function calls when the caller requires a
valid TOC pointer. If the callee shares the same TOC or does not clobber
the TOC pointer then a direct call can be made. If the callee does not
share the TOC a thunk must be added to save the TOC pointer for the caller.

Up until PC Relative was introduced all local calls on medium and large code
models were assumed to share a TOC. This is no longer the case because
if the caller requires a TOC and the callee is PC Relative then the callee
can clobber the TOC even if it is in the same DSO.

This patch is to add support for a TOC caller calling a PC Relative callee that
clobbers the TOC.

Reviewed By: sfertile, MaskRay

Differential Revision: https://reviews.llvm.org/D82950
2020-07-09 09:50:19 -05:00
Igor Kudrin ca4d8da0c3 [DebugInfo] Add more checks to parsing .debug_pub* sections.
The patch adds checking for various potential issues in parsing name
lookup tables and reporting them as recoverable errors, similarly as we
do for other tables.

Differential Revision: https://reviews.llvm.org/D83050
2020-07-09 19:15:31 +07:00
Igor Kudrin 68f5a8b204 [DebugInfo] Do not hang when parsing a malformed .debug_pub* section.
The parsing method did not check reading errors and might easily fall
into an infinite loop on an invalid input because of that.

Differential Revision: https://reviews.llvm.org/D83049
2020-07-09 19:15:11 +07:00
Fangrui Song f86d96a964 [ELF] Enforce double-dash form for --warn-backrefs-exclude
This is an LLD-specific option. We have enforced double-dash forms for
other options (reduce collision with short options) but missed this one.
2020-07-08 11:45:01 -07:00
Fangrui Song 169ec2d6b0 [ELF] Rename canRelax to toExecRelax. NFC
In the absence of TLS relaxation (rewrite of code sequences),
there is still an applicable optimization:

[gd]: General Dynamic: resolve DTPMOD to 1 and/or resolve DTPOFF statically

All the other relaxations are only performed when transiting to
executable (`!config->shared`).
Since [gd] is handled differently, we can fold `!config->shared` into canRelax
and simplify its use sites. Rename the variable to reflect to new semantics.

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D83243
2020-07-08 10:27:31 -07:00
Fangrui Song 4ce56b8122 [ELF] Add -z dead-reloc-in-nonalloc=<section_glob>=<value>
... to customize the tombstone value we use for an absolute relocation
referencing a discarded symbol. This can be used as a workaround when
some debug processing tool has trouble with current -1 tombstone value
(https://bugs.chromium.org/p/chromium/issues/detail?id=1102223#c11 )

For example, to get the current built-in rules (not considering the .debug_line special case for ICF):

```
-z dead-reloc-in-nonalloc='.debug_*=0xffffffffffffffff'
-z dead-reloc-in-nonalloc=.debug_loc=0xfffffffffffffffe
-z dead-reloc-in-nonalloc=.debug_ranges=0xfffffffffffffffe
```

To get GNU ld (as of binutils 2.35)'s behavior:

```
-z dead-reloc-in-nonalloc='*=0'
-z dead-reloc-in-nonalloc=.debug_ranges=1
```

This option has other use cases. For example, if we want to check
whether a non-SHF_ALLOC section has dead relocations.
With this patch, we can run a regular LLD and run another with a special
-z dead-reloc-in-nonalloc=, then compare their output.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D83264
2020-07-08 10:15:16 -07:00
Fangrui Song 09b81a72ac [ELF] Ignore --no-relax for RISC-V
In GNU ld, --no-relax can disable x86-64 GOTPCRELX relaxation.
It is not useful, so we don't implement it.

For RISC-V, --no-relax disables linker relaxations which have larger
impact.
Linux kernel specifies --no-relax when CONFIG_DYNAMIC_FTRACE is specified
(since http://git.kernel.org/linus/a1d2a6b4cee858a2f27eebce731fbf1dfd72cb4e ).
LLD has not implemented the relaxations, so this option is a no-op.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D81359
2020-07-07 09:48:13 -07:00
Wouter van Oortmerssen 4d135b0446 [WebAssembly] 64-bit memory limits 2020-07-06 12:40:45 -07:00
William S. Moses dc6b3f03a8 [ELF] Drop an unneeded reference to `symtab` from SymbolTable::addSymbol
The Symbol Table in LLD references the global object to add a symbol rather than adding it to itself.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D83184
2020-07-06 12:05:54 -07:00
Fangrui Song c1a5f73a4a [ELF][ARM] Represent R_ARM_LDO32 as R_DTPREL instead of R_ABS
Follow-up to D82899. Note, we need to disable R_DTPREL relaxation
because ARM psABI does not define TLS relaxation.

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D83138
2020-07-06 09:47:53 -07:00
Fangrui Song 6fa1343bb3 [ELF] Resolve R_DTPREL in .debug_* referencing discarded symbols to -1
The location of a TLS variable is encoded as a DW_OP_const4u/DW_OP_const8u
followed by a DW_OP_push_tls_address (or DW_OP_GNU_push_tls_address https://sourceware.org/bugzilla/show_bug.cgi?id=11616 ).

This change follows up to D81784 and makes relocations types generalized as
R_DTPREL (e.g. R_X86_64_DTPOFF{32,64}, R_PPC64_DTPREL64) use -1 as the
tombstone value as well. This works for both TLS Variant I and Variant II
architectures.

* arm: .long tls(tlsldo)   # not working currently (R_ARM_TLS_LDO32 is R_ABS)
* mips64: .dtpreldword tls+32768
* ppc64: .quad tls@DTPREL+0x8000
* riscv: neither GCC nor clang has implemented DW_AT_location. It is likely .long/.quad tls@dtprel+0x800
* x86-32: .long tls@DTPOFF
* x86-64: .long tls@DTPOFF; .quad tls@DTPOFF

tls has a non-negative st_value, so such relocations (st_value+addend)
never resolve to -1 in a normal (not discarded) case.

```
// clang -fuse-ld=lld -g -ffunction-sections a.c -Wl,--gc-sections
// foo and tls will be discarded by --gc-sections.
// DW_AT_location [DW_FORM_exprloc] (DW_OP_const8u 0xffffffffffffffff, DW_OP_GNU_push_tls_address)
thread_local int tls;
int foo() { return ++tls; }
int main() {}
```

Also, drop logic added in D26201 intended to address PR30793. It added a test
(gc-debuginfo-tls.s) using a non-SHF_ALLOC section and a local symbol, which
does not reflect the intended scenario: a relocation in a SHF_ALLOC section
referencing a discarded non-local symbol. For such a non .debug_* section, just
emit an error.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82899
2020-07-03 09:50:30 -07:00
Ben Dunbobbin f42f599d37 [LLD][ELF][Windows] Allow LLD to overwrite existing output files that are in use
On Windows co-operative programs can be expected to open LLD's
output in FILE_SHARE_DELETE mode. This allows us to delete the
file (by moving it to a temporary filename and then deleting
it) so that we can link another output file that overwrites
the existing file, even if the current file is in use.

A similar strategy is documented here:
https://boostgsoc13.github.io/boost.afio/doc/html/afio/FAQ/deleting_open_files.html

Differential Revision: https://reviews.llvm.org/D82567
2020-07-03 13:08:56 +01:00
Jez Ng 53eb7fda51 [lld-macho] Support binding dysyms to any section
Previously, we only supported binding dysyms to the GOT. This
diff adds support for binding them to any arbitrary section. C++
programs appear to use this, I believe for vtables and type_info.

This diff also makes our bind opcode encoding a bit smarter -- we now
encode just the differences between bindings, which will make things
more compact.

I was initially concerned about the performance overhead of iterating
over these relocations, but it turns out that the number of such
relocations is small. A quick analysis of my llvm-project build
directory showed that < 1.3% out of ~7M relocations are RELOC_UNSIGNED
bindings to symbols (including both dynamic and static symbols).

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D83103
2020-07-02 21:21:01 -07:00
Nico Weber 425fb21e03 ld64.lld: Make janky support for tbd files actually work sometimes
Also fix a bug in the test input that made the test miss this issue.
2020-07-02 15:31:21 -04:00
Nemanja Ivanovic 16989f6f5c [LLD] Add required dependency after shared libs break due to ba5087f130
The dependency on TextAPI was not added and is required for shared
libs builds.
2020-07-02 12:28:17 -05:00
James Henderson 5012ddd5af [ELF][test] Add some additional .eh_frame/.eh_frame_hdr testing
This patch adds a few extra cases to the existing testing for eh_frame
and eh_frame_hdr behaviour in LLD. They all come from a private
testsuite we are trying to migrate to lit.

Reviewed by: grimar, MaskRay

Differential Revision: https://reviews.llvm.org/D82852
2020-07-02 15:51:33 +01:00
Craig Topper 0aad82943a [X86] Enable multibyte NOPs in 64-bit mode for padding/alignment.
The default CPU used by llvm-mc doesn't have the NOPL feature, but
if we know we're compiling in 64-bit mode we should be able to
use nopl.
2020-07-01 23:59:01 -07:00
Nico Weber ba5087f130 ld64.lld: Add janky support for tbd files
With this, a simple hello world links against libSystem.tbd and the
old ld64.lld linker kind of works again with newer SDKs.

The motivation here is to have an arm64 cross linker that's good
enough to be able to run simple configure link checks on non-mac
systems for generating config.h files. Once -flavor darwinnew can
link arm64, we'll switch to that.
2020-07-02 00:05:52 -04:00
Nico Weber 7be1661fc6 lld/MachO: Remove a useless temporary 2020-07-02 00:05:43 -04:00
Jez Ng 7996a1ef70 [lld-macho] Make sure ZeroFill sections are at the end of their segments
Summary:
ld64 does this, and references an internal rdar:// number as an explanation. No
idea what that rdar issue is, but in practice, it seems that not putting a BSS
section at the end can cause subsequent sections in the same segment to be
overwritten with zeroes.

Reviewers: #lld-macho

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81888
2020-07-01 19:39:29 -07:00
Douglas Yung 6f13299c02 Revert "Update lto.ll test after 3367e9da enabled multibyte NOPs in 64-bit mode."
This reverts commit 79f6a814ab.

Didn't notice that someone had reverted the commit that caused the problem.
2020-07-01 16:34:57 -07:00
Douglas Yung 79f6a814ab Update lto.ll test after 3367e9da enabled multibyte NOPs in 64-bit mode.
This should fix the PS4 linux build bot:
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/70409
2020-07-01 16:25:25 -07:00
Fangrui Song e6ad78fe05 [ELF] Don't resolve a relocation in .debug_line referencing an ICF folded symbol to the tombstone value
After D81784, we resolve a relocation in .debug_* referencing an ICF folded
section symbol to a tombstone value.

Doing this for .debug_line has a problem (https://reviews.llvm.org/D81784#2116925 ):
.debug_line may describe folded lines as having addresses UINT64_MAX or
some wraparound small addresses.

```
int foo(int x) {
  return x; // line 2
}

int bar(int x) {
  return x; // line 6
}
```

```
Address            Line   Column File   ISA Discriminator Flags
------------------ ------ ------ ------ --- ------------- -------------
0x00000000002016c0      1      0      1   0             0  is_stmt
0x00000000002016c7      2      9      1   0             0  is_stmt
prologue_end
0x00000000002016ca      2      2      1   0             0
0x00000000002016cc      2      2      1   0             0  end_sequence
// UINT64_MAX and wraparound small addresses
0xffffffffffffffff      5      0      1   0             0  is_stmt
0x0000000000000006      6      9      1   0             0  is_stmt
prologue_end
0x0000000000000009      6      2      1   0             0
0x000000000000000b      6      2      1   0             0  end_sequence
0x00000000002016d0      9      0      1   0             0  is_stmt
0x00000000002016df     10      6      1   0             0  is_stmt prologue_end
0x00000000002016e6     11     11      1   0             0  is_stmt
...
```

These entries can confuse debuggers:

gdb before 2020-07-01 (binutils-gdb a8caed5d7faa639a1e6769eba551d15d8ddd9510 "Recognize -1 as a tombstone value in .debug_line")
(can't continue due to a breakpoint in an invalid region of memory):
```
Warning:
Cannot insert breakpoint 1.
Cannot access memory at address 0x6
```
lldb (breakpoint has no effect):
```
(lldb) b 6
Breakpoint 1: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
```

This patch special cases .debug_line to not use the tombstone value,
restoring the previous behavior: .debug_line will have entries with the
same addresses (ICF) but different line numbers. A breakpoint on line 2
or 6 will trigger on both functions.

Reviewed By: dblaikie, jhenderson

Differential Revision: https://reviews.llvm.org/D82828
2020-07-01 13:38:16 -07:00
Fangrui Song f5e0ae240f [ELF][test] Fix ppc32-gnu-ifunc.s and remove XFAIL: * 2020-06-30 22:05:51 -07:00
Sam Clegg 73e575a88e [lld][WebAssembly] Give better error message on bad archive member
Include the archive name as well as the member name when an error
is encountered parsing bitcode archives.

Differential Revision: https://reviews.llvm.org/D82884
2020-06-30 21:36:36 -07:00
Reid Kleckner b7402edce3 [PDB] Defer public serialization until PDB writing
This reduces peak memory on my test case from 1960.14MB to 1700.63MB
(-260MB, -13.2%) with no measurable impact on CPU time. I'm currently
working with a publics stream that is about 277MB. Before this change,
we would allocate 277MB of heap memory, serialize publics into them,
hold onto that heap memory, open the PDB, and commit into it.  After
this change, we defer the serialization until commit time.

In the last change I made to public writing, I re-sorted the list of
publics multiple times in place to avoid allocating new temporary data
structures. Deferring serialization until later requires that we don't
reorder the publics. Instead of sorting the publics, I partially
construct the hash table data structures, store a publics index in them,
and then sort the hash table data structures. Later, I replace the index
with the symbol record offset.

This change also addresses a FIXME and moves the list of global and
public records from GSIHashStreamBuilder to GSIStreamBuilder. Now that
publics aren't being serialized, it makes even less sense to store them
as a list of CVSymbol records. The hash table used to deduplicate
globals is moved as well, since that is specific to globals, and not
publics.

Reviewed By: aganea, hans

Differential Revision: https://reviews.llvm.org/D81296
2020-06-30 11:28:04 -07:00
Fangrui Song d94526bb5f [ELF] --warn-backrefs: check that D79300 fixed an issue due to `mb = {}`
D79300 forgot to change `getBuffer().empty()` in LazyObjFile::parse to
`fetched`. This caused incorrect iterating after the current LazyObjFile was
fetched. This issue is benign and can just cause loss of "undefined symbols"
and "backward reference" diagnostics.

Before D79300 `mb = {}` caused --warn-backrefs-exclude to be useless for
a fetched LazyObjFile.

Add two test cases.
2020-06-26 20:31:47 -07:00
Dan Gohman 46a3268312 [WebAssembly] Add warnings for -shared and -pie
The meaning of -shared and -pie are expected to be changed in the
future when Module Linking-style libraries are implemented. Begin
issuing warnings to give people a heads-up that they will be changing.

For compatibility with Emscripten, add a --experimental-pic flag which
disables these warnings.

Differential Revision: https://reviews.llvm.org/D81760
2020-06-25 15:55:46 -07:00
Wouter van Oortmerssen b9a539c010 [WebAssembly] Adding 64-bit versions of __stack_pointer and other globals
We have 6 globals, all of which except for __table_base are 64-bit under wasm64.

Differential Revision: https://reviews.llvm.org/D82130
2020-06-25 15:52:44 -07:00
Fangrui Song 40bc99538c [lld-macho] Remove using namespace llvm::MachO
llvm/include/llvm/TextAPI/MachO/ inappropriately uses the llvm::MachO namespace (this is for BinaryFormat and Object) and causes conflicts in some MSVC builds::

http://lab.llvm.org:8011/builders/sanitizer-windows/builds/65324/steps/stage%201%20build/logs/stdio

Removing `using namespace llvm::MachO` should decrease name collisions.
2020-06-24 13:46:13 -07:00
Fangrui Song 18db086dca [lld-macho] Use namespace qualifiers (macho::) instead of `namespace lld { namespace macho {`
See https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions
Similar to D79982.

Reviewed By: #lld-macho, int3

Differential Revision: https://reviews.llvm.org/D82432
2020-06-24 12:34:06 -07:00
Fangrui Song 4542c18ef2 [ELF] -r: don't parse @ (symbol versioning) for .symver inline asm in bitcode
Fixes PR46420
Similar to D43307 for non-LTO.

Module-level inline assembly can use .symver to create a symbol with `@` in the name.
For relocatable output, @ should be retained in the symbol name. `@ver` should
not be parsed and dropped.

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D82433
2020-06-24 08:22:22 -07:00
Stefan Pintilie 8131ef5d63 [LLD][PowerPC] Add support for R_PPC64_GOT_PCREL34
Add support for the 34bit relocation R_PPC64_GOT_PCREL34 for
PC Relative in LLD.

Reviewers: sfertile, MaskRay

Differential Revision: https://reviews.llvm.org/D81948
2020-06-24 07:40:35 -05:00
Leonard Chan 723b5a1785 [lld][ELF][AArch64] Handle R_AARCH64_PLT32 relocation
This is the followup to D77647 which implements handling for the new
R_AARCH64_PLT32 relocation type in lld. This relocation would benefit the
PIC-friendly vtables feature described in D72959.

Differential Revision: https://reviews.llvm.org/D81184
2020-06-23 16:10:07 -07:00
Petr Hosek fffd05d525 [ELF] Add -z start-stop-visibility= to set __start_/__stop_ symbol visibility
This matches the equivalent flag implemented in GNU linkers, see
https://sourceware.org/pipermail/binutils/2020-June/111685.html for
the associated discussion.

Differential Revision: https://reviews.llvm.org/D55682
2020-06-23 15:59:59 -07:00
Stefan Pintilie 3a55a2a97f [LLD][PowerPC] Add support for R_PPC64_PCREL34
Add support for the 34bit relocation R_PPC64_PCREL34 for PC Relative in LLD.
2020-06-23 14:59:19 -05:00
Fangrui Song e618ccbf43 [ELF] Resolve relocations in .debug_* referencing (discarded symbols or ICF folded section symbols) to tombstone values
See D59553, https://lists.llvm.org/pipermail/llvm-dev/2020-May/141885.html and
https://sourceware.org/pipermail/binutils/2020-May/111357.html for
extensive discussions on a tombstone value.
See http://www.dwarfstd.org/ShowIssue.php?issue=200609.1
(Reserve an address value for "not present") for a DWARF enhancement proposal.

We resolve such relocations to a tombstone value to indicate that the address is invalid.
This solves several problems (the normal behavior is to resolve the relocation to the addend):

* For an empty function in a collected section, a pair of (0,0) can
  terminate .debug_loc and .debug_ranges (as of binutils 2.34, GNU ld
  resolves such a relocation to 1 to avoid the .debug_ranges issue)
* If DW_AT_high_pc is sufficiently large, the address range can collide
  with a regular code range of low address (https://bugs.llvm.org/show_bug.cgi?id=41124 )
* If a text section is folded into another by ICF, we may leave entries
  in multiple CUs claiming ownership of the same range of code, which can
  confuse consumers.
* Debug information associated with COMDAT sections can have problems
  similar to ICF, but is more complex - thus not addressed by this patch.

For pre-DWARF-v5 .debug_loc and .debug_ranges, a pair of 0 can terminate
entries (invalidating subsequent ranges).
-1 is a reserved value with special meaning (base address selection entry) which can't be used either.
Use -2 instead.

For all other .debug_*, use UINT32_MAX for 32-bit targets and UINT64_MAX
for 64-bit targets. In the code, we intentionally use
`uint64_t tombstone = UINT64_MAX` for 32-bit targets as well: this matches
SignExtend64 as used in `relocateAlloc`. (Actually UINT32_MAX does not work for R_386_32)

Note 0, we only special case `target->symbolicRel` (R_X86_64_64, R_AARCH64_ABS64, R_PPC64_ADDR64), not
short-range absolute relocations (e.g. R_X86_64_32). Only forms like DW_FORM_addr need to be special cased.
They can hold an arbitrary address (must be 64-bit on a 64-bit target). (In theory,
producers can make use of small code model to emit 32-bit relocations. This doesn't seem to be leveraged.)

Note 1, we have to ignore the addend, because we don't want to resolve
DW_AT_low_pc (which may have a non-zero addend) to -1+addend (wrap
around to a low address):

  __attribute__((section(".text.x"))) void f1() { }
  __attribute__((section(".text.x"))) void f2() { } // DW_AT_low_pc has a non-zero addend

Note 2, if the prevailing copy does not have debugging information while
a non-prevailing copy has (partial debug build), we don't do extra work
to attach debugging information to the prevailing definition.  (clang
has a lot of debug info optimizations that are on-by-default that assume
the whole program is built with debug info).

  clang -c -ffunction-sections a.cc    # prevailing copy has no debug info
  clang -c -ffunction-sections -g b.cc

Reviewed By: dblaikie, avl, jhenderson

Differential Revision: https://reviews.llvm.org/D81784
2020-06-23 11:48:46 -07:00
Sam Clegg 79aad89d8d [WebAssembly] Add support for externalref to MC and wasm-ld
This allows code for handling externref values to be processed by the
assembler and linker.

Differential Revision: https://reviews.llvm.org/D81977
2020-06-22 15:57:24 -07:00
Peter Collingbourne bd7defeb94 llvm-nm: Implement --special-syms.
Differential Revision: https://reviews.llvm.org/D82251
2020-06-22 13:05:47 -07:00
Jez Ng 3646ee503d [lld-macho] Refactor segment/section creation, sorting, and merging
Summary:
There were a few issues with the previous setup:

1. The section sorting comparator used a declarative map of section names to
  determine the correct order, but it turns out we need to match on more than
  just names -- in particular, an upcoming diff will sort based on whether the
  S_ZERO_FILL flag is set. This diff changes the sorter to a more imperative but
  flexible form.

2. We were sorting OutputSections stored in a MapVector, which left the
  MapVector in an inconsistent state -- the wrong keys map to the wrong values!
  In practice, we weren't doing key lookups (only container iteration) after the
  sort, so this was fine, but it was still a dubious state of affairs. This diff
  copies the OutputSections to a vector before sorting them.

3. We were adding unneeded OutputSections to OutputSegments and then filtering
  them out later, which meant that we had to remember whether an OutputSegment
  was in a pre- or post-filtered state. This diff only adds the sections to the
  segments if they are needed.

In addition to those major changes, two minor ones worth noting:

1. I renamed all OutputSection variable names to `osec`, to parallel `isec`.
  Previously we were using some inconsistent combination of `osec`, `os`, and
  `section`.

2. I added a check (and a test) for InputSections with names that clashed with
  those of our synthetic OutputSections.

Reviewers: #lld-macho

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81887
2020-06-21 17:13:59 -07:00
Eric Christopher 058ec20677 [lld] As part of using inclusive language within the llvm
project, migrate away from the use of blacklist and whitelist.
2020-06-19 21:50:14 -07:00
Fangrui Song 8ffb2097cc [ELF] Refine LMA offset propagation rule in D76995
If neither AT(lma) nor AT>lma_region is specified,
D76995 keeps `lmaOffset` (LMA - VMA) if the previous section is in the
default LMA region.

This patch additionally checks that the two sections are in the same
memory region.

Add a test case derived from https://bugs.llvm.org/show_bug.cgi?id=45313

  .mdata : AT(0xfb01000) { *(.data); } > TCM
  // It is odd to make .bss inherit lmaOffset, because the two sections
  // are in different memory regions.
  .bss : { *(.bss) } > DDR

With this patch, section VMA/LMA match GNU ld. Note, GNU ld supports
out-of-order (w.r.t sh_offset) sections and places .text and .bss in the
same PT_LOAD. We don't have that behavior.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D81986
2020-06-19 09:11:33 -07:00
Fangrui Song c4d13f72a6 [ELF] Refactor ObjFile<ELFT>::initializeSymbols to enforce the invariant: InputFile::symbols has non null entry
Fixes PR46348.

ObjFile<ELFT>::initializeSymbols contains two symbol iteration loops:

```
for each symbol
  if non-inheriting && non-local
    fill in this->symbols[i]

for each symbol
  if local
    fill in this->symbols[i]
  else
    symbol resolution
```

Symbol resolution can trigger a duplicate symbol error which will call
InputSectionBase::getObjMsg to iterate over InputFile::symbols.  If a
non-local symbol appears after the non-local symbol being resolved
(violating ELF spec), its `this->symbols[i]` entry has not been filled
in, InputSectionBase::getObjMsg will crash due to
`dyn_cast<Defined>(nullptr)`.

To fix the bug, reorganize the two loops to ensure this->symbols is
complete before symbol resolution. This enforces the invariant:
InputFile::symbols has none null entry when InputFile::getSymbols() is called.

```
for each symbol
  if non-inheriting
    fill in this->symbols[i]

for each symbol starting from firstGlobal
  if non-local
    symbol resolution
```

Additionally, move the (non-local symbol in local part of .symtab)
diagnostic from Writer<ELFT>::copyLocalSymbols() to initializeSymbols().

Reviewed By: grimar, jhenderson

Differential Revision: https://reviews.llvm.org/D81988
2020-06-19 09:05:37 -07:00
Alexandre Ganea 2ae0df5be7 [CodeView] Revert 8374bf4363 and 403f953792
This reverts:
8374bf4363 [CodeView] Fix generated command-line expansion in LF_BUILDINFO. Fix the 'pdb' entry which was previously a null reference, now an empty string.
403f953792 [CodeView] Add full repro to LF_BUILDINFO record

This is causing the lld/test/COFF/pdb-relative-source-lines.test to fail: http://lab.llvm.org:8011/builders/lld-x86_64-win/builds/1096/steps/test-check-all/logs/FAIL%3A%20lld%3A%3Apdb-relative-source-lines.test
And clang/test/CodeGen/debug-info-codeview-buildinfo.c fails as well: http://lab.llvm.org:8011/builders/clang-s390x-linux/builds/33346/steps/ninja%20check%201/logs/FAIL%3A%20Clang%3A%3Adebug-info-codeview-buildinfo.c
2020-06-18 16:18:46 -04:00
Sam Clegg 21e14cce23 [lld][WebAssembly] Allow ctors functions that return values
Some projects use the constructor attribute on functions that also
return values.  In this case we just ignore them.

The error was reported in the libgpg-error project that marks
gpg_err_init with the `__constructor__` attribute.

Differential Revision: https://reviews.llvm.org/D81962
2020-06-18 13:11:40 -07:00
Nico Weber 9bcd59fdef fix a typo to cycle bots 2020-06-18 11:31:47 -04:00
Alexandre Ganea 403f953792 [CodeView] Add full repro to LF_BUILDINFO record
This patch adds some missing information to the LF_BUILDINFO which allows for rebuilding an .OBJ without any external dependency but the .OBJ itself (other than the compiler executable).

Some tools need this information to reproduce a build without any knowledge of the build system. The LF_BUILDINFO therefore stores a full path to the compiler, the PWD (which is the CWD at program startup), a relative or absolute path to the TU, and the full CC1 command line. The command line needs to be freestanding (not depend on any environment variable). In the same way, MSVC doesn't store the provided command-line, but an expanded version (somehow their equivalent of CC1) which is also freestanding.

For more information see PR36198 and D43002.

Differential Revision: https://reviews.llvm.org/D80833
2020-06-18 09:17:15 -04:00
Greg McGary f7453a97ce [lld-macho] Add missing search-paths-darwin.test
Summary: Forgot to `git add` it when patching D80677
2020-06-17 20:51:26 -07:00
Greg McGary d50f44a2f7 [lld-macho] Handle framework search path, alongside library search path
Summary:
Add front-end support for `lld::macho::Configuration::frameworkSearchPath`.

Depends on D80582.

Reviewers: ruiu, pcc, MaskRay, smeenai, int3, Ktwu, alexshap, christylee

Reviewed By: int3

Subscribers: ormris, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80677
2020-06-17 20:41:28 -07:00
Jez Ng 525c7d8cda [lld-macho] Handle alignment correctly when merging InputSections
Summary:
Previously, we weren't updating isecAddr when aligning InputSections,
resulting in truncated sections under the right conditions.

Reviewers: #lld-macho, compnerd

Reviewed By: #lld-macho, compnerd

Subscribers: smeenai, compnerd, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81298
2020-06-17 20:41:28 -07:00
Jez Ng 74871cdad7 [lld-macho] Ensure __bss sections we output have file offset of zero
Summary:
llvm-mc emits `__bss` sections with an offset of zero, but we weren't expecting
that in our input, so we were copying non-zero data from the start of the file and
putting it in `__bss`, with obviously undesirable runtime results. (It appears that
the kernel will copy those nonzero bytes as long as the offset is nonzero, regardless
of whether S_ZERO_FILL is set.)

I debated on whether to make a special ZeroFillSection -- separate from a
regular InputSection -- but it seemed like too much work for now. But I'm happy
to refactor if anyone feels strongly about having it as a separate class.

Depends on D80857.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80859
2020-06-17 20:41:28 -07:00
Jez Ng a12e7d406d [lld-macho] Handle GOT relocations of non-dylib symbols
Summary:
Turns out this case is actually really common -- it happens whenever there's
a reference to an `extern` variable that ends up statically linked.

Depends on D80856.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80857
2020-06-17 20:41:28 -07:00
Jez Ng c3d98ea89f [lld-macho] Support X86_64_RELOC_GOT
Summary:
As far as I can tell, it's identical to _GOT_LOAD. llvm-mc has the following
comment explaining why _GOT exists:

```
// x86_64 distinguishes movq foo@GOTPCREL so that the linker can
// rewrite the movq to an leaq at link time if the symbol ends up in
// the same linkage unit.
```

Depends on D80855.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: MaskRay, smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80856
2020-06-17 20:41:28 -07:00
Jez Ng fcde378dcb [lld-macho] Support non-pcrel section relocs
Summary: Depends on D80854.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80855
2020-06-17 20:41:28 -07:00
Jez Ng 2f4cfa3c7a [lld-macho] Avoid explicit -arch in tests by defaulting to x86-64
Summary:
As mentioned in https://reviews.llvm.org/D81326#2093931, I'm not sure it
makes sense to use the default target triple to determine -arch.
Long-term we should probably detect it from the input object files, but
in the meantime it would be nice not to have to add it to all our tests
by using a convenient default.

Reviewers: #lld-macho

Subscribers: arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81983
2020-06-17 20:41:27 -07:00
Fangrui Song 49279ca160 [ELF] Improve --export-dynamic-symbol performance by checking whether wildcard is really used
A hasWildcard pattern iterates over symVector, which can be slow when there
are many --export-dynamic-symbol. In optimistic cases, most patterns don't use
a wildcard character. hasWildcard: false can avoid a symbol table iteration.

While here, add two tests using `[` and `?`, respectively.
2020-06-17 17:12:10 -07:00
Jez Ng a2d096df26 [lld-macho] Use uint64_t for getSize() instead of size_t
Summary:
So things work on 32-bit machines. (@vzakhari reported the
breakage starting from D80177).

Reviewers: #lld-macho, vzakhari

Subscribers: llvm-commits, vzakhari

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81982
2020-06-16 18:42:45 -07:00
Fangrui Song a649802c59 [lld-macho][test] Fix dylink-lazy.s after 0a5228a02c 2020-06-16 17:57:59 -07:00
Saleem Abdulrasool 0a5228a02c test: add library search path to repair the test
Update the test to no longer rely on the testing workaround in the
linker and use the external libSystem stub.
2020-06-16 17:06:21 -07:00
Saleem Abdulrasool 73312976ad lld: remove old test support path
This removes the stub library that lld injected to satisfy the
dependency on the libSystem.  Now with TBD support, we can provide the
stub library to permit the tests to function properly as they would on a
real system.

Reviewed By: smeenai
Differential Revision: https://reviews.llvm.org/D81418
2020-06-16 15:57:58 -07:00
Greg McGary 7df80e3f23 [lld-macho] Specify the complete set of command-line options for ld64
This is a complete Options.td compiled from ld(1) dated 2018-03-07 and
cross checked with ld64 source code version 512.4 dated 2018-03-18.

This is the first in a series of diffs for argument handling. Follow-ups
will include switch cases for all the new instances of `OPT_foo`, and
parsing/validation of arguments attached to options, e.g., more code
akin to `OPT_platform_version` and associated `parsePlatformVersion()`.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80582
2020-06-15 12:50:20 -07:00
Shoaib Meenai 72e096fd1e [MachO] Fix typo in comment
The case the calculation works for is when r_length = 2.
2020-06-15 12:26:55 -07:00
Wouter van Oortmerssen 3b29376e3f [WebAssembly] Adding 64-bit version of R_WASM_MEMORY_ADDR_* relocs
This adds 4 new reloc types.

A lot of code that previously assumed any memory or offset values could be contained in a uint32_t (and often truncated results from functions returning 64-bit values) have been upgraded to uint64_t. This is not comprehensive: it is only the values that come in contact with the new relocation values and their dependents.

A new tablegen mapping was added to automatically upgrade loads/stores in the assembler, which otherwise has no way to select for these instructions (since they are indentical other than for the offset immediate). It follows a similar technique to https://reviews.llvm.org/D53307

Differential Revision: https://reviews.llvm.org/D81704
2020-06-15 10:07:42 -07:00
Wouter van Oortmerssen d9e0bbd17b [WebAssembly] Adding 64-bit versions of all load & store ops.
Context: https://github.com/WebAssembly/memory64/blob/master/proposals/memory64/Overview.md
This is just a first step, adding the new instruction variants while keeping the existing 32-bit functionality working.
Some of the basic load/store tests have new wasm64 versions that show that the basics of the target are working.
Further features need implementation, but these will be added in followups to keep things reviewable.

Differential Revision: https://reviews.llvm.org/D80769
2020-06-15 08:31:56 -07:00
Kirill Bobyrev 9baba7cf66
Revert "[lld-macho] No need to explicitly specify -arch in tests"
This reverts commit 51c5baacf3 and also
337fb8c767 - "[lld-macho] Set REQUIRES:
x86 on more tests".

These patches cause test crashes:

http://lab.llvm.org:8011/builders/ppc64le-lld-multistage-test/builds/10054
2020-06-15 12:27:30 +02:00
Jez Ng 337fb8c767 [lld-macho] Set REQUIRES: x86 on more tests
Summary: Fixes the build break caused by D81802.
2020-06-14 19:05:12 -07:00
Jez Ng 53c796b948 [lld-macho] Properly handle & validate relocation r_length
Summary:
We should be reading / writing our addends / relocated addresses based on
r_length, and not just based on the type of the relocation. But since only
some r_length values are valid for a given reloc type, I've also added some
validation.

ld64 has code to allow for r_length = 0 in X86_64_RELOC_BRANCH relocs, but I'm
not sure how to create such a relocation...

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80854
2020-06-14 16:35:23 -07:00
Jez Ng 51c5baacf3 [lld-macho] No need to explicitly specify -arch in tests
Summary: After {D81326} landed, some tests started failing if they did
not have `-arch` specified. I think one of the reasons happened was due
to the fact that we were taking a reference to a temporary value that
was freed too early. Fixing that got the error to go away on my local
Linux machine.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D81802
2020-06-14 16:35:21 -07:00
Pavel Labath fccaa89e23 [llvm/DWARFDebugLine] Fix a typo in one warning message 2020-06-11 13:04:52 +02:00
Hongtao Yu 2638aafe12 [LLD][ThinLTO] Add --thinlto-single-module to allow compiling partial modules.
This change introduces an LLD switch --thinlto-single-module to allow compiling only a part of the input modules. This is specifically enables:

  1. Fast investigating/debugging modules of interest without spending time on compiling unrelated modules.
  2. Compiler debug dump with -mllvm -debug-only= for specific modules.

It will be useful for large applications which has 1K+ input modules for thinLTO.

The switch can be combined with `--lto-obj-path=` or `--lto-emit-asm` to obtain intermediate object files or assembly files. So far the module name matching is implemented as a fuzzy name lookup where the modules with name containing the switch value are compiled.

E.g,
Command:
     ld.lld main.o thin.a --thinlto-single-module=thin.a --lto-obj-path=single.o
log:
     [ThinLTO] Selecting thin.a(thin1.o at 168) to compile
     [ThinLTO] Selecting thin.a(thin2.o at 228) to compile
Command:
     ld.lld main.o thin.a --thinlto-single-module=thin1.o --lto-obj-path=single.o
log:
     [ThinLTO] Selecting thin.a(thin1.o at 168) to compile

Differential Revision: https://reviews.llvm.org/D80406
2020-06-10 15:32:30 -07:00
Pavel Labath de522035b9 [lld] Fix test/ELF/undef.s for 6f55b5a1
There's been a slight change in wording for the warning message about
debug_line problems.
2020-06-10 16:36:10 +02:00
gbreynoo be44b79257 [lld][test] Expand testing for dynamic-list and export-dynamic
- Expanded testing for --dynamic-list and --export-dynamic
- Fixed invalid-dynamic-list.test

Differential Revision: https://reviews.llvm.org/D80311
2020-06-10 14:06:30 +01:00
Fangrui Song b114e134bd [ELF] Fix --thinlto-index-only regression after D79300
After D79300, we don't rewrite InputFile::mb to an empty buffer.
In thinLTOCreateEmptyIndexFiles(), we should check LazyObjFile::fetched
as well as checking whether mb is a bitcode, otherwise we would overwrite (path + .thinlto.bc) with an empty index.
2020-06-09 23:10:30 -07:00
Fangrui Song ba890da287 [ELF] Demote lazy symbols relative to a discarded section to Undefined
Fixes PR45594.

In `ObjFile<ELFT>::initializeSymbols()`, for a defined symbol relative to
a discarded section (due to section group rules), it may have been
inserted as a lazy symbol. We need to demote it to an Undefined to
enable the `discarded section` error happened in a later pass.

Add `LazyObjFile::fetched` (if true) and `ArchiveFile::parsed` (if
false) to represent that there is an ongoing lazy symbol fetch and we
should replace the current lazy symbol with an Undefined, instead of
calling `Symbol::resolve` (`Symbol::resolve` should be called if the lazy
symbol was added by an unrelated archive/lazy object).

As a side result, one small issue in start-lib-comdat.s is now fixed.
The hack motivating D51892 will be unsupported: if
`.gnu.linkonce.t.__i686.get_pc_thunk.bx` in an archive is referenced
by another section, this will likely be errored unless the function is
also defined in a regular object file.
(Bringing back rL330869 would error `undefined symbol` instead of the
more relevant `discarded section`.)

Note, glibc i386's crti.o still works (PR31215), because
`.gnu.linkonce.t.__x86.get_pc_thunk.bx` is in crti.o (one of the first
regular object files in a linker command line).

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D79300
2020-06-09 11:27:34 -07:00
Saleem Abdulrasool 5fda192fed test: repair lld builder
Add the explicit `-arch x86_64` to the invocation.
2020-06-08 20:13:54 -07:00
Saleem Abdulrasool 6fe27b5fed lld: initial pass at supporting TBD
Add support to lld to use Text Based API stubs for linking.  This is
support is incomplete not filtering out platforms.  It also does not
account for architecture specific API handling and potentially does not
correctly handle trees of re-exports with inlined libraries being
treated as direct children of the top level library.
2020-06-08 18:15:40 -07:00
Michael Liao 43793b89a0 [lld] Fix shared library build by adding the missing dependency. 2020-06-08 16:12:58 -04:00
Saleem Abdulrasool fcdf7578aa lld: improve the `-arch` handling for MachO
Use the default target triple configured by the user to determine the
default architecture for `ld64.lld`.  Stash the architecture in the
configuration as when linking against TBDs, we will need to filter out
the symbols based upon the architecture.  Treat the Haswell slice as it
is equivalent to `x86_64` but with the extra Haswell extensions (e.g.
AVX2, FMA3, BMI1, etc).  This will make it easier to add new
architectures in the future.

This change also changes the failure mode where an invalid `-arch`
parameter will result in the linker exiting without further processing.
2020-06-08 11:04:19 -07:00
Fangrui Song ac6abc99e2 [ELF] Don't cause assertion failure if --dynamic-list or --version-script takes an empty file
Fixes PR46184
Report line 1 of the last memory buffer.
2020-06-05 15:59:54 -07:00
Saleem Abdulrasool e78431354b lld: use modern library search ordering
This merges the static and shared library and behaves as if
`-search_paths_first` was specified which is also the default behaviour
on ld64 (and now lld). Unify the paths, and use `llvm::sys::path` to
deal with the path to be truly agnostic to the host.
2020-06-05 12:12:26 -07:00
Fangrui Song 7bee6e30fe [ELF] Handle -u before input files
If both a.a and b.so define foo

```
ld.bfd -u foo a.a b.so  # foo is defined
ld.bfd a.a b.so -u foo  # foo is defined
ld.bfd -u foo b.so a.a  # foo is undefined (provided at runtime by b.so)
ld.bfd b.so a.a -u foo  # foo is undefined (provided at runtime by b.so)
```

In all cases we make foo undefined in the output.  I tend to think the
GNU ld behavior makes more sense.

* In their model, they have to treat -u as a fake object file with an
  undefined symbol before all input files, otherwise the first archive would not be fetched.
* Following their behavior allows us to drop a --warn-backrefs special case.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D81052
2020-06-05 08:44:38 -07:00
Saleem Abdulrasool 116e38fd8b lld: add basic static library search
This is a very basic static library search addition. This is the pre-Xcode4
behaviour of searching all paths for the shared version before searching for
the static version of the library. This behaviour is supposed to be inverted
with `-search_paths_first` being the default. This adds the library search
with the intention of providing the setup to merge the paths into one path
and making it controllable by `OPT_search_paths_first`.
2020-06-03 23:32:05 +00:00
Saleem Abdulrasool 9cb2badc59 lld: use `std::make_unique` (NFC)
The LLVM code base already uses C++14, use std::make_unique
to avoid the explicit constructor invocation via new and to avoid
spelling out the type twice.
2020-06-03 19:23:13 +00:00
Saleem Abdulrasool 37d93b528c lld: ignore the `-search_paths_first` option on MachO
ld64 provides the `-search_path_firsts` which will search each path in
the library search path order for both `lib[name].dylib`, `lib[name].a`
before moving on (searching all paths for the dylib and then falling
back to the static library if a shared library was not found).

This option has been the default for a long time, but the command line
flag still exists.  Ignore it for compatibility.
2020-06-03 15:36:35 +00:00
Fangrui Song 3eb4bf13ba [ELF] Append " [--no-allow-shlib-undefined]" to the corresponding diagnostics
--no-allow-shlib-undefined (enabled by default when linking an
executable) rejects unresolved references in shared objects.

Users may be confused by the common diagnostics of unresolved symbols in
object files (LLD: "undefined symbol: foo"; GNU ld/gold: "undefined reference to")

Learn from GCC/clang " [-Wfoo]": append the option name to the
diagnostics. Users can find relevant information by searching
"--no-allow-shlib-undefined".  It should also be obvious to them that
the positive form --allow-shlib-undefined can suppress the error.

Also downgrade the error to a warning if --noinhibit-exec is used (compatible
with GNU ld and gold).

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D81028
2020-06-03 07:59:37 -07:00
Jez Ng d767de44bf [lld-macho] Fix PAGEZERO=4GB errors on Windows by ensuring enum is uint64_t
It appears that MSVC doesn't resize the enum properly to fit the
constants.
2020-06-02 15:24:31 -07:00
Reid Kleckner 3eb16fe4e9 [LLD] Have only one SpecificAllocator per type
Previously, the SpecificAllocator was a static local in the `make<T>`
function template. Using static locals is nice because they are only
constructed and registered if they are accessed. However, if there are
multiple calls to make<> with different constructor parameters, we would
get multiple static local variable instances. This is undesirable and
leads to extra memory allocations. I noticed there were two sources of
DefinedRegular allocations while checking heap profiles.
2020-06-02 14:09:09 -07:00
Jez Ng 1e1a3f67ee [lld-macho] Ensure reads from nlist_64 structs are aligned when necessary
My test refactoring in D80217 seems to have caused yaml2obj to emit
unaligned nlist_64 structs, causing ASAN'd lld to be unhappy. I don't
think this is an issue with yaml2obj though -- llvm-mc also seems to
emit unaligned nlist_64s. This diff makes lld able to safely do aligned
reads under ASAN builds while hopefully creating no overhead for regular
builds on architectures that support unaligned reads.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D80414
2020-06-02 13:19:38 -07:00
Jez Ng f04d1c3b90 [lld-macho] Move all tests for erroneous inputs under invalid/
For consistency.

The no-id-dylib test was originally referencing the Inputs/ folder via a
relative path. Instead of updating that path, I decided to make the test
self-contained.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D80217
2020-06-02 13:19:38 -07:00
Jez Ng a04c133564 [lld-macho] Set __PAGEZERO size to 4GB
That's what ld64 uses for 64-bit targets. I figured it's best to make
this change sooner rather than later since a bunch of our tests are
relying on hardcoded addresses that depend on this value.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80177
2020-06-02 13:19:38 -07:00
Jez Ng df2a5778c3 [lld-macho] Error on encountering undefined symbols
... instead of silently emitting a reference to the zero address.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80169
2020-06-02 13:19:38 -07:00
Jez Ng 6f6d91867d [lld-macho] Add some relocation validation logic
I considered making a `Target::validate()` method, but I wasn't sure how
I felt about the overhead of doing yet another switch-dispatch on the
relocation type, so I put the validation in `relocateOne` instead...
might be a bit of a micro-optimization, but `relocateOne` does assume
certain things about the relocations it gets, and this error handling
makes that explicit, so it's not a totally unreasonable code
organization.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80049
2020-06-02 13:19:38 -07:00
Igor Kudrin 4933ab2ccb [DebugInfo] Report the format of call frame information entries [6/10]
Differential Revision: https://reviews.llvm.org/D80523
2020-06-02 17:55:30 +07:00
Simon Atanasyan b00f0d4238 [mips] Support 64-bit relative relocations
MIPS 64-bit ABI does not provide special PC-relative relocation like
R_MIPS_PC32 in 32-bit case. But we can use a "chain of relocation"
defined by N64 ABIs. In that case one relocation record might contain up
to three relocations which applied sequentially. Width of a final relocation
mask applied to the result of relocation depends on the last relocation
in the chain. In case of 64-bit PC-relative relocation we need the following
chain: `R_MIPS_PC32 | R_MIPS_64`. The first relocation calculates an
offset, but does not truncate the result. The second relocation just
apply calculated result as a 64-bit value.

The 64-bit PC-relative relocation might be useful in generation of
`.eh_frame` sections to escape passing `-Wl,-z,notext` flags to linker.

Differential Revision: https://reviews.llvm.org/D80390
2020-06-02 11:44:11 +03:00
Sriraman Tallam e0bca46b08 Options for Basic Block Sections, enabled in D68063 and D73674.
This patch adds clang options:
-fbasic-block-sections={all,<filename>,labels,none} and
-funique-basic-block-section-names.
LLVM Support for basic block sections is already enabled.

+ -fbasic-block-sections={all, <file>, labels, none} : Enables/Disables basic
block sections for all or a subset of basic blocks. "labels" only enables
basic block symbols.
+ -funique-basic-block-section-names: Enables unique section names for
basic block sections, disabled by default.

Differential Revision: https://reviews.llvm.org/D68049
2020-06-02 00:23:32 -07:00
Fangrui Song a6ae333a0c [ELF] --wrap: don't error `undefined reference to __real_foo` (--no-allow-shlib-undefined) if foo is a wrapped definition
This is a regression after D51283.

Also, export `foo` if `__real_foo` is referenced by a shared object.
2020-06-01 23:00:51 -07:00
Reid Kleckner 11d1aa0bcc [COFF] Free some memory used for chunks
First, do not reserve numSections in the Chunks array. In cases where
there are many non-prevailing sections, this will overallocate memory
which will not be used.

Second, free the memory for sparseChunks after initializeSymbols. After
that, it is never used.

This saves 50MB of 627MB for my use case without affecting performance.
2020-06-01 18:51:47 -07:00
Reid Kleckner 45fd3e4688 [PDB] Share code to relocate .debug$[SF] sections, NFC
Sink relocateDebugChunk near the only call site.
2020-06-01 13:16:57 -07:00
Reid Kleckner 8f0a660030 [PDB] Use inlinee file checksum offsets directly
The inlinees section contains references to the file checksum table. The
file checksum table in the PDB must have the same layout as the file
checksum table in the object file, so all the existing file id
references should stay valid.

Previously, we would do this:
  for all inlined functions:
    - lookup filename from checksum and string table
    - make that filename absolute
    - look up the new file id for that filename up in the new checksum
      table

This lead to pdbMakeAbsolute and remove_dots ending up in the hot path.
We should only need to absolutify the source path once, not once every
time we process an inline function from that source file.

This speeds up linking chrome PGO stage 1 net_unittests.exe from 9.203s
to 8.500s (-7.6%). Looking just at time to process symbol records, it
goes from ~2000ms to ~1300ms, which is consistent with the overall
speedup of about 700ms. This will be less noticeable in debug builds,
which have fewer inlined functions records.
2020-06-01 12:28:32 -07:00
Fangrui Song 751f18e7d4 [ELF] Refine --export-dynamic-symbol semantics to be compatible GNU ld 2.35
GNU ld from binutils 2.35 onwards will likely support
--export-dynamic-symbol but with different semantics.
https://sourceware.org/pipermail/binutils/2020-May/111302.html

Differences:

1. -export-dynamic-symbol is not supported
2. --export-dynamic-symbol takes a glob argument
3. --export-dynamic-symbol can suppress binding the references to the definition within the shared object if (-Bsymbolic or -Bsymbolic-functions)
4. --export-dynamic-symbol does not imply -u

I don't think the first three points can affect any user.
For the fourth point, Not implying -u can lead to some archive members unfetched.
Add -u foo to restore the previous behavior.

Exact semantics:

* -no-pie or -pie: matched non-local defined symbols will be added to the dynamic symbol table.
* -shared: matched non-local STV_DEFAULT symbols will not be bound to definitions within the shared object
  even if they would otherwise be due to -Bsymbolic, -Bsymbolic-functions, or --dynamic-list.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80487
2020-06-01 11:30:03 -07:00
Fangrui Song ee9a251caf [ELF] Set DF_1_PIE for -pie
DF_1_PIE originated from Solaris (https://docs.oracle.com/cd/E36784_01/html/E36857/chapter6-42444.html ).
GNU ld since
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=5fe2850dd96483f176858fd75c098313d5b20bc2
sets the flag on non-Solaris platforms.

It can help distinguish PIE from ET_DYN.
eu-classify from elfutils uses this to recognize PIE (https://sourceware.org/git/?p=elfutils.git;a=commit;h=3f489b5c7c78df6d52f8982f79c36e9a220e8951 )

glibc uses this flag to reject dlopen'ing a PIE (https://sourceware.org/bugzilla/show_bug.cgi?id=24323 )

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80872
2020-06-01 10:19:41 -07:00
Fangrui Song ce1fadca60 [ELF][docs] Update supported targets
PowerPC, PowerPC64 and x86-32 have production quality.
Mention Hexagon, RISC-V and SPARC V9.
2020-05-30 19:33:50 -07:00
Fangrui Song 881c5eef98 [ELF] Add -z rel and -z rela
LLD supports both REL and RELA for static relocations, but emits either
of REL and RELA for dynamic relocations. The relocation entry format is
specified by each psABI.

musl ld.so supports both REL and RELA. For such ld.so implementations,
REL (.rel.dyn .rel.plt) has size benefits even if the psABI chooses RELA:
sizeof(Elf64_Rel)=16 < sizeof(Elf64_Rela)=24.

* COPY, GLOB_DAT and J[U]MP_SLOT always have 0 addend. A ld.so
  implementation does not need to read the implicit addend.
  REL is strictly better.
* A RELATIVE has a non-zero addend. Such relocations can be packed
  compactly with the RELR relocation entry format, which is out of scope
  of this patch.
* For other dynamic relocation types (e.g. symbolic relocation R_X86_64_64),
  a ld.so implementation needs to read the implicit addend. REL may have
  minor performance impact, because reading implicit addends forces
  random access reads instead of being able to blast out a bunch of
  writes while chasing the relocation array.

This patch adds -z rel and -z rela to change the relocation entry format
for dynamic relocations. I have tested that a -z rel produced x86-64
executable works with musl ld.so

-z rela may be useful for debugging purposes on processors whose psABIs
specify REL as the canonical format: addends can be easily read by a tool.

Reviewed By: grimar, mcgrathr

Differential Revision: https://reviews.llvm.org/D80496
2020-05-29 14:22:03 -07:00
Sam Clegg 81443ac1bc [WebAssembly] Add placeholders for R_WASM_TABLE_INDEX_REL_SLEB relocations
Previously in the object format we punted on this and simply wrote
zeros (and didn't include the function in the elem segment).  With
this change we write a meaningful value which is the segment
relative table index of the associated function.

This matches the that wasm-ld produces in `-r` mode.  This inconsistency
between the output the MC object writer and the wasm-ld object
writer could cause warnings to be emitted when reading back in the
output of `wasm-ld -r`.  See:
https://github.com/emscripten-core/emscripten/issues/11217

This only applies to this one relocation type which is only generated
when compiling in PIC mode.

Differential Revision: https://reviews.llvm.org/D80774
2020-05-29 10:57:26 -07:00
Sam Clegg fd1c894a4a [lld][WebAssembly] Convert some lld tests to assembly
When we originally wrote these tests we didn't have a stable and
fleshed out assembly format.  Now we do so we should prefer that
over llvm ir for lld tests to avoid including more part of llvm
than necessary in order to run the test.

This change converts just 30 out of about 130 test files. More to
come when I have some more time.

Differential Revision: https://reviews.llvm.org/D80361
2020-05-28 16:52:01 -07:00
Hiroshi Yamauchi a7fa35a629 [ThinLTO] Compute the basic block count across modules.
Summary:
Count the per-module number of basic blocks when the module summary is computed
and sum them up during Thin LTO indexing.

This is used to estimate the working set size under the partial sample PGO.

This is split off of D79831.

Reviewers: davidxl, espindola

Subscribers: emaste, inglorion, hiraditya, MaskRay, steven_wu, dexonsmith, arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80403
2020-05-28 10:33:05 -07:00
Simon Pilgrim f6417f5db8 FileOutputBuffer.h - remove unused includes. NFC.
Move dependent includes down to source files where necessary.
2020-05-28 14:38:12 +01:00
Fangrui Song 1224e619d9 [ELF][test] Fix wrap-no-real.s after D51283
Give %t3.so a DT_SONAME so that the DT_NEEDED entry in a dependent executable has a fixed length.
2020-05-27 17:43:33 -07:00
Rui Ueyama 54d2896852 [ELF] --wrap: Drop __real_ symbol from the symbol table
In D34993, we discussed and concluded that we should drop `__real_
symbol from the symbol table, but I did the opposite in D50569.
This patch is to drop `__real_` symbol.

MaskRay's note: omitting `__real_` is important if it is undefined:
otherwise a subsequent link may error due to the undefined `__real_` in .dynsym

Differential Revision: https://reviews.llvm.org/D51283
2020-05-27 16:58:00 -07:00
Fangrui Song b8a3c618d6 [ELF] Allow misaligned SHT_GNU_verneed
Bazel created interface shared objects (.ifso) may be misaligned.  We use
llvm::support::detail::packed_endian_specific_integral under the hood
which allows reading of misaligned values, so there is not a need to
diagnose (in LLD we don't intend to support sophisticated parsing for
SHT_GNU_*).
2020-05-26 11:18:19 -07:00
Fangrui Song bae7cf6746 [ELF][PPC64] Synthesize _savegpr[01]_{14..31} and _restgpr[01]_{14..31}
In the 64-bit ELF V2 API Specification: Power Architecture, 2.3.3.1. GPR
Save and Restore Functions defines some special functions which may be
referenced by GCC produced assembly (LLVM does not reference them).

With GCC -Os, when the number of call-saved registers exceeds a certain
threshold, GCC generates `_savegpr0_* _restgpr0_*` calls and expects the
linker to define them. See
https://sourceware.org/pipermail/binutils/2002-February/017444.html and
https://sourceware.org/pipermail/binutils/2004-August/036765.html . This
is weird because libgcc.a would be the natural place. However, the linker
generation approach has the advantage that the linker can generate
multiple copies to avoid long branch thunks. We don't consider the
advantage significant enough to complicate our trunk implementation, so
we take a simple approach.

* Check whether `_savegpr0_{14..31}` are used
* If yes, define needed symbols and add an InputSection with the code sequence.

`_savegpr1_*` `_restgpr0_*` and `_restgpr1_*` are similar.

Reviewed By: sfertile

Differential Revision: https://reviews.llvm.org/D79977
2020-05-26 09:35:41 -07:00
Kang Zhang 86e3abc9e6 [PowerPC] Add some InstAlias definitions
Summary:
This patch add the InstAlias definitions for below instructions.

ADDI ADDIS ADDI8 ADDIS8
RLWINM8
ISEL ISEL8
OR OR_rec ORI ORI8 XORI8
CNTLZW8 CNTLZW8_rec
TEND TSR
RFEBB
NOR NOR_rec
MTCRF
SUBF SUBF_rec SUBFC SUBFC_rec
RLDICL_32_64
TW

Reviewed By: steven.zhang

Differential Revision: https://reviews.llvm.org/D77559
2020-05-24 14:05:28 +00:00
Tobias Hieta f794808bb9 [LLD/MinGW]: Expose --thinlto-cache-dir
Differential Revision: https://reviews.llvm.org/D80438
2020-05-24 12:30:56 +03:00
Fangrui Song e32f04cdc9 [ELF] Parse SHT_GNU_verneed and respect versioned undefined symbols in shared objects
An undefined symbol in a shared object can be versioned, like `f@v1`.
We currently insert `f` as an Undefined into the symbol table, but we
should insert `f@v1` instead.

The string `v1` is inferred from SHT_GNU_versym and SHT_GNU_verneed.
This patch implements the functionality.

Failing to do this can cause two issues:

* If a versioned symbol referenced by a shared object is defined in the
  executable, we will fail to export it.
* If a versioned symbol referenced by a shared object in another object
  file, --no-allow-shlib-undefined may spuriously report an
  "undefined reference to " error. See https://bugs.llvm.org/show_bug.cgi?id=44842
  (Linking -lfftw3 -lm on Arch Linux can cause
  `undefined reference to __log_finite`)

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D80059
2020-05-23 09:55:48 -07:00
Thomas Lively d851fce4cb [lld][WebAssembly] Do not emit initialization for .bss segments
Summary:
This patch fixes a bug where initialization code for .bss segments was
emitted in the memory initialization function even though the .bss
segments were discounted in the datacount section and omitted in the
data section. This was producing invalid binaries due to out-of-bounds
segment indices on the memory.init and data.drop instructions that
were trying to operate on the nonexistent .bss segments.

Reviewers: sbc100

Subscribers: dschuff, jgravelle-google, aheejin, sunfish, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80354
2020-05-21 11:33:25 -07:00
Reid Kleckner 3508c1d8fb [LLD] Make scoped timers thread safe
Summary:
This is a pre-requisite to parallelizing PDB symbol and type merging.
Currently this timer usage would not be thread safe.

Reviewers: aganea, MaskRay

Subscribers: jfb, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80298
2020-05-20 16:16:08 -07:00
Jez Ng ce0d8beebc [lld-macho][re-land] Support X86_64_RELOC_UNSIGNED
This reverts commit db8559eee4.
2020-05-19 12:31:55 -07:00
Jez Ng 4eb6f4854e [lld-macho][re-land] Support .subsections_via_symbols
Summary:
This diff restores and builds upon @pcc and @ruiu's initial work on
subsections.

The .subsections_via_symbols directive indicates we can split each
section along symbol boundaries, unless those symbols have been marked
with `.alt_entry`.

We exercise this functionality in our tests by using order files that
rearrange those symbols.

Depends on D79668.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: thakis, llvm-commits, pcc, ruiu

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D79926
2020-05-19 12:31:54 -07:00
Fangrui Song 6467649974 [ELF] Make --trace-symbol track preempted shared definitions
Note, we still name a preempted SharedSymbol "shared definition",
instead of "reference" as printed by GNU ld. This difference should not matter.

```
// GNU ld
ld.bfd: t: definition of f@v1
ld.bfd: t.so: reference to f@v1
```

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D80143
2020-05-19 08:56:35 -07:00
Jez Ng 70fbbcdd34 Revert "[lld-macho] Support .subsections_via_symbols"
Due to build breakage mentioned in https://reviews.llvm.org/D79926.

This reverts commit e270b2f172.
2020-05-19 08:30:02 -07:00
Jez Ng db8559eee4 Revert "[lld-macho] Support X86_64_RELOC_UNSIGNED"
This reverts commit 1f820e3559.
2020-05-19 08:30:02 -07:00
Jez Ng 1f820e3559 [lld-macho] Support X86_64_RELOC_UNSIGNED
Note that it's only used for non-pc-relative contexts.

Reviewed By: MaskRay, smeenai

Differential Revision: https://reviews.llvm.org/D80048
2020-05-19 07:46:57 -07:00
Jez Ng e270b2f172 [lld-macho] Support .subsections_via_symbols
This diff restores and builds upon @pcc and @ruiu's initial work on
subsections.

The .subsections_via_symbols directive indicates we can split each
section along symbol boundaries, unless those symbols have been marked
with `.alt_entry`.

We exercise this functionality in our tests by using order files that
rearrange those symbols.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D79926
2020-05-19 07:46:57 -07:00
Jez Ng 55e9eb416e [lld-macho] Support -order_file
The order file indicates how input sections should be sorted within each
output section, based on the symbols contained within those sections.

This diff sets the stage for implementing and testing
`.subsections_via_symbols`, where we will break up InputSections by each
symbol and sort them more granularly.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D79668
2020-05-19 07:46:57 -07:00
Greg McGary 1aadd6ce61 [lld] Remove unused lld/test/Driver/Inputs/**/libtest.a
Under `lld/test/Driver/Inputs/`, all instances of `libtest.a` are
unreferenced. FYI, all of these are empty archives, and the files
contain only a magic number.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D80182
2020-05-19 07:19:10 -07:00
Hongtao Yu 90af55d8a9 [LLD][ELF] Use offset in thin archives to disambiguate thinLTO members
This is fixing a thinLTO module collision issue for thin archives. The problem is that we always use a zero offset to name members in a thin archive and that causes the following build error:

    ld.lld: error: Expected at most one ThinLTO module per bitcode file

which happens to a thin archive that has two members with the same object file name (whose paths will be ignored by thinLTO driver)

The fix here is to use real member offset instead as is done for non-thin archives.

Differential Revision: https://reviews.llvm.org/D79880
2020-05-15 12:02:08 -07:00
Fangrui Song e36223c85c [ELF] Enforce two dashes for Flag options not supported by GNU ld (i.e. no compatibility burden)
Announced on https://lists.llvm.org/pipermail/llvm-dev/2020-May/141416.html

Similar to D79371, but for `multiclass B` (convenience helper for defining --foo and --no-foo)

Some changed options are also used by gold, but I haven't seen their
one-dash use cases outside of lld's testsuite.
2020-05-15 11:07:25 -07:00
Fangrui Song 07837b8f49 [ELF] Use namespace qualifiers (lld:: or elf::) instead of `namespace lld { namespace elf {`
Similar to D74882. This reverts much code from commit
bd8cfe65f5 (D68323) and fixes some
problems before D68323.

Sorry for the churn but D68323 was a mistake. Namespace qualifiers avoid
bugs where the definition does not match the declaration from the
header. See
https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions (D74515)

Differential Revision: https://reviews.llvm.org/D79982
2020-05-15 08:49:53 -07:00
Georgii Rymar 969c63a2ec [LLD][ELF] - Fix section-alignment.test after yaml2obj change.
yaml2obj changed in rG7ccae2cece72. Previous version of the
yaml produced a different error and a 500mb binary.
2020-05-15 12:00:37 +03:00
Russell Gallop 85bb9b71b7 [ELF] Update release notes and man page for LLD time-trace
Differential Revision: https://reviews.llvm.org/D79780
2020-05-15 08:35:58 +01:00
Kellie Medlin 2b920ae78c [lld] Add archive file support to Mach-O backend
With this change, basic archive files can be linked together. Input
section discovery has been refactored into a function since archive
files lazily resolve their symbols / the object files containing those
symbols.

Reviewed By: int3, smeenai

Differential Revision: https://reviews.llvm.org/D78342
2020-05-14 12:58:35 -07:00