Commit Graph

13099 Commits

Author SHA1 Message Date
Eric Christopher 058ec20677 [lld] As part of using inclusive language within the llvm
project, migrate away from the use of blacklist and whitelist.
2020-06-19 21:50:14 -07:00
Fangrui Song 8ffb2097cc [ELF] Refine LMA offset propagation rule in D76995
If neither AT(lma) nor AT>lma_region is specified,
D76995 keeps `lmaOffset` (LMA - VMA) if the previous section is in the
default LMA region.

This patch additionally checks that the two sections are in the same
memory region.

Add a test case derived from https://bugs.llvm.org/show_bug.cgi?id=45313

  .mdata : AT(0xfb01000) { *(.data); } > TCM
  // It is odd to make .bss inherit lmaOffset, because the two sections
  // are in different memory regions.
  .bss : { *(.bss) } > DDR

With this patch, section VMA/LMA match GNU ld. Note, GNU ld supports
out-of-order (w.r.t sh_offset) sections and places .text and .bss in the
same PT_LOAD. We don't have that behavior.

Reviewed By: grimar

Differential Revision: https://reviews.llvm.org/D81986
2020-06-19 09:11:33 -07:00
Fangrui Song c4d13f72a6 [ELF] Refactor ObjFile<ELFT>::initializeSymbols to enforce the invariant: InputFile::symbols has non null entry
Fixes PR46348.

ObjFile<ELFT>::initializeSymbols contains two symbol iteration loops:

```
for each symbol
  if non-inheriting && non-local
    fill in this->symbols[i]

for each symbol
  if local
    fill in this->symbols[i]
  else
    symbol resolution
```

Symbol resolution can trigger a duplicate symbol error which will call
InputSectionBase::getObjMsg to iterate over InputFile::symbols.  If a
non-local symbol appears after the non-local symbol being resolved
(violating ELF spec), its `this->symbols[i]` entry has not been filled
in, InputSectionBase::getObjMsg will crash due to
`dyn_cast<Defined>(nullptr)`.

To fix the bug, reorganize the two loops to ensure this->symbols is
complete before symbol resolution. This enforces the invariant:
InputFile::symbols has none null entry when InputFile::getSymbols() is called.

```
for each symbol
  if non-inheriting
    fill in this->symbols[i]

for each symbol starting from firstGlobal
  if non-local
    symbol resolution
```

Additionally, move the (non-local symbol in local part of .symtab)
diagnostic from Writer<ELFT>::copyLocalSymbols() to initializeSymbols().

Reviewed By: grimar, jhenderson

Differential Revision: https://reviews.llvm.org/D81988
2020-06-19 09:05:37 -07:00
Alexandre Ganea 2ae0df5be7 [CodeView] Revert 8374bf4363 and 403f953792
This reverts:
8374bf4363 [CodeView] Fix generated command-line expansion in LF_BUILDINFO. Fix the 'pdb' entry which was previously a null reference, now an empty string.
403f953792 [CodeView] Add full repro to LF_BUILDINFO record

This is causing the lld/test/COFF/pdb-relative-source-lines.test to fail: http://lab.llvm.org:8011/builders/lld-x86_64-win/builds/1096/steps/test-check-all/logs/FAIL%3A%20lld%3A%3Apdb-relative-source-lines.test
And clang/test/CodeGen/debug-info-codeview-buildinfo.c fails as well: http://lab.llvm.org:8011/builders/clang-s390x-linux/builds/33346/steps/ninja%20check%201/logs/FAIL%3A%20Clang%3A%3Adebug-info-codeview-buildinfo.c
2020-06-18 16:18:46 -04:00
Sam Clegg 21e14cce23 [lld][WebAssembly] Allow ctors functions that return values
Some projects use the constructor attribute on functions that also
return values.  In this case we just ignore them.

The error was reported in the libgpg-error project that marks
gpg_err_init with the `__constructor__` attribute.

Differential Revision: https://reviews.llvm.org/D81962
2020-06-18 13:11:40 -07:00
Nico Weber 9bcd59fdef fix a typo to cycle bots 2020-06-18 11:31:47 -04:00
Alexandre Ganea 403f953792 [CodeView] Add full repro to LF_BUILDINFO record
This patch adds some missing information to the LF_BUILDINFO which allows for rebuilding an .OBJ without any external dependency but the .OBJ itself (other than the compiler executable).

Some tools need this information to reproduce a build without any knowledge of the build system. The LF_BUILDINFO therefore stores a full path to the compiler, the PWD (which is the CWD at program startup), a relative or absolute path to the TU, and the full CC1 command line. The command line needs to be freestanding (not depend on any environment variable). In the same way, MSVC doesn't store the provided command-line, but an expanded version (somehow their equivalent of CC1) which is also freestanding.

For more information see PR36198 and D43002.

Differential Revision: https://reviews.llvm.org/D80833
2020-06-18 09:17:15 -04:00
Greg McGary f7453a97ce [lld-macho] Add missing search-paths-darwin.test
Summary: Forgot to `git add` it when patching D80677
2020-06-17 20:51:26 -07:00
Greg McGary d50f44a2f7 [lld-macho] Handle framework search path, alongside library search path
Summary:
Add front-end support for `lld::macho::Configuration::frameworkSearchPath`.

Depends on D80582.

Reviewers: ruiu, pcc, MaskRay, smeenai, int3, Ktwu, alexshap, christylee

Reviewed By: int3

Subscribers: ormris, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80677
2020-06-17 20:41:28 -07:00
Jez Ng 525c7d8cda [lld-macho] Handle alignment correctly when merging InputSections
Summary:
Previously, we weren't updating isecAddr when aligning InputSections,
resulting in truncated sections under the right conditions.

Reviewers: #lld-macho, compnerd

Reviewed By: #lld-macho, compnerd

Subscribers: smeenai, compnerd, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81298
2020-06-17 20:41:28 -07:00
Jez Ng 74871cdad7 [lld-macho] Ensure __bss sections we output have file offset of zero
Summary:
llvm-mc emits `__bss` sections with an offset of zero, but we weren't expecting
that in our input, so we were copying non-zero data from the start of the file and
putting it in `__bss`, with obviously undesirable runtime results. (It appears that
the kernel will copy those nonzero bytes as long as the offset is nonzero, regardless
of whether S_ZERO_FILL is set.)

I debated on whether to make a special ZeroFillSection -- separate from a
regular InputSection -- but it seemed like too much work for now. But I'm happy
to refactor if anyone feels strongly about having it as a separate class.

Depends on D80857.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80859
2020-06-17 20:41:28 -07:00
Jez Ng a12e7d406d [lld-macho] Handle GOT relocations of non-dylib symbols
Summary:
Turns out this case is actually really common -- it happens whenever there's
a reference to an `extern` variable that ends up statically linked.

Depends on D80856.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80857
2020-06-17 20:41:28 -07:00
Jez Ng c3d98ea89f [lld-macho] Support X86_64_RELOC_GOT
Summary:
As far as I can tell, it's identical to _GOT_LOAD. llvm-mc has the following
comment explaining why _GOT exists:

```
// x86_64 distinguishes movq foo@GOTPCREL so that the linker can
// rewrite the movq to an leaq at link time if the symbol ends up in
// the same linkage unit.
```

Depends on D80855.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Reviewed By: MaskRay, smeenai

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80856
2020-06-17 20:41:28 -07:00
Jez Ng fcde378dcb [lld-macho] Support non-pcrel section relocs
Summary: Depends on D80854.

Reviewers: ruiu, pcc, MaskRay, smeenai, alexshap, gkm, Ktwu, christylee

Subscribers: llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D80855
2020-06-17 20:41:28 -07:00
Jez Ng 2f4cfa3c7a [lld-macho] Avoid explicit -arch in tests by defaulting to x86-64
Summary:
As mentioned in https://reviews.llvm.org/D81326#2093931, I'm not sure it
makes sense to use the default target triple to determine -arch.
Long-term we should probably detect it from the input object files, but
in the meantime it would be nice not to have to add it to all our tests
by using a convenient default.

Reviewers: #lld-macho

Subscribers: arphaman, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81983
2020-06-17 20:41:27 -07:00
Fangrui Song 49279ca160 [ELF] Improve --export-dynamic-symbol performance by checking whether wildcard is really used
A hasWildcard pattern iterates over symVector, which can be slow when there
are many --export-dynamic-symbol. In optimistic cases, most patterns don't use
a wildcard character. hasWildcard: false can avoid a symbol table iteration.

While here, add two tests using `[` and `?`, respectively.
2020-06-17 17:12:10 -07:00
Jez Ng a2d096df26 [lld-macho] Use uint64_t for getSize() instead of size_t
Summary:
So things work on 32-bit machines. (@vzakhari reported the
breakage starting from D80177).

Reviewers: #lld-macho, vzakhari

Subscribers: llvm-commits, vzakhari

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D81982
2020-06-16 18:42:45 -07:00
Fangrui Song a649802c59 [lld-macho][test] Fix dylink-lazy.s after 0a5228a02c 2020-06-16 17:57:59 -07:00
Saleem Abdulrasool 0a5228a02c test: add library search path to repair the test
Update the test to no longer rely on the testing workaround in the
linker and use the external libSystem stub.
2020-06-16 17:06:21 -07:00
Saleem Abdulrasool 73312976ad lld: remove old test support path
This removes the stub library that lld injected to satisfy the
dependency on the libSystem.  Now with TBD support, we can provide the
stub library to permit the tests to function properly as they would on a
real system.

Reviewed By: smeenai
Differential Revision: https://reviews.llvm.org/D81418
2020-06-16 15:57:58 -07:00
Greg McGary 7df80e3f23 [lld-macho] Specify the complete set of command-line options for ld64
This is a complete Options.td compiled from ld(1) dated 2018-03-07 and
cross checked with ld64 source code version 512.4 dated 2018-03-18.

This is the first in a series of diffs for argument handling. Follow-ups
will include switch cases for all the new instances of `OPT_foo`, and
parsing/validation of arguments attached to options, e.g., more code
akin to `OPT_platform_version` and associated `parsePlatformVersion()`.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80582
2020-06-15 12:50:20 -07:00
Shoaib Meenai 72e096fd1e [MachO] Fix typo in comment
The case the calculation works for is when r_length = 2.
2020-06-15 12:26:55 -07:00
Wouter van Oortmerssen 3b29376e3f [WebAssembly] Adding 64-bit version of R_WASM_MEMORY_ADDR_* relocs
This adds 4 new reloc types.

A lot of code that previously assumed any memory or offset values could be contained in a uint32_t (and often truncated results from functions returning 64-bit values) have been upgraded to uint64_t. This is not comprehensive: it is only the values that come in contact with the new relocation values and their dependents.

A new tablegen mapping was added to automatically upgrade loads/stores in the assembler, which otherwise has no way to select for these instructions (since they are indentical other than for the offset immediate). It follows a similar technique to https://reviews.llvm.org/D53307

Differential Revision: https://reviews.llvm.org/D81704
2020-06-15 10:07:42 -07:00
Wouter van Oortmerssen d9e0bbd17b [WebAssembly] Adding 64-bit versions of all load & store ops.
Context: https://github.com/WebAssembly/memory64/blob/master/proposals/memory64/Overview.md
This is just a first step, adding the new instruction variants while keeping the existing 32-bit functionality working.
Some of the basic load/store tests have new wasm64 versions that show that the basics of the target are working.
Further features need implementation, but these will be added in followups to keep things reviewable.

Differential Revision: https://reviews.llvm.org/D80769
2020-06-15 08:31:56 -07:00
Kirill Bobyrev 9baba7cf66
Revert "[lld-macho] No need to explicitly specify -arch in tests"
This reverts commit 51c5baacf3 and also
337fb8c767 - "[lld-macho] Set REQUIRES:
x86 on more tests".

These patches cause test crashes:

http://lab.llvm.org:8011/builders/ppc64le-lld-multistage-test/builds/10054
2020-06-15 12:27:30 +02:00
Jez Ng 337fb8c767 [lld-macho] Set REQUIRES: x86 on more tests
Summary: Fixes the build break caused by D81802.
2020-06-14 19:05:12 -07:00
Jez Ng 53c796b948 [lld-macho] Properly handle & validate relocation r_length
Summary:
We should be reading / writing our addends / relocated addresses based on
r_length, and not just based on the type of the relocation. But since only
some r_length values are valid for a given reloc type, I've also added some
validation.

ld64 has code to allow for r_length = 0 in X86_64_RELOC_BRANCH relocs, but I'm
not sure how to create such a relocation...

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80854
2020-06-14 16:35:23 -07:00
Jez Ng 51c5baacf3 [lld-macho] No need to explicitly specify -arch in tests
Summary: After {D81326} landed, some tests started failing if they did
not have `-arch` specified. I think one of the reasons happened was due
to the fact that we were taking a reference to a temporary value that
was freed too early. Fixing that got the error to go away on my local
Linux machine.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D81802
2020-06-14 16:35:21 -07:00
Pavel Labath fccaa89e23 [llvm/DWARFDebugLine] Fix a typo in one warning message 2020-06-11 13:04:52 +02:00
Hongtao Yu 2638aafe12 [LLD][ThinLTO] Add --thinlto-single-module to allow compiling partial modules.
This change introduces an LLD switch --thinlto-single-module to allow compiling only a part of the input modules. This is specifically enables:

  1. Fast investigating/debugging modules of interest without spending time on compiling unrelated modules.
  2. Compiler debug dump with -mllvm -debug-only= for specific modules.

It will be useful for large applications which has 1K+ input modules for thinLTO.

The switch can be combined with `--lto-obj-path=` or `--lto-emit-asm` to obtain intermediate object files or assembly files. So far the module name matching is implemented as a fuzzy name lookup where the modules with name containing the switch value are compiled.

E.g,
Command:
     ld.lld main.o thin.a --thinlto-single-module=thin.a --lto-obj-path=single.o
log:
     [ThinLTO] Selecting thin.a(thin1.o at 168) to compile
     [ThinLTO] Selecting thin.a(thin2.o at 228) to compile
Command:
     ld.lld main.o thin.a --thinlto-single-module=thin1.o --lto-obj-path=single.o
log:
     [ThinLTO] Selecting thin.a(thin1.o at 168) to compile

Differential Revision: https://reviews.llvm.org/D80406
2020-06-10 15:32:30 -07:00
Pavel Labath de522035b9 [lld] Fix test/ELF/undef.s for 6f55b5a1
There's been a slight change in wording for the warning message about
debug_line problems.
2020-06-10 16:36:10 +02:00
gbreynoo be44b79257 [lld][test] Expand testing for dynamic-list and export-dynamic
- Expanded testing for --dynamic-list and --export-dynamic
- Fixed invalid-dynamic-list.test

Differential Revision: https://reviews.llvm.org/D80311
2020-06-10 14:06:30 +01:00
Fangrui Song b114e134bd [ELF] Fix --thinlto-index-only regression after D79300
After D79300, we don't rewrite InputFile::mb to an empty buffer.
In thinLTOCreateEmptyIndexFiles(), we should check LazyObjFile::fetched
as well as checking whether mb is a bitcode, otherwise we would overwrite (path + .thinlto.bc) with an empty index.
2020-06-09 23:10:30 -07:00
Fangrui Song ba890da287 [ELF] Demote lazy symbols relative to a discarded section to Undefined
Fixes PR45594.

In `ObjFile<ELFT>::initializeSymbols()`, for a defined symbol relative to
a discarded section (due to section group rules), it may have been
inserted as a lazy symbol. We need to demote it to an Undefined to
enable the `discarded section` error happened in a later pass.

Add `LazyObjFile::fetched` (if true) and `ArchiveFile::parsed` (if
false) to represent that there is an ongoing lazy symbol fetch and we
should replace the current lazy symbol with an Undefined, instead of
calling `Symbol::resolve` (`Symbol::resolve` should be called if the lazy
symbol was added by an unrelated archive/lazy object).

As a side result, one small issue in start-lib-comdat.s is now fixed.
The hack motivating D51892 will be unsupported: if
`.gnu.linkonce.t.__i686.get_pc_thunk.bx` in an archive is referenced
by another section, this will likely be errored unless the function is
also defined in a regular object file.
(Bringing back rL330869 would error `undefined symbol` instead of the
more relevant `discarded section`.)

Note, glibc i386's crti.o still works (PR31215), because
`.gnu.linkonce.t.__x86.get_pc_thunk.bx` is in crti.o (one of the first
regular object files in a linker command line).

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D79300
2020-06-09 11:27:34 -07:00
Saleem Abdulrasool 5fda192fed test: repair lld builder
Add the explicit `-arch x86_64` to the invocation.
2020-06-08 20:13:54 -07:00
Saleem Abdulrasool 6fe27b5fed lld: initial pass at supporting TBD
Add support to lld to use Text Based API stubs for linking.  This is
support is incomplete not filtering out platforms.  It also does not
account for architecture specific API handling and potentially does not
correctly handle trees of re-exports with inlined libraries being
treated as direct children of the top level library.
2020-06-08 18:15:40 -07:00
Michael Liao 43793b89a0 [lld] Fix shared library build by adding the missing dependency. 2020-06-08 16:12:58 -04:00
Saleem Abdulrasool fcdf7578aa lld: improve the `-arch` handling for MachO
Use the default target triple configured by the user to determine the
default architecture for `ld64.lld`.  Stash the architecture in the
configuration as when linking against TBDs, we will need to filter out
the symbols based upon the architecture.  Treat the Haswell slice as it
is equivalent to `x86_64` but with the extra Haswell extensions (e.g.
AVX2, FMA3, BMI1, etc).  This will make it easier to add new
architectures in the future.

This change also changes the failure mode where an invalid `-arch`
parameter will result in the linker exiting without further processing.
2020-06-08 11:04:19 -07:00
Fangrui Song ac6abc99e2 [ELF] Don't cause assertion failure if --dynamic-list or --version-script takes an empty file
Fixes PR46184
Report line 1 of the last memory buffer.
2020-06-05 15:59:54 -07:00
Saleem Abdulrasool e78431354b lld: use modern library search ordering
This merges the static and shared library and behaves as if
`-search_paths_first` was specified which is also the default behaviour
on ld64 (and now lld). Unify the paths, and use `llvm::sys::path` to
deal with the path to be truly agnostic to the host.
2020-06-05 12:12:26 -07:00
Fangrui Song 7bee6e30fe [ELF] Handle -u before input files
If both a.a and b.so define foo

```
ld.bfd -u foo a.a b.so  # foo is defined
ld.bfd a.a b.so -u foo  # foo is defined
ld.bfd -u foo b.so a.a  # foo is undefined (provided at runtime by b.so)
ld.bfd b.so a.a -u foo  # foo is undefined (provided at runtime by b.so)
```

In all cases we make foo undefined in the output.  I tend to think the
GNU ld behavior makes more sense.

* In their model, they have to treat -u as a fake object file with an
  undefined symbol before all input files, otherwise the first archive would not be fetched.
* Following their behavior allows us to drop a --warn-backrefs special case.

Reviewed By: psmith

Differential Revision: https://reviews.llvm.org/D81052
2020-06-05 08:44:38 -07:00
Saleem Abdulrasool 116e38fd8b lld: add basic static library search
This is a very basic static library search addition. This is the pre-Xcode4
behaviour of searching all paths for the shared version before searching for
the static version of the library. This behaviour is supposed to be inverted
with `-search_paths_first` being the default. This adds the library search
with the intention of providing the setup to merge the paths into one path
and making it controllable by `OPT_search_paths_first`.
2020-06-03 23:32:05 +00:00
Saleem Abdulrasool 9cb2badc59 lld: use `std::make_unique` (NFC)
The LLVM code base already uses C++14, use std::make_unique
to avoid the explicit constructor invocation via new and to avoid
spelling out the type twice.
2020-06-03 19:23:13 +00:00
Saleem Abdulrasool 37d93b528c lld: ignore the `-search_paths_first` option on MachO
ld64 provides the `-search_path_firsts` which will search each path in
the library search path order for both `lib[name].dylib`, `lib[name].a`
before moving on (searching all paths for the dylib and then falling
back to the static library if a shared library was not found).

This option has been the default for a long time, but the command line
flag still exists.  Ignore it for compatibility.
2020-06-03 15:36:35 +00:00
Fangrui Song 3eb4bf13ba [ELF] Append " [--no-allow-shlib-undefined]" to the corresponding diagnostics
--no-allow-shlib-undefined (enabled by default when linking an
executable) rejects unresolved references in shared objects.

Users may be confused by the common diagnostics of unresolved symbols in
object files (LLD: "undefined symbol: foo"; GNU ld/gold: "undefined reference to")

Learn from GCC/clang " [-Wfoo]": append the option name to the
diagnostics. Users can find relevant information by searching
"--no-allow-shlib-undefined".  It should also be obvious to them that
the positive form --allow-shlib-undefined can suppress the error.

Also downgrade the error to a warning if --noinhibit-exec is used (compatible
with GNU ld and gold).

Reviewed By: grimar, psmith

Differential Revision: https://reviews.llvm.org/D81028
2020-06-03 07:59:37 -07:00
Jez Ng d767de44bf [lld-macho] Fix PAGEZERO=4GB errors on Windows by ensuring enum is uint64_t
It appears that MSVC doesn't resize the enum properly to fit the
constants.
2020-06-02 15:24:31 -07:00
Reid Kleckner 3eb16fe4e9 [LLD] Have only one SpecificAllocator per type
Previously, the SpecificAllocator was a static local in the `make<T>`
function template. Using static locals is nice because they are only
constructed and registered if they are accessed. However, if there are
multiple calls to make<> with different constructor parameters, we would
get multiple static local variable instances. This is undesirable and
leads to extra memory allocations. I noticed there were two sources of
DefinedRegular allocations while checking heap profiles.
2020-06-02 14:09:09 -07:00
Jez Ng 1e1a3f67ee [lld-macho] Ensure reads from nlist_64 structs are aligned when necessary
My test refactoring in D80217 seems to have caused yaml2obj to emit
unaligned nlist_64 structs, causing ASAN'd lld to be unhappy. I don't
think this is an issue with yaml2obj though -- llvm-mc also seems to
emit unaligned nlist_64s. This diff makes lld able to safely do aligned
reads under ASAN builds while hopefully creating no overhead for regular
builds on architectures that support unaligned reads.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D80414
2020-06-02 13:19:38 -07:00
Jez Ng f04d1c3b90 [lld-macho] Move all tests for erroneous inputs under invalid/
For consistency.

The no-id-dylib test was originally referencing the Inputs/ folder via a
relative path. Instead of updating that path, I decided to make the test
self-contained.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D80217
2020-06-02 13:19:38 -07:00
Jez Ng a04c133564 [lld-macho] Set __PAGEZERO size to 4GB
That's what ld64 uses for 64-bit targets. I figured it's best to make
this change sooner rather than later since a bunch of our tests are
relying on hardcoded addresses that depend on this value.

Reviewed By: smeenai

Differential Revision: https://reviews.llvm.org/D80177
2020-06-02 13:19:38 -07:00