Commit Graph

303 Commits

Author SHA1 Message Date
Vy Nguyen 9a2e2de15f [lld-macho] Change loadReexport to handle the case where a TAPI re-exports to reference documents nested within other TBD.
Currently, it was delibrately impleneted to not handle this case, but as it has turnt out, we need this feature.
The concrete use case is
       `System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa` reexports
               /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit , which then rexports
                    /System/Library/PrivateFrameworks/UIFoundation.framework/Versions/A/UIFoundation

The current implemention uses a global currentTopLevelTapi, which is not reset until it finishes loading the whole tree.
This is a problem because if the top-level is set to Cocoa, then when we get to UIFoundation, it will try to find UIFoundation in the current top level, which is Cocoa and will not find it.

The right thing should be:
 - When loading a library from a TBD file, re-exports need to be looked up in the auxiliary documents within the same TBD.
 - When loading from an actual dylib, no additional TBD documents need to be examined.
 - In no case does a re-export mentioned in one TBD file need to be looked up in a document in an auxiliary document from a different TBD file

Differential Revision: https://reviews.llvm.org/D97438
2021-03-02 12:14:31 -05:00
Nico Weber 0658fc654c [lld/mac] Implement the missing bits of -undefined
This adds support for `-undefined dynamic_lookup`, and for
`-undefined warning` and `-undefined suppress` with `-flat_namespace`.

We just replace undefined symbols with a DynamicLookup when we hit them.

With this, `check-llvm` passes when using ld64.lld.darwinnew as host linker.

Differential Revision: https://reviews.llvm.org/D97642
2021-03-01 15:30:53 -05:00
Nico Weber 8174f33dc9 [lld/mac] Add support for -flat_namespace
-flat_namespace makes lld emit binaries that use name lookup that's more in
line with other POSIX systems: Instead of looking up symbols as (dylib,name)
pairs by dyld, they're instead looked up just by name.

-flat_namespace has three effects:

1. MH_TWOLEVEL and MH_NNOUNDEFS are no longer set in the Mach-O header
2. All symbols use BIND_SPECIAL_DYLIB_FLAT_LOOKUP as ordinal
3. When a dylib is added to the link, its dependent dylibs are also added,
   so that lld can verify that no undefined symbols remain at the end of
   a link with -flat_namespace. These transitive dylibs are added for symbol
   resolution, but they are not emitted in LC_LOAD_COMMANDs.

-undefined with -flat_namespace still isn't implemented. Before this change,
it was impossible to hit that combination because -flat_namespace caused a
diagnostic. Now that it no longer does, emit a dedicated temporary diagnostic
when both flags are used.

Differential Revision: https://reviews.llvm.org/D97641
2021-03-01 15:25:10 -05:00
Nico Weber ab45289d2e [lld/mac] Make -v print version and search paths in additon to linking, not instead of linking
This matches ld64's behavior.

Differential Revision: https://reviews.llvm.org/D97718
2021-03-01 15:09:46 -05:00
Nico Weber bacacb9d5c [lld/mac] Prefix errors with "ld64.lld" instead of just "lld"
Matches the ELF and COFF ports, which use ld.lld and lld-link, respectively.

While here, also move up `cleanupCallback` to match ELF / COFF.

Differential Revision: https://reviews.llvm.org/D97715
2021-03-01 15:08:19 -05:00
Jez Ng f083f652c3 [lld-macho][nfc] Remove TODO regarding addends
There was initially some concern around the correct handling of pcrel
section relocations with r_length != 2. But it looks like there are no such
relocations in practice -- x86_64's pcrel section relocs all have r_length == 2,
and ARM64 doesn't even have pcrel section relocs. So we can replace the TODO
with an assert.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D97576
2021-03-01 12:30:08 -05:00
Nico Weber 860e862f34 [lld/mac] Simplify encodeDylibOrdinal() a bit
Only one of the two callers used the lastBinding parameter, so
do that work at that one call site. Extract a ordinalForDylibSymbol()
helper to make this tidy.

No behavior change.

Differential Revision: https://reviews.llvm.org/D97597
2021-02-28 09:16:45 -05:00
Greg McGary 0223ab035c [lld-macho] check minimum header length when opening linkable input files
Bifurcate the `readFile()` API into ...
* `readRawFile()` which performs no checks, and
* `readLinkableFile()` which enforces minimum length of 20 bytes, same as ld64

There are no new tests because tweaks to existing tests are sufficient.

Differential Revision: https://reviews.llvm.org/D97610
2021-02-27 14:41:40 -08:00
Greg McGary 6f9dd843db [lld-macho] Implement options -rename_section -rename_segment
Implement command-line options to rename output sections & segments.

Differential Revision: https://reviews.llvm.org/D97600
2021-02-27 11:44:12 -08:00
Jez Ng 82b3da6f6f [lld-macho] Extract embedded addends for arm64 UNSIGNED relocations
On arm64, UNSIGNED relocs are the only ones that use embedded addends
instead of the ADDEND relocation.

Also ensure that the addend works when UNSIGNED is part of a SUBTRACTOR
pair.

Reviewed By: #lld-macho, alexshap

Differential Revision: https://reviews.llvm.org/D97105
2021-02-27 12:31:34 -05:00
Jez Ng 541390131e [lld-macho] Don't emit rebase opcodes for subtractor minuend relocs
Also add a few asserts to verify that we are indeed handling an
UNSIGNED relocation as the minued. I haven't made it an actual
user-facing error since I don't think llvm-mc is capable of generating
SUBTRACTOR relocations without an associated UNSIGNED.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D97103
2021-02-27 12:31:34 -05:00
Jez Ng cc5c03e109 [lld-macho] Properly test subtractor relocations & fix their attributes
`llvm-mc` doesn't generate any relocations for subtractions
between local symbols -- they must be global -- so the previous test
wasn't actually testing any relocation logic. I've fixed that and
extended the test to cover r_length=3 relocations as well as both x86_64
and arm64.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D97057
2021-02-27 12:31:34 -05:00
Nico Weber cafb6cd10c [lld/mac] Add some support for dynamic lookup symbols, and implement -U
Dynamic lookup symbols are symbols that work like dynamic symbols
in ELF: They're not bound to a dylib like normal Mach-O twolevel lookup
symbols, but they live in a global pool and dyld resolves them against
exported symbols from all loaded dylibs.

This adds support for dynamical lookup symbols to lld/mac. They are
represented as DylibSymbols with file set to nullptr.

This also uses this support to implement the -U flag, which makes
a specific symbol that's undefined at the end of the link a
dynamic lookup symbol.

For -U, it'd be sufficient to just to a pass over remaining undefined symbols
at the end of the link and to replace them with dynamic lookup symbols then.
But I'd like to use this code to implement flat_namespace too, and that will
require real support for resolving dynamic lookup symbols in SymbolTable. So
this patch adds this now already.

While writing tests for this, I noticed that we didn't set N_WEAK_DEF in the
symbol table for DylibSymbols, so this fixes that too.

Differential Revision: https://reviews.llvm.org/D97521
2021-02-26 16:50:53 -05:00
Jez Ng 84579fc24f [lld-macho] Basic support for linkage and visibility attributes in LTO
When parsing bitcode, convert LTO Symbols to LLD Symbols in order to perform
resolution. The "winning" symbol will then be marked as Prevailing at LTO
compilation time. This is similar to what the other LLD ports do.

This change allows us to handle `linkonce` symbols correctly, and to deal with
duplicate bitcode symbols gracefully. Previously, both scenarios would result in
an assertion failure inside the LTO code, complaining that multiple Prevailing
definitions are not allowed.

While at it, I also added basic logic around visibility. We don't do anything
useful with it yet, but we do check that its value is valid. LLD-ELF appears to
use it only to set FinalDefinitionInLinkageUnit for LTO, which I think is just a
performance optimization.

From my local experimentation, the linker itself doesn't seem to do anything
differently when encountering linkonce / linkonce_odr / weak / weak_odr. So I've
only written a test for one of them. LLD-ELF has more, but they seem to mostly
be testing the intermediate bitcode output of their LTO backend...? I'm far from
an expert here though, so I might very well be missing things.

Reviewed By: #lld-macho, MaskRay, smeenai

Differential Revision: https://reviews.llvm.org/D94342
2021-02-25 13:27:40 -05:00
Greg McGary 151990dd94 [lld-macho] add code signature for native arm64 macOS
Differential Revision: https://reviews.llvm.org/D96164
2021-02-24 17:05:23 -08:00
Jez Ng 4a5e111aea [lld-macho] Better deduplication of personality pointers
{D95809} introduced a mechanism for synthetic symbol creation of personality
pointers. When multiple section relocations referred to the same personality
pointer, it would deduplicate them. However, it neglected to consider that we
could have symbol relocations that also refer to the same personality pointer.
This diff fixes it.

In practice, this mix of relocations arises when there is a statically-linked
personality routine that is referenced from multiple object files. Within the
same object file, it will be referred to via section relocations, but
(obviously) other object files will refer to it via symbol relocations. Failing
to deduplicate these references resulted in us going over the
3-personality-pointer limit when linking some larger applications.

Fixes llvm.org/PR48389.

Reviewed By: #lld-macho, thakis, alexshap

Differential Revision: https://reviews.llvm.org/D97245
2021-02-23 22:02:38 -05:00
Jez Ng 4752cdc9a2 [lld-macho] Check for arch compatibility when loading ObjFiles and TBDs
The silent failures had confused me a few times.

I haven't added a similar check for platform yet as we don't yet have logic to
infer the platform automatically, and so adding that check would require
updating dozens of test files.

Reviewed By: #lld-macho, thakis, alexshap

Differential Revision: https://reviews.llvm.org/D97209
2021-02-23 22:02:38 -05:00
Jez Ng 5e851733c5 [lld-macho] Fix semantics & add tests for ARM64 GOT/TLV relocs
I've adjusted the RelocAttrBits to better fit the semantics of
the relocations. In particular:

1. *_UNSIGNED relocations are no longer marked with the `TLV` bit, even
   though they can occur within TLV sections. Instead the `TLV` bit is
   reserved for relocations that can reference thread-local symbols, and
   *_UNSIGNED relocations have their own `UNSIGNED` bit. The previous
   implementation caused TLV and regular UNSIGNED semantics to be
   conflated, resulting in rebase opcodes being incorrectly emitted for TLV
   relocations.

2. I've added a new `POINTER` bit to denote non-relaxable GOT
   relocations. This distinction isn't important on x86 -- the GOT
   relocations there are either relaxable or non-relaxable loads -- but
   arm64 has `GOT_LOAD_PAGE21` which loads the page that the referent
   symbol is in (regardless of whether the symbol ends up in the GOT). This
   relocation must reference a GOT symbol (so must have the `GOT` bit set)
   but isn't itself relaxable (so must not have the `LOAD` bit). The
   `POINTER` bit is used for relocations that *must* reference a GOT
   slot.

3. A similar situation occurs for TLV relocations.

4. ld64 supports both a pcrel and an absolute version of
   ARM64_RELOC_POINTER_TO_GOT. But the semantics of the absolute version
   are pretty weird -- it results in the value of the GOT slot being
   written, rather than the address. (That means a reference to a
   dynamically-bound slot will result in zeroes being written.) The
   programs I've tried linking don't use this form of the relocation, so
   I've dropped our partial support for it by removing the relevant
   RelocAttrBits.

Reviewed By: alexshap

Differential Revision: https://reviews.llvm.org/D97031
2021-02-23 22:02:38 -05:00
Jez Ng e5d780e049 [lld-macho] Use full input file name in invalid relocation error message
Just something I noticed while debugging arm relocations...

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D97078
2021-02-23 22:02:38 -05:00
Vy Nguyen 5a856f5b44 Reland [lld-macho]Implement bundle_loader
Reland 1a0afcf518
  https://reviews.llvm.org/D95913

New change: fix UB bug caused by copying empty path/name. (since the executable does not have a name)
2021-02-22 14:05:12 -05:00
Jez Ng 8c4638c367 [lld-macho] Clean up comments 2021-02-22 12:13:17 -05:00
Jez Ng 5bfdbdeb40 [lld-macho] Fix cpuSubtype for non-x86_64 archs
dyld on iOS will complain if the LIB64 bit is set.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D96565
2021-02-22 12:13:17 -05:00
Nico Weber 28d9953af9 [lld/mac] reject -undefined warning and -undefined suppress with -twolevel_namespace
See discussion on https://reviews.llvm.org/D93263

-flat_namespace isn't implemented yet, and neither is -undefined dynamic,
so this makes -undefined pretty pointless in lld/MachO for now. But once
we implement -flat_namespace (which we need to do anyways to get check-llvm
to pass with lld as host linker), the code's already there.

Follow-up to https://reviews.llvm.org/D93263#2491865

Differential Revision: https://reviews.llvm.org/D96963
2021-02-20 13:35:22 -05:00
Vitaly Buka c17547df44 Revert "Implement -bundle_loader"
D95913 passes null pointer into memcpy

This reverts commit 1a0afcf518.
2021-02-19 17:40:07 -08:00
Vy Nguyen 1a0afcf518 Implement -bundle_loader
Differential Revision: https://reviews.llvm.org/D95913

Usage: -bundle_loader <executable>
This option specifies the executable that will load the build output file being linked.
When building a bundle, users can use the --bundle_loader  to specify an executable
that contains symbols referenced, but not implemented in the bundle.
2021-02-18 16:11:37 -05:00
Mikael Holmen caff023b77 [lld] Silence compiler warnings by removing always true/false comparisons
type is an uint8_t so
 type >= 0
is always true and
 type < 0
is always false.
2021-02-17 08:16:02 +01:00
Nico Weber e0b8604e5d [lld/mac] Implement -u flag
Since we emit diagnostics for undefineds in Writer::scanRelocations()
and symbols referenced by -u flags aren't referenced by any relocations,
this needs some manual code (similar to the entry point).

Differential Revision: https://reviews.llvm.org/D94371
2021-02-09 08:23:06 -05:00
Greg McGary 87104faac4 [lld-macho] Add ARM64 target arch
This is an initial base commit for ARM64 target arch support. I don't represent that it complete or bug-free, but wish to put it out for review now that some basic things like branch target & load/store address relocs are working.

I can add more tests to this base commit, or add them in follow-up commits.

It is not entirely clear whether I use the "ARM64" (Apple) or "AArch64" (non-Apple) naming convention. Guidance is appreciated.

Differential Revision: https://reviews.llvm.org/D88629
2021-02-08 18:14:07 -07:00
Jez Ng ac9dd247da [lld-macho] Try to make ubsan happy
Summary: We should avoid passing a null pointer to memcpy.
2021-02-08 14:51:36 -05:00
Jez Ng 5112035751 [lld-macho] Emit LSDA info in compact unwind
The LSDA pointers are encoded as offsets from the image base,
and arranged in one big contiguous array. Each second-level page records
the offset within that LSDA array which corresponds to the LSDA for its
first CU entry.

Reviewed By: clayborg

Differential Revision: https://reviews.llvm.org/D95810
2021-02-08 13:48:00 -05:00
Jez Ng 525bfa10ec [lld-macho] Emit personalities in compact unwind
Note that there is a triple indirection involved with
personalities and compact unwind:

1. Two bits of each CU encoding are used as an offset into the
   personality array.
2. Each entry of the personality array is an offset from the image base.
   The resulting address (after adding the image base) should point within the
   GOT.
3. The corresponding GOT entry contains the actual pointer to the
   personality function.

To further complicate things, when the personality function is in the
object file (as opposed to a dylib), its references in
`__compact_unwind` may refer to it via a section + offset relocation
instead of a symbol relocation. Since our GOT implementation can only
create entries for symbols, we have to create a synthetic symbol at the
given section offset.

Reviewed By: clayborg

Differential Revision: https://reviews.llvm.org/D95809
2021-02-08 13:47:59 -05:00
Greg McGary c3e4f3b231 [lld-macho] Fix alignment & layout to match ld64 and satisfy kernel & codesign
The Mach kernel & codesign on arm64 macOS has strict requirements for alignment and sequence of segments and sections. Dyld probably is just as picky, though kernel & codesign reject malformed Mach-O files before dyld ever has a chance.

I developed this diff by incrementally changing alignments & sequences to match the output of ld64. I stopped when my hello-world test program started working: `codesign --verify` succeded, and `execve(2)` didn't immediately fail with `errno == EBADMACHO` = `"Malformed Mach-O file"`.

Differential Revision: https://reviews.llvm.org/D94935
2021-02-05 17:22:03 -07:00
Jez Ng ea5b75de49 [lld-macho] Try to fix Windows build 2021-02-03 16:05:23 -05:00
Jez Ng 2d2e0000d3 [lld-macho] Rename VERSION CONTROL to VERSION TARGETING in helptext
Per https://reviews.llvm.org/D94938#inline-896740.
2021-02-03 13:43:47 -05:00
Jez Ng 4b2169fb6b [lld-macho] Remove stray ehFrame change
Per https://reviews.llvm.org/D95121#inline-897943.
2021-02-03 13:43:47 -05:00
Jez Ng f843bb82c0 [lld-macho] Force-loading should share code path with regular archive loads
This extends {D92539} to work even when we are loading archive
members via `-force_load`. I uncovered this issue while trying to
force-load archives containing bitcode -- we were segfaulting.

In addition to fixing the `-force_load` case, this diff also addresses
the behavior of `-ObjC` when LTO bitcode is involved -- we need to
force-load those archive members if they contain ObjC categories.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D95265
2021-02-03 13:43:47 -05:00
Jez Ng 163dcd8513 [lld-macho] Associate each Symbol with an InputFile
This makes our error messages more informative. But the bigger motivation is for
LTO symbol resolution, which will be in an upcoming diff. The changes in this
one are largely mechanical.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D94316
2021-02-03 13:43:47 -05:00
Greg McGary 3a9d2f1488 [lld-macho][NFC] refactor relocation handling
Add per-reloc-type attribute bits and migrate code from per-target file into target independent code, driven by reloc attributes.

Many cleanups

Differential Revision: https://reviews.llvm.org/D95121
2021-02-02 10:54:53 -07:00
Greg McGary 0ef25cf558 [lld-macho][NFC] Add new option group for versions
Coalesce all version control options into a group

Differential Revision: https://reviews.llvm.org/D94938
2021-01-29 22:27:41 -07:00
Jez Ng 3dd5ea9dd8 [lld-macho] Link against ObjCARCOpts instead of ObjCARC
Not sure what the difference is, but using the latter appears to cause
issues in standalone builds. See llvm.org/PR48853.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D95359
2021-01-25 19:01:48 -05:00
Sam Clegg 299b0e5ee9 [lld] Consistent help text for `--save-temps`
I noticed that this option was not appearing at all in the `--help`
messages for `wasm-ld` or `ld.lld`.

Add help text and make it consistent across all ports.

Differential Revision: https://reviews.llvm.org/D94925
2021-01-25 10:27:18 -08:00
Jez Ng 041f3ee664 [lld-macho] Ignore -lto_library
Just getting rid of some logspew as I test LLD under existing build
systems.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D95213
2021-01-22 16:48:50 -05:00
Jez Ng 34e8fcf63f [lld-macho] Add dependency on ObjCARC to fix shared build 2021-01-20 20:41:51 -05:00
Jez Ng 697f4e429b [lld-macho] Run ObjCContractPass during LTO
Run the ObjCARCContractPass during LTO. The legacy LTO backend (under
LTO/ThinLTOCodeGenerator.cpp) already does this; this diff just adds that
behavior to the new LTO backend. Without that pass, the objc.clang.arc.use
intrinsic will get passed to the instruction selector, which doesn't know how to
handle it.

In order to test both the new and old pass managers, I've also added support for
the `--[no-]lto-legacy-pass-manager` flags.

P.S. Not sure if the ordering of the pass within the pipeline matters...

Reviewed By: fhahn

Differential Revision: https://reviews.llvm.org/D94547
2021-01-20 14:21:32 -05:00
Jez Ng b3e73dc5af [lld-macho][easy] Create group for LLD-specific CLI flags
Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D94545
2021-01-20 14:21:31 -05:00
Kazu Hirata fb98a1be43 Fix the warnings on unused variables (NFC) 2021-01-13 13:32:40 -08:00
Nico Weber 47991a15d1 [lld/mac] llvm style fix: no else after return 2021-01-10 09:35:00 -05:00
Nico Weber 1198478c42 [lld/mac] remove redundant null check
This is already checked two lines up. No behavior change.
2021-01-09 21:18:32 -05:00
Jez Ng e98b441a09 [lld-macho] Remove unnecessary llvm:: namespace prefixes 2021-01-09 12:44:35 -05:00
Jez Ng daaaed6bb8 [lld-macho] Fix TLV data initialization
We were mishandling the case where both `__tbss` and `__thread_data` sections were
present.

TLVP relocations should be encoded as offsets from the start of `__thread_data`,
even if the symbol is actually located in `__thread_bss`. Previously, we were
writing the offset from the start of the containing section, which doesn't
really make sense since there's no way `tlv_get_addr()` can know which section a
given `tlv$init` symbol is in at runtime.

In addition, this patch ensures that we place `__thread_data` immediately before
`__thread_bss`. This is what ld64 does, likely for performance reasons. Zerofill
sections must also be at the end of their segments; we were already doing this,
but now we ensure that `__thread_bss` occurs before `__bss`, so that it's always
possible to have it contiguous with `__thread_data`.

Fixes llvm.org/PR48657.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D94329
2021-01-08 18:48:12 -05:00
Fangrui Song 7916fd71e9 [lld-macho] Fix GCC -Wunused-variable in -DLLVM_ENABLE_ASSERTIONS=off build 2021-01-06 10:58:46 -08:00
Nico Weber 568824798f fix typo to cycle bots 2021-01-01 22:28:11 -05:00
Thorsten Schütt 4a290a5905 [lld/mac] fix typo 2020-12-31 08:23:29 +01:00
Nico Weber 8886be242d [lld/mac] Add -adhoc_codesign / -no_adhoc_codesign flags
These are new in Xcode 12's ld64. lld never codesigns at the moment, so
-no_adhoc_codesign doesn't even have to warn that it's not implemented.
2020-12-30 20:57:25 -05:00
Jez Ng 9d1140e18e [lld-macho] Simulator & DriverKit executables should always be PIE
We didn't have support for parsing DriverKit in our `-platform`
flag, so add that too. Also remove a bunch of unnecessary namespace
prefixes.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D93741
2020-12-23 11:24:12 -05:00
Nico Weber 77fb45e59e [lld/mac] Add --version flag
It's an extension to ld64, but all the other ports have it, and
someone asked for it in PR43721.

While here, change the COFF help text to match the other ports.

Differential Revision: https://reviews.llvm.org/D93491
2020-12-22 22:06:39 -05:00
Nico Weber 57ffbe020a glld/mac] Don't add names of unreferenced symbols to string table
Before this, a hello world program would contain many many unnecessary
entries in its string table.

No behavior change, just makes the string table in the output smaller
and more like ld64's.

Differential Revision: https://reviews.llvm.org/D93711
2020-12-22 15:52:33 -05:00
Nico Weber 13f439a187 [lld/mac] Implement support for private extern symbols
Private extern symbols are used for things scoped to the linkage unit.
They cause duplicate symbol errors (so they're in the symbol table,
unlike TU-scoped truly local symbols), but they don't make it into the
export trie. They are created e.g. by compiling with
-fvisibility=hidden.

If two weak symbols have differing privateness, the combined symbol is
non-private external. (Example: inline functions and some TUs that
include the header defining it were built with
-fvisibility-inlines-hidden and some weren't).

A weak private external symbol implicitly has its "weak" dropped and
behaves like a regular strong private external symbol: Weak is an export
trie concept, and private symbols are not in the export trie.

If a weak and a strong symbol have different privateness, the strong
symbol wins.

If two common symbols have differing privateness, the larger symbol
wins. If they have the same size, the privateness of the symbol seen
later during the link wins (!) -- this is a bit lame, but it matches
ld64 and this behavior takes 2 lines less to implement than the less
surprising "result is non-private external), so match ld64.
(Example: `int a` in two .c files, both built with -fcommon,
one built with -fvisibility=hidden and one without.)

This also makes `__dyld_private` a true TU-local symbol, matching ld64.
To make this work, make the `const char*` StringRefZ ctor to correctly
set `size` (without this, writing the string table crashed when calling
getName() on the __dyld_private symbol).

Mention in CommonSymbol's comment that common symbols are now disabled
by default in clang.

Mention in -keep_private_externs's HelpText that the flag only has an
effect with `-r` (which we don't implement yet -- so this patch here
doesn't regress any behavior around -r + -keep_private_externs)). ld64
doesn't explicitly document it, but the commit text of
http://reviews.llvm.org/rL216146 does, and ld64's
OutputFile::buildSymbolTable() checks `_options.outputKind() ==
Options::kObjectFile` before calling `_options.keepPrivateExterns()`
(the only reference to that function).

Fixes PR48536.

Differential Revision: https://reviews.llvm.org/D93609
2020-12-21 21:23:33 -05:00
Fangrui Song 791fe7ac57 [lld-macho] Fix memcpy ub after D93267 2020-12-20 20:01:20 -08:00
Jez Ng 64e4757200 [lld-macho] Have order files support filtering by archive member paths
Also remove iteration over ArchiveFile symbols in buildInputSectionPriorities --
that was rendered unnecessary after D92539, which included ObjFiles from
ArchiveFiles inside the `inputFiles` vector.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D93569
2020-12-20 13:49:18 -05:00
Jez Ng 5f9896d3b2 [lld-macho] Support Obj-C symbols in order files
Obj-C symbols may have spaces and colons, which our previous order file
parser would be confused by. The order file format has made the very unfortunate
choice of using colons for its delimiters, which means that we have to use
heuristics to determine if a given colon is part of a symbol or not...

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D93567
2020-12-20 13:49:18 -05:00
Greg McGary 99930719c6 Handle overflow beyond the 127 common encodings limit
The common encodings table holds only 127 entries. The encodings index for compact entries is 8 bits wide, and indexes 127..255 are stored locally to each second-level page. Prior to this diff, lld would `fatal()` if encodings overflowed the 127 limit.

This diff populates a per-second-level-page encodings table as needed. When the per-page encodings table hits its limit, we must terminate the page. If such early termination would consume fewer entries than a regular (non-compact) encoding page, then we prefer the regular format.

Caveat: one reason the common-encoding table might overflow is because of DWARF debug-info references, which are not yet implemented and will come with a later diff.

Differential Revision: https://reviews.llvm.org/D93267
2020-12-19 14:54:37 -08:00
Greg McGary d4ec3346b1 [lld-macho][nfc] Refactor to accommodate paired relocs
This is a refactor to pave the way for supporting paired-ADDEND for ARM64. The only paired reloc type for X86_64 is SUBTRACTOR. In a later diff, I will add SUBTRACTOR for both X86_64 and ARM64.

* s/`getImplicitAddend`/`getAddend`/ because it handles all forms of addend: implicit, explicit, paired.
* add predicate `bool isPairedReloc()`
* check range of `relInfo.r_symbolnum` is internal, unrelated to user-input, so use `assert()`, not `error()`
* minor cleanups & rearrangements in `InputFile::parseRelocations()`

Differential Revision: https://reviews.llvm.org/D90614
2020-12-17 20:21:41 -08:00
Greg McGary cc1cf6332a [lld-macho] Implement option: -undefined TREATMENT
TREATMENT can be `error`, `warning`, `suppress`, or `dynamic_lookup`
The `dymanic_lookup` remains unimplemented for now.

Differential Revision: https://reviews.llvm.org/D93263
2020-12-17 17:40:50 -08:00
Nico Weber f710bb7063 lld: Replace some lld::outs()s with message()
No behavior change.
2020-12-17 16:19:09 -05:00
Jez Ng 4c8276cdc1 [lld-macho] Use LC_LOAD_WEAK_DYLIB for dylibs with only weakrefs
Note that dylibs without *any* refs will still be loaded in the usual
(strong) fashion.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D93435
2020-12-17 08:49:17 -05:00
Jez Ng 811444d7a1 [lld-macho] Add support for weak references
Weak references need not necessarily be satisfied at runtime (but they must
still be satisfied at link time). So symbol resolution still works as per usual,
but we now pass around a flag -- ultimately emitting it in the bind table -- to
indicate if a given dylib symbol is a weak reference.

ld64's behavior for symbols that have both weak and strong references is
a bit bizarre. For non-function symbols, it will emit a weak import. For
function symbols (those referenced by BRANCH relocs), it will emit a
regular import. I'm not sure what value there is in that behavior, and
since emulating it will make our implementation more complex, I've
decided to treat regular weakrefs like function symbol ones for now.

Fixes PR48511.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D93369
2020-12-17 08:49:16 -05:00
dfukalov 9ed8e0caab [NFC] Reduce include files dependency and AA header cleanup (part 2).
Continuing work started in https://reviews.llvm.org/D92489:

Removed a bunch of includes from "AliasAnalysis.h" and "LoopPassManager.h".

Reviewed By: RKSimon

Differential Revision: https://reviews.llvm.org/D92852
2020-12-17 14:04:48 +03:00
Nico Weber ec88746a05 [lld/mac] fill in current and compatibility version for LC_LOAD_(WEAK_)DYLIB
Not sure if anything actually depends on this, but it makes `otool -L`
output look nicer.

Differential Revision: https://reviews.llvm.org/D93332
2020-12-15 19:34:59 -05:00
Nico Weber 09edd9df6e [mac/lld] simplify code using PackedVersion instead of VersionTuple
PackedVersion already does the correct range checks.

No behavior change.

Differential Revision: https://reviews.llvm.org/D93338
2020-12-15 19:23:07 -05:00
Jez Ng 3aa8e071dd [lld-macho] Add implicit dylib support for frameworks
{D93000} applied to frameworks. Partial fix for PR48511.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D93277
2020-12-15 15:58:26 -05:00
Jez Ng 8a5e068823 [lld-macho] Support -sub_umbrella
From what I can tell, it's essentially identical to
`-sub_library`, but it doesn't match files ending in ".dylib".

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D93276
2020-12-15 15:58:26 -05:00
Jez Ng 3184519909 [lld-macho] Don't emit rebase opcodes for relocs in TLV sections
Their addresses are already encoded as section-relative offsets, so
there's no need to rebase them at runtime. {D85080} has some context
on the weirdness of TLV sections.

Fixes llvm.org/PR48491.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D93257
2020-12-15 15:58:26 -05:00
Jez Ng 544148ae70 [lld-macho] -weak_{library,framework} should always take priority
We were not setting forceWeakImport for file paths given by
`-weak_library` if we had already loaded the file. This diff fixes that
by having `loadDylib` return a cached DylibFile instance even if we have
already loaded that file.

We still avoid emitting multiple LC_LOAD_DYLIBs, but we achieve this by
making inputFiles a SetVector instead of relying on the `loadedDylibs`
cache.

Reviewed By: #lld-macho, smeenai

Differential Revision: https://reviews.llvm.org/D93255
2020-12-15 15:58:26 -05:00
Nico Weber 601f0fb846 [lld/mac] Set ordinal on dynamic undefined symbols in symbol table
This lets `nm -m` print "(from libfoo)" in its output, which is more
accessible than dumping the bind table.

See https://reviews.llvm.org/D57190#2455761 for the somewhat
surprising `AltEntry` that appears in symtab.s.

Differential Revision: https://reviews.llvm.org/D93318
2020-12-15 14:39:36 -05:00
Nico Weber d058b69b1c [lld/mac] implement -compatibility_version, -current_version
Differential Revision: https://reviews.llvm.org/D93237
2020-12-14 18:41:36 -05:00
Jez Ng 76c36c11a9 [lld-macho] Don't load dylibs more than once
Also remove `DylibFile::reexported` since it's unused.

Fixes llvm.org/PR48393.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D93001
2020-12-10 15:57:52 -08:00
Jez Ng 6a348f6158 [lld-macho] Implement `-no_implicit_dylibs`
Dylibs that are "public" -- i.e. top-level system libraries -- are considered
implicitly linked when another library re-exports them. That is, we should load
them & bind directly to their symbols instead of via their re-exporting
umbrella library. This diff implements that behavior by default, as well as an
opt-out flag.

In theory, this is just a performance optimization, but in practice it seems
that it's needed for correctness.

Fixes llvm.org/PR48395.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D93000
2020-12-10 15:57:52 -08:00
Jez Ng 74d799926e [lld-macho] Initialize AsmParsers earlier
We need to initialize AsmParsers before any calls to `addFile`, as
bitcode files may require them. Otherwise we trigger `Assertion T &&
T->hasMCAsmParser()' failed`.

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D92913
2020-12-10 15:57:52 -08:00
Jez Ng 29d3b0e471 [lld-macho] Add support for -mcpu, -mattr, -code-model in LTO
`-mcpu` and `-code-model` tests were copied from similar ones in
LLD-ELF.

There doesn't seem to be an equivalent test for `-mattr` in LLD-ELF, so
I've verified our behavior by cribbing a test from
CodeGen/X86/recip-fastmath.ll.

Reviewed By: #lld-macho, compnerd, MaskRay

Differential Revision: https://reviews.llvm.org/D92912
2020-12-10 15:57:51 -08:00
Jez Ng 863f7a745e [lld-macho] Don't attempt to emit rebase opcodes for debug sections
This was causing a crash as we were attempting to look up the
nonexistent parent OutputSection of the debug sections. We didn't detect
it earlier because there was no test for PIEs with debug info (PIEs
require us to emit rebases for X86_64_RELOC_UNSIGNED).

This diff filters out the debug sections while loading the ObjFiles. In
addition to fixing the above problem, it also lets us avoid doing
redundant work -- we no longer parse / apply relocations / attempt to
emit dyld opcodes for these sections that we don't emit.

Fixes llvm.org/PR48392.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D92904
2020-12-10 15:57:51 -08:00
Jez Ng 95831a56d0 [lld-macho] Implement -object_path_lto
This makes it possible for STABS entries to reference the debug info
contained in the LTO-compiled output.

I'm not sure how to test the file mtime within llvm-lit -- GNU and BSD
`stat` take different command-line arguments. I've omitted the check for
now.

Reviewed By: clayborg

Differential Revision: https://reviews.llvm.org/D92537
2020-12-10 15:57:51 -08:00
Nico Weber 9d6177c2a5 [lld/mac] Use xxhash instead of MD5 for computing the UUID
15% faster for linking Chromium's base_unittests.txt, according to ministat:

```
    N           Min           Max        Median           Avg        Stddev
x  10      0.650213    0.69287586    0.65793395    0.66127126   0.012365407
+  10    0.54993701    0.59006906    0.55885506    0.56146643   0.013215349
Difference at 95.0% confidence
        -0.0998048 +/- 0.0120244
        -15.0929% +/- 1.81838%
        (Student's t, pooled s = 0.0127974)
```

And matches what we do on the other ports.

Differential Revision: https://reviews.llvm.org/D92736
2020-12-09 21:06:17 -05:00
Jez Ng 78976bf3da [lld-macho] Support parsing of bitcode within archives
Also error out if we find anything other than an object or bitcode file
in the archive.

Note that we were previously inserting the symbols and sections of the
unpacked ObjFile into the containing ArchiveFile. This was actually
unnecessary -- we can just insert the ObjectFile (or BitcodeFile) into
the `inputFiles` vector. This is the approach taken by LLD-ELF.

Reviewed By: thakis

Differential Revision: https://reviews.llvm.org/D92539
2020-12-08 10:34:32 -08:00
Jez Ng 7b007ac080 [lld-macho][nfc] Move some methods from InputFile to ObjFile
Additionally:
1. Move the helper functions in InputSection.h below the definition of
   `InputSection`, so the important stuff is on top
2. Remove unnecessary `explicit`

Reviewed By: #lld-macho, compnerd

Differential Revision: https://reviews.llvm.org/D92453
2020-12-08 10:34:32 -08:00
Nico Weber feadc3798d [lld/mac] Make X86_64::getImplicitAddend not do heap allocations
Speeds up linking Chromium's base_unittests almost 10%. According to ministat:

    N           Min           Max        Median           Avg        Stddev
x   5    0.72193289    0.73073196    0.72560811    0.72565799  0.0032265649
+   5    0.64069581    0.67173195    0.65876389    0.65796089   0.011349451
Difference at 95.0% confidence
	-0.0676971 +/- 0.0121682
	-9.32906% +/- 1.67685%
	(Student's t, pooled s = 0.00834328)

Differential Revision: https://reviews.llvm.org/D92734
2020-12-07 09:23:51 -05:00
Fangrui Song 4701cb41ed [lld] Delete unused declarations
Notes:

* runMSVCLinker: remnant of r338615
* wasm markSymbol: remnant of r374275
* wasm addDataAddressGlobal: accidentally added by r372779
* MachO Writer::createSymtabContents: accidentally added by D76839
2020-12-06 15:26:37 -08:00
Nico Weber 16b1f6e385 [mac/lld] Add support for the LC_LINKER_OPTION load command in o files
clang puts `-framework CoreFoundation` in this load command for files
that use @available / __builtin_available. Without support for this,
binaries that don't explicitly link to CoreFoundation fail to link.

Differential Revision: https://reviews.llvm.org/D92624
2020-12-04 08:46:53 -05:00
Nico Weber 7cb0a373d1 [mac/lld] Implement -t
Goes well with `-why_load` to get an idea of load order.

Differential Revision: https://reviews.llvm.org/D92583
2020-12-03 16:02:38 -05:00
Nico Weber 3422f3cc6e Reland "[mac/lld] Implement -why_load".
The problem was that `sym` became replaced in the call
to make<ObjFile> and referring to it afer that read memory that now
stored a different kind of symbol (a Defined instead of a LazySymbol).
Since this happens only once per archive, just copy the symbol to the
stack before make<ObjFile> and read the copy instead.

Originally reviewed at https://reviews.llvm.org/D92496
2020-12-03 08:35:12 -05:00
Nico Weber ea0029f55d Revert "[mac/lld] Implement -why_load"
This reverts commit 542d3b609d.
Seems to break check-lld. Reverting while I take a look.
2020-12-02 18:57:46 -05:00
Nico Weber 542d3b609d [mac/lld] Implement -why_load
This is useful for debugging why lld loads .o files it shouldn't load.
It's also useful for users of lld -- I've used ld64's version of this a
few times.

Differential Revision: https://reviews.llvm.org/D92496
2020-12-02 18:33:12 -05:00
Nico Weber ca634393fc [mac/lld] Make --reproduce work with thin archives
See http://reviews.llvm.org/rL268229 and
http://reviews.llvm.org/rL313832 which did the same for the ELF port.

Differential Revision: https://reviews.llvm.org/D92456
2020-12-02 09:48:31 -05:00
Nico Weber b2f00f24a3 [mac/lld] Include archive name in diagnostics
Also, for .o files, include full path as given on link command line.

Before:
    lld: error: undefined symbol [...], referenced from sandbox_logging.o

After:
    lld: error: undefined symbol [...], referenced from libseatbelt.a(sandbox_logging.o)

Move archiveName up to InputFile so we can consistently use toString()
to print InputFiles in diags, and pass it to the ObjFile ctor. This
matches the ELF and COFF ports.

Differential Revision: https://reviews.llvm.org/D92437
2020-12-01 23:00:25 -05:00
Heejin Ahn 6fb88c6cd5 [lld-macho] Add dependency to DebugInfoDWARF
Without this `-DBUILD_SHARED_LIBS=ON` doesn't work.
2020-12-01 19:10:46 -08:00
Nico Weber 126f58e838 fix typos to cycle bots 2020-12-01 20:27:33 -05:00
Nico Weber 07ab597bb0 [lld/mac] Fix issues around thin archives
- most importantly, fix a use-after-free when using thin archives,
  by putting the archive unique_ptr to the arena allocator. This
  ports D65565 to MachO

- correctly demangle symbol namess from archives in diagnostics

- add a test for thin archives -- it finds this UaF, but only when
  running it under asan (it also finds the demangling fix)

- make forceLoadArchive() use addFile() with a bool to have the archive
  loading code in fewer places. no behavior change; matches COFF port a
  bit better

Differential Revision: https://reviews.llvm.org/D92360
2020-12-01 18:48:29 -05:00
Jez Ng c7dbaec396 [lld-macho] Add isCodeSection()
This is the same logic that ld64 uses to determine which sections
contain functions. This was added so that we could determine which
STABS entries should be N_FUN.

Reviewed By: clayborg

Differential Revision: https://reviews.llvm.org/D92430
2020-12-01 15:05:21 -08:00
Jez Ng 78f6498cdc [lld-macho] Flesh out STABS implementation
This addresses a lot of the comments in {D89257}. Ideally it'd have been
done in the same diff, but the commits in between make that difficult.

This diff implements:
* N_GSYM and N_STSYM, the STABS for global and static symbols
* Has the STABS reflect the section IDs of their referent symbols
* Ensures we don't fail when encountering absolute symbols or files with
  no debug info
* Sorts STABS symbols by file to minimize the number of N_OSO entries

Reviewed By: clayborg

Differential Revision: https://reviews.llvm.org/D92366
2020-12-01 15:05:21 -08:00
Jez Ng b768d57b36 [lld-macho] Add archive name and file modtime to STABS output
We should also set the modtime when running LTO. That will be done in a
future diff, together with support for the `-object_path_lto` flag.

Reviewed By: clayborg

Differential Revision: https://reviews.llvm.org/D91318
2020-12-01 15:05:21 -08:00